gen_ai_hub.proxy.core package

gen_ai_hub.proxy.core.set_proxy_version(proxy_version: str, catalog: ProxyClients | None = None) → None

Set the global proxy version.

Parameters:

• proxy_version (str) – The proxy version to set.

• catalog (Optional[ProxyClients], optional) – The catalog for which the proxy version is to be set. If none is provided, the proxy version is set for the default proxy_clients catalog.

Raises:

ValueError – If proxy_version is not a string.

gen_ai_hub.proxy.core.get_proxy_version(catalog: ProxyClients | None = None) → str

Get the current proxy version. The version is selected in the following order:

• thread-local overwrite (set with proxy_version_context)

• global overwrite (set with set_proxy_version)

• environment variable (LLM_PROXY_VERSION)

• first registered proxy version

Parameters:

catalog (Optional[ProxyClients], optional) – The catalog from which to get the proxy version. If none is provided,

Raises:

ValueError – If no proxy version is set.

Returns:

The current proxy version.

Return type:

str

gen_ai_hub.proxy.core.proxy_version_context(proxy_version: str, catalog: ProxyClients | None = None) → None

Context manager to set a thread-local proxy version.

Parameters:

• proxy_version (str) – The proxy version to set.

• catalog (Optional[ProxyClients], optional) – The catalog for which the proxy version is to be set. If none is provided, the proxy version is set for the default proxy_clients catalog.

Raises:

ValueError – If proxy_version is not a string.

gen_ai_hub.proxy.core.get_proxy_client(proxy_version: str | None = None, catalog: ProxyClients | None = None, **kwargs) → BaseProxyClient

Get a proxy client for the given proxy version.

Parameters:

• proxy_version (str | None, optional) – The version of the proxy client to retrieve. If not provided, the function will attempt to retrieve the version using the get_proxy_version function.

• catalog (Optional[ProxyClients], optional) – The catalog from which to retrieve the proxy client. If not provided, the function will default to the proxy_clients catalog

• kwargs (dict) – Arbitrary keyword arguments that will be passed to the constructor of the proxy client class.

Returns:

An instance of the proxy client.

Return type:

BaseProxyClient

Submodules

• gen_ai_hub.proxy.core.base module
  • BaseDeployment
    • BaseDeployment.model_config
    • BaseDeployment.additional_request_body_kwargs()
    • BaseDeployment.prediction_url
    • BaseDeployment.get_model_identification_kwargs()
    • BaseDeployment.get_main_model_identification_kwargs()
  • InstanceCacheMeta
    • InstanceCacheMeta.clear_cache()
  • CombinedMeta
  • BaseProxyClient
    • BaseProxyClient.model_config
    • BaseProxyClient.refresh_instance_cache()
    • BaseProxyClient.request_header
    • BaseProxyClient.deployments
    • BaseProxyClient.deployment_class
    • BaseProxyClient.select_deployment()
• gen_ai_hub.proxy.core.proxy_clients module
  • ProxyClients
    • ProxyClients.__init__()
    • ProxyClients.register()
    • ProxyClients.get_proxy_cls()
    • ProxyClients.get_proxy_cls_name()
  • proxy_version_context()
  • set_proxy_version()
  • get_proxy_version()
  • get_proxy_client()
• gen_ai_hub.proxy.core.utils module
  • lru_cache_extended()
  • if_set()
  • if_str_set()
  • kwargs_if_set()
  • warn_once()