gen_ai_hub.proxy.langchain.openai module¶

LangChain wrappers for OpenAI models via Generative AI Hub.

gen_ai_hub.proxy.langchain.openai.get_client_params(values)¶: Get the client parameters. :param values: The client values :return: client values + proxy_client

Bases: BaseAuth

Base class for OpenAI models using a proxy.

Parameters:: BaseAuth (class) – Base authentication class
Returns:: The ProxyOpenAI class
Return type:: class

model_config: ClassVar[ConfigDict] = {'extra': 'allow'}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_clients(values: Dict) → Dict¶

Validate and initialize OpenAI clients.

Parameters:: values (Dict) – The input values
Returns:: The validated values
Return type:: Dict

deployment_id: str | None¶

config_name: str | None¶

config_id: str | None¶

proxy_model_name: str | None¶

class gen_ai_hub.proxy.langchain.openai.ChatOpenAI(*args, name: str | None = None, cache: ~langchain_core.caches.BaseCache | bool | None = None, verbose: bool = <factory>, callbacks: list[~langchain_core.callbacks.base.BaseCallbackHandler] | ~langchain_core.callbacks.base.BaseCallbackManager | None = None, tags: list[str] | None = None, metadata: dict[str, ~typing.Any] | None = None, custom_get_token_ids: ~collections.abc.Callable[[str], list[int]] | None = None, rate_limiter: ~langchain_core.rate_limiters.BaseRateLimiter | None = None, disable_streaming: bool | ~typing.Literal['tool_calling'] = False, output_version: str | None = <factory>, profile: ~langchain_core.language_models.model_profile.ModelProfile | None = None, client: ~typing.Any = None, async_client: ~typing.Any = None, root_client: ~typing.Any = None, root_async_client: ~typing.Any = None, model_name: str | None = None, temperature: float | None = None, model_kwargs: dict[str, ~typing.Any] = <factory>, api_key: ~pydantic.types.SecretStr | None | ~collections.abc.Callable[[], str] | ~collections.abc.Callable[[], ~collections.abc.Awaitable[str]] = None, base_url: str | None = None, organization: str | None = None, openai_proxy: str | None = <factory>, timeout: float | tuple[float, float] | ~typing.Any | None = None, stream_usage: bool | None = None, max_retries: int | None = None, presence_penalty: float | None = None, frequency_penalty: float | None = None, seed: int | None = None, logprobs: bool | None = None, top_logprobs: int | None = None, logit_bias: dict[int, int] | None = None, streaming: bool = False, n: int | None = None, top_p: float | None = None, max_completion_tokens: int | None = None, reasoning_effort: str | None = None, reasoning: dict[str, ~typing.Any] | None = None, verbosity: str | None = None, tiktoken_model_name: str | None = None, default_headers: ~collections.abc.Mapping[str, str] | None = None, default_query: ~collections.abc.Mapping[str, object] | None = None, http_client: ~typing.Any | None = None, http_async_client: ~typing.Any | None = None, http_socket_options: ~collections.abc.Sequence[tuple[int, int, int]] | None = None, stream_chunk_timeout: float | None = <factory>, stop_sequences: list[str] | str | None = None, extra_body: ~collections.abc.Mapping[str, ~typing.Any] | None = None, include_response_headers: bool = False, disabled_params: dict[str, ~typing.Any] | None = None, context_management: list[dict[str, ~typing.Any]] | None = None, include: list[str] | None = None, prompt_cache_options: dict[str, ~typing.Any] | None = None, service_tier: str | None = None, store: bool | None = None, truncation: str | None = None, use_previous_response_id: bool = False, use_responses_api: bool | None = None, proxy_client: ~typing.Any | None = None, deployment_id: str | None = None, config_name: str | None = None, config_id: str | None = None, proxy_model_name: str | None = None, api_version: str | None = None, **kwargs)¶

Bases: ProxyOpenAI, ChatOpenAI

ChatOpenAI model using a proxy.

Parameters:

ProxyOpenAI (class) – Base class for OpenAI models using a proxy
ChatOpenAI (class) – ChatOpenAI class from langchain_openai

model_name: str | None¶: Model name to use.

openai_api_version: str | None¶

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'populate_by_name': True, 'protected_namespaces': (), 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, **kwargs)¶: Initialize the ChatOpenAI object.

classmethod validate_environment(values: Dict) → Dict¶

Validates the environment.

Parameters:: values (Dict) – The input values
Raises:: ValueError – n must be at least 1.
Returns:: The validated values
Return type:: Dict

deployment_id: str | None¶

config_name: str | None¶

config_id: str | None¶

proxy_model_name: str | None¶

max_tokens: int | None¶: Maximum number of tokens to generate.

client: Any¶

async_client: Any¶

root_client: Any¶

root_async_client: Any¶

temperature: float | None¶: What sampling temperature to use.

model_kwargs: dict[str, Any]¶: Holds any model parameters valid for create call not explicitly specified.

openai_api_key: SecretStr | None | Callable[[], str] | Callable[[], Awaitable[str]]¶

API key to use.

Can be inferred from the OPENAI_API_KEY environment variable, or specified as a string, or sync or async callable that returns a string.

??? example “Specify with environment variable”

`bash export OPENAI_API_KEY=... ` ```python from langchain_openai import ChatOpenAI

model = ChatOpenAI(model=”gpt-5-nano”) ```

??? example “Specify with a string”

```python from langchain_openai import ChatOpenAI

model = ChatOpenAI(model=”gpt-5-nano”, api_key=”…”) ```

??? example “Specify with a sync callable”

```python from langchain_openai import ChatOpenAI

def get_api_key() -> str:
# Custom logic to retrieve API key return “…”

model = ChatOpenAI(model=”gpt-5-nano”, api_key=get_api_key) ```

??? example “Specify with an async callable”

```python from langchain_openai import ChatOpenAI

async def get_api_key() -> str:
# Custom async logic to retrieve API key return “…”

model = ChatOpenAI(model=”gpt-5-nano”, api_key=get_api_key) ```

openai_api_base: str | None¶

Base URL path for API requests, leave blank if not using a proxy or service emulator.

Resolution order (first match wins):

Explicit base_url (or openai_api_base) kwarg.
Env var OPENAI_API_BASE (read by LangChain at init).
Env var OPENAI_BASE_URL (read by the underlying openai SDK client).

OPENAI_BASE_URL is also inspected by LangChain only to decide whether to default-enable stream_usage — when set, the default is left off because many non-OpenAI endpoints do not support streaming token usage.

openai_organization: str | None¶: Automatically inferred from env var OPENAI_ORG_ID if not provided.

openai_proxy: str | None¶

request_timeout: float | tuple[float, float] | Any | None¶

Timeout for requests to OpenAI completion API.

Can be float, httpx.Timeout or None.

stream_usage: bool | None¶

Whether to include usage metadata in streaming output.

If enabled, an additional message chunk will be generated during the stream including usage metadata.

This parameter is enabled unless openai_api_base is set or the model is initialized with a custom client, as many chat completions APIs do not support streaming token usage.

!!! version-added “Added in langchain-openai 0.3.9”

!!! warning “Behavior changed in langchain-openai 0.3.35”

Enabled for default base URL and client.

max_retries: int | None¶: Maximum number of retries to make when generating.

presence_penalty: float | None¶: Penalizes repeated tokens.

frequency_penalty: float | None¶: Penalizes repeated tokens according to frequency.

seed: int | None¶: Seed for generation

logprobs: bool | None¶: Whether to return logprobs.

top_logprobs: int | None¶

Number of most likely tokens to return at each token position, each with an associated log probability.

logprobs must be set to true if this parameter is used.

logit_bias: dict[int, int] | None¶: Modify the likelihood of specified tokens appearing in the completion.

streaming: bool¶: Whether to stream the results or not.

n: int | None¶: Number of chat completions to generate for each prompt.

top_p: float | None¶: Total probability mass of tokens to consider at each step.

reasoning_effort: str | None¶

Constrains effort on reasoning for reasoning models.

For use with the Chat Completions API. Reasoning models only.

Currently supported values are ‘minimal’, ‘low’, ‘medium’, and ‘high’. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.

reasoning: dict[str, Any] | None¶

Reasoning parameters for reasoning models. None disables reasoning.

For use with the Responses API.

```python reasoning={

“effort”: None, # Default None; can be “low”, “medium”, or “high” “summary”: “auto”, # Can be “auto”, “concise”, or “detailed”

}¶

!!! version-added “Added in langchain-openai 0.3.24”

verbosity: str | None¶

Controls the verbosity level of responses for reasoning models.

For use with the Responses API.

Currently supported values are ‘low’, ‘medium’, and ‘high’.

!!! version-added “Added in langchain-openai 0.3.28”

tiktoken_model_name: str | None¶

The model name to pass to tiktoken when using this class.

Tiktoken is used to count the number of tokens in documents to constrain them to be under a certain limit.

By default, when set to None, this will be the same as the embedding model name. However, there are some cases where you may want to use this Embedding class with a model name not supported by tiktoken. This can include when using Azure embeddings or when using one of the many model providers that expose an OpenAI-like API but with different models. In those cases, in order to avoid erroring when tiktoken is called, you can specify a model name to use here.

default_headers: Mapping[str, str] | None¶

default_query: Mapping[str, object] | None¶

http_client: Any | None¶

Optional httpx.Client.

Only used for sync invocations. Must specify http_async_client as well if you’d like a custom client for async invocations.

http_async_client: Any | None¶

Optional httpx.AsyncClient.

Only used for async invocations. Must specify http_client as well if you’d like a custom client for sync invocations.

http_socket_options: Sequence[tuple[int, int, int]] | None¶

TCP socket options applied to the httpx transports built by this instance.

Defaults to a conservative TCP-keepalive + TCP_USER_TIMEOUT profile that targets a ~2-minute bound on silent connection hangs (silent mid-stream peer loss, gVisor/NAT idle timeouts, silent TCP black holes) on platforms that support the full option set. On platforms that only support a subset (macOS without TCP_USER_TIMEOUT, Windows with only SO_KEEPALIVE, minimal kernels), unsupported options are silently dropped and the bound degrades to whatever the remaining options + OS defaults provide — still better than indefinite hang.

Accepted values:

None (default): use env-driven defaults. Matches the “unset” convention
used by http_client elsewhere on this class.
() (empty): disable socket-option injection entirely. Inherits the OS
defaults and restores httpx’s native env-proxy auto-detection.
A non-empty sequence of (level, option, value) tuples: explicit
override; passed verbatim to the transport (not filtered). Unsupported options raise OSError at connect time rather than being silently dropped — the user chose them explicitly.

Environment variables (only consulted when this field is None): LANGCHAIN_OPENAI_TCP_KEEPALIVE (set to 0 to disable entirely — the kill-switch), LANGCHAIN_OPENAI_TCP_KEEPIDLE, LANGCHAIN_OPENAI_TCP_KEEPINTVL, LANGCHAIN_OPENAI_TCP_KEEPCNT, LANGCHAIN_OPENAI_TCP_USER_TIMEOUT_MS.

Applied per side: if http_client is supplied, the sync path uses that user-owned client’s socket options as-is; the async path still gets http_socket_options applied to its default builder (and vice-versa for http_async_client). Supply both to take full control.

!!! note “Interaction with env-proxy auto-detection”

When a custom httpx transport is active, httpx disables its native env-proxy auto-detection (HTTP_PROXY / HTTPS_PROXY / ALL_PROXY / NO_PROXY and macOS/Windows system proxy settings).

To keep the default shape safe, ChatOpenAI detects the “proxy-env-shadow” pattern and skips the custom transport entirely when all of the following hold:

http_socket_options is left at its default (None)

No http_client or http_async_client supplied

No openai_proxy supplied

A proxy env var or system proxy is visible to httpx

On that specific shape, the instance falls back to pre-PR behavior and httpx’s env-proxy auto-detection applies (a one-time INFO log records the bypass for observability).

If you explicitly set http_socket_options=[…] while a proxy env var is also set, no bypass — you opted into the transport, and a one-time WARNING records the shadowing. Set http_socket_options=() or LANGCHAIN_OPENAI_TCP_KEEPALIVE=0 to disable transport injection explicitly, or pass a fully-configured http_async_client / http_client to take full control. The openai_proxy constructor kwarg is unaffected — socket options are applied cleanly through the proxied transport on that path.

stream_chunk_timeout: float | None¶

Per-chunk wall-clock timeout (seconds) on async streaming responses.

Applies to async invocations only (astream, ainvoke with streaming, etc.). Sync streaming (stream) is not affected.

Fires between content chunks yielded by the openai SDK’s streaming iterator (i.e., each call to __anext__ on the response). Crucially, this is not the same as httpx’s timeout.read:

httpx’s read timeout is inter-byte and gets reset every time any bytes
arrive on the socket — including OpenAI’s SSE keepalive comments (: keepalive) that trickle down during long model generations. A stream that’s silent on content but still producing keepalives looks alive forever to httpx.
stream_chunk_timeout measures the gap between parsed chunks. The
openai SDK’s SSE parser consumes keepalive comments internally and does not emit them as chunks, so keepalives do not reset this timer. It fires on genuine content silence.

When it fires, a StreamChunkTimeoutError (subclass of asyncio.TimeoutError) is raised with a self-describing message naming this knob, the env-var override, the model, and the number of chunks received before the stall. A WARNING log with extra={“source”: “stream_chunk_timeout”, “timeout_s”: <value>, “model_name”: <value>, “chunks_received”: <value>} also fires so aggregate logging can distinguish app-layer timeouts from transport-layer failures.

Defaults to 120s. Set to None or 0 to disable. Overridable via the LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S env var. Negative values (from either the env var or the constructor kwarg — e.g., hydrated from YAML/JSON configs) fall back to the default with a WARNING log rather than silently disabling the wrapper, so a misconfigured value still boots safely and the fallback is visible.

stop: list[str] | str | None¶: Default stop sequences.

extra_body: Mapping[str, Any] | None¶

Optional additional JSON properties to include in the request parameters when making requests to OpenAI compatible APIs, such as vLLM, LM Studio, or other providers.

This is the recommended way to pass custom parameters that are specific to your OpenAI-compatible API provider but not part of the standard OpenAI API.

Examples: - [LM Studio](https://lmstudio.ai/) TTL parameter: extra_body={“ttl”: 300} - [vLLM](https://github.com/vllm-project/vllm) custom parameters:

extra_body={“use_beam_search”: True}

Any other provider-specific parameters

!!! warning

Do not use model_kwargs for custom parameters that are not part of the standard OpenAI API, as this will cause errors when making API calls. Use extra_body instead.

include_response_headers: bool¶

Whether to include response headers in the output message response_metadata.

Note: some inference providers return additional metadata (such as served model names) in the response headers. Enable to capture these metadata.

disabled_params: dict[str, Any] | None¶

Parameters of the OpenAI client or chat.completions endpoint that should be disabled for the given model.

Should be specified as {“param”: None | [‘val1’, ‘val2’]} where the key is the parameter and the value is either None, meaning that parameter should never be used, or it’s a list of disabled values for the parameter.

For example, older models may not support the ‘parallel_tool_calls’ parameter at all, in which case disabled_params={“parallel_tool_calls”: None} can be passed in.

If a parameter is disabled then it will not be used by default in any methods, e.g. in with_structured_output. However this does not prevent a user from directly passed in the parameter during invocation.

context_management: list[dict[str, Any]] | None¶: Configuration for [context management](https://developers.openai.com/api/docs/guides/compaction).

include: list[str] | None¶

Additional fields to include in generations from Responses API.

Supported values:

‘file_search_call.results’
‘message.input_image.image_url’
‘computer_call_output.output.image_url’
‘reasoning.encrypted_content’
‘code_interpreter_call.outputs’

!!! version-added “Added in langchain-openai 0.3.24”

prompt_cache_options: dict[str, Any] | None¶

Options controlling OpenAI prompt cache behavior.

!!! version-added “Added in langchain-openai 1.3.5”

service_tier: str | None¶

Latency tier for request.

Options are ‘auto’, ‘default’, or ‘flex’.

Relevant for users of OpenAI’s scale tier service.

store: bool | None¶

If True, OpenAI may store response data for future use.

Defaults to True for the Responses API and False for the Chat Completions API.

!!! version-added “Added in langchain-openai 0.3.24”

truncation: str | None¶

Truncation strategy (Responses API).

Can be ‘auto’ or ‘disabled’ (default).

If ‘auto’, model may drop input items from the middle of the message sequence to fit the context window.

!!! version-added “Added in langchain-openai 0.3.24”

use_previous_response_id: bool¶

If True, always pass previous_response_id using the ID of the most recent response. Responses API only.

Input messages up to the most recent response will be dropped from request payloads.

For example, the following two are equivalent:

```python model = ChatOpenAI(

model=”…”, use_previous_response_id=True,

) model.invoke(

[
HumanMessage(“Hello”), AIMessage(“Hi there!”, response_metadata={“id”: “resp_123”}), HumanMessage(“How are you?”),

]

)¶

`python model = ChatOpenAI(model="...", use_responses_api=True) model.invoke([HumanMessage("How are you?")], previous_response_id="resp_123") `

!!! version-added “Added in langchain-openai 0.3.26”

use_responses_api: bool | None¶

Whether to use the Responses API instead of the Chat API.

If not specified then will be inferred based on invocation params.

!!! version-added “Added in langchain-openai 0.3.9”

output_version: str | None¶

Version of AIMessage output format to use.

This field is used to roll-out new output formats for chat model AIMessage responses in a backwards-compatible way.

Supported values:

‘v0’: AIMessage format as of langchain-openai 0.3.x.
‘responses/v1’: Formats Responses API output items into AIMessage content blocks
(Responses API only)
‘v1’: v1 of LangChain cross-provider standard.

!!! warning “Behavior changed in langchain-openai 1.0.0”

Default updated to “responses/v1”.

rate_limiter: BaseRateLimiter | None¶: An optional rate limiter to use for limiting the number of requests.

disable_streaming: bool | Literal['tool_calling']¶

Whether to disable streaming for this model.

If streaming is bypassed, then stream/astream/astream_events will defer to invoke/ainvoke.

If True, will always bypass streaming case.
If ‘tool_calling’, will bypass streaming case only when the model is called
with a tools keyword argument. In other words, LangChain will automatically switch to non-streaming behavior (invoke) only when the tools argument is provided. This offers the best of both worlds.
If False (Default), will always use streaming case if available.

The main reason for this flag is that code might be written using stream and a user may want to swap out a given model for another model whose implementation does not properly support streaming.

profile: ModelProfile | None¶

Profile detailing model capabilities.

!!! warning “Beta feature”

This is a beta feature. The format of model profiles is subject to change.

If not specified, automatically loaded from the provider package on initialization if data is available.

Example profile data includes context window sizes, supported modalities, or support for tool calling, structured output, and other features.

!!! version-added “Added in langchain-core 1.1.0”

cache: BaseCache | bool | None¶

Whether to cache the response.

If True, will use the global cache.
If False, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

verbose: bool¶: Whether to print out response text.

callbacks: Callbacks¶: Callbacks to add to the run trace.

tags: list[str] | None¶: Tags to add to the run trace.

metadata: builtins.dict[str, Any] | None¶: Metadata to add to the run trace.

custom_get_token_ids: Callable[[str], list[int]] | None¶: Optional encoder to use for counting tokens.

name: str | None¶

The name of the Runnable.

Used for debugging and tracing.

class gen_ai_hub.proxy.langchain.openai.OpenAI(*args, name: str | None = None, cache: ~langchain_core.caches.BaseCache | bool | None = None, verbose: bool = <factory>, callbacks: list[~langchain_core.callbacks.base.BaseCallbackHandler] | ~langchain_core.callbacks.base.BaseCallbackManager | None = None, tags: list[str] | None = None, metadata: dict[str, ~typing.Any] | None = None, custom_get_token_ids: ~collections.abc.Callable[[str], list[int]] | None = None, client: ~typing.Any = None, async_client: ~typing.Any = None, model_name: str | None = None, temperature: float = 0.7, max_tokens: int = 256, top_p: float = 1, frequency_penalty: float = 0, presence_penalty: float = 0, n: int = 1, best_of: int = 1, model_kwargs: dict[str, ~typing.Any] = <factory>, api_key: ~pydantic.types.SecretStr | None | ~collections.abc.Callable[[], str] = <factory>, base_url: str | None = <factory>, organization: str | None = <factory>, openai_proxy: str | None = <factory>, batch_size: int = 20, timeout: float | tuple[float, float] | ~typing.Any | None = None, logit_bias: dict[str, float] | None = None, max_retries: int = 2, seed: int | None = None, logprobs: int | None = None, streaming: bool = False, allowed_special: ~typing.Literal['all'] | set[str] = {}, disallowed_special: ~typing.Literal['all'] | ~collections.abc.Collection[str] = 'all', tiktoken_model_name: str | None = None, default_headers: ~collections.abc.Mapping[str, str] | None = None, default_query: ~collections.abc.Mapping[str, object] | None = None, http_client: ~typing.Any | None = None, http_async_client: ~typing.Any | None = None, extra_body: ~collections.abc.Mapping[str, ~typing.Any] | None = None, proxy_client: ~typing.Any | None = None, deployment_id: str | None = None, config_name: str | None = None, config_id: str | None = None, proxy_model_name: str | None = None, api_version: str | None = None, **kwargs)¶

Bases: ProxyOpenAI, OpenAI

OpenAI model using a proxy.

model_name: str | None¶: Model name to use.

openai_api_version: str | None¶

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'populate_by_name': True, 'protected_namespaces': (), 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, **kwargs)¶: Initialize the OpenAI object.

classmethod validate_environment(values: Dict) → Dict¶

Validates the environment.

Parameters:: values (Dict) – The input values
Returns:: The validated values
Return type:: Dict

deployment_id: str | None¶

config_name: str | None¶

config_id: str | None¶

proxy_model_name: str | None¶

client: Any¶

async_client: Any¶

temperature: float¶: What sampling temperature to use.

max_tokens: int¶: The maximum number of tokens to generate in the completion. -1 returns as many tokens as possible given the prompt and the models maximal context size.

top_p: float¶: Total probability mass of tokens to consider at each step.

frequency_penalty: float¶: Penalizes repeated tokens according to frequency.

presence_penalty: float¶: Penalizes repeated tokens.

n: int¶: How many completions to generate for each prompt.

best_of: int¶: Generates best_of completions server-side and returns the “best”.

model_kwargs: dict[str, Any]¶: Holds any model parameters valid for create call not explicitly specified.

openai_api_key: SecretStr | None | Callable[[], str]¶: Automatically inferred from env var OPENAI_API_KEY if not provided.

openai_api_base: str | None¶

Base URL path for API requests, leave blank if not using a proxy or service emulator.

Resolution order (first match wins):

Explicit base_url (or openai_api_base) kwarg.
Env var OPENAI_API_BASE (read by LangChain at init).
Env var OPENAI_BASE_URL (read by the underlying openai SDK client).

openai_organization: str | None¶: Automatically inferred from env var OPENAI_ORG_ID if not provided.

openai_proxy: str | None¶

batch_size: int¶: Batch size to use when passing multiple documents to generate.

request_timeout: float | tuple[float, float] | Any | None¶: Timeout for requests to OpenAI completion API. Can be float, httpx.Timeout or None.

logit_bias: dict[str, float] | None¶: Adjust the probability of specific tokens being generated.

max_retries: int¶: Maximum number of retries to make when generating.

seed: int | None¶: Seed for generation

logprobs: int | None¶: Include the log probabilities on the logprobs most likely output tokens, as well the chosen tokens.

streaming: bool¶: Whether to stream the results or not.

allowed_special: Literal['all'] | set[str]¶: Set of special tokens that are allowed。

disallowed_special: Literal['all'] | Collection[str]¶: Set of special tokens that are not allowed。

tiktoken_model_name: str | None¶

The model name to pass to tiktoken when using this class.

Tiktoken is used to count the number of tokens in documents to constrain them to be under a certain limit.

default_headers: Mapping[str, str] | None¶

default_query: Mapping[str, object] | None¶

http_client: Any | None¶

Optional httpx.Client.

Only used for sync invocations. Must specify http_async_client as well if you’d like a custom client for async invocations.

http_async_client: Any | None¶

Optional httpx.AsyncClient.

Only used for async invocations. Must specify http_client as well if you’d like a custom client for sync invocations.

extra_body: Mapping[str, Any] | None¶: Optional additional JSON properties to include in the request parameters when making requests to OpenAI compatible APIs, such as vLLM.

cache: BaseCache | bool | None¶

Whether to cache the response.

If True, will use the global cache.
If False, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

verbose: bool¶: Whether to print out response text.

callbacks: Callbacks¶: Callbacks to add to the run trace.

tags: list[str] | None¶: Tags to add to the run trace.

metadata: builtins.dict[str, Any] | None¶: Metadata to add to the run trace.

custom_get_token_ids: Callable[[str], list[int]] | None¶: Optional encoder to use for counting tokens.

name: str | None¶

The name of the Runnable.

Used for debugging and tracing.

class gen_ai_hub.proxy.langchain.openai.OpenAIEmbeddings(*args, client: ~typing.Any = None, async_client: ~typing.Any = None, model: str | None = None, dimensions: int | None = None, deployment: str | None = 'text-embedding-ada-002', api_version: str | None = None, base_url: str | None = <factory>, openai_api_type: str | None = <factory>, openai_proxy: str | None = <factory>, embedding_ctx_length: int = 8191, api_key: ~pydantic.types.SecretStr | None | ~collections.abc.Callable[[], str] | ~collections.abc.Callable[[], ~collections.abc.Awaitable[str]] = <factory>, organization: str | None = <factory>, allowed_special: ~typing.Literal['all'] | set[str] | None = None, disallowed_special: ~typing.Literal['all'] | set[str] | ~collections.abc.Sequence[str] | None = None, chunk_size: int = 16, max_retries: int = 2, timeout: float | tuple[float, float] | ~typing.Any | None = None, headers: ~typing.Any = None, tiktoken_enabled: bool = True, tiktoken_model_name: str | None = 'text-embedding-ada-002', show_progress_bar: bool = False, model_kwargs: dict[str, ~typing.Any] = <factory>, skip_empty: bool = False, default_headers: ~collections.abc.Mapping[str, str] | None = None, default_query: ~collections.abc.Mapping[str, object] | None = None, retry_min_seconds: int = 4, retry_max_seconds: int = 20, http_client: ~typing.Any | None = None, http_async_client: ~typing.Any | None = None, check_embedding_ctx_length: bool = True, proxy_client: ~typing.Any | None = None, deployment_id: str | None = None, config_name: str | None = None, config_id: str | None = None, proxy_model_name: str | None = None, input_type: str | None = None, **kwargs)¶

Bases: ProxyOpenAI, OpenAIEmbeddings

OpenAI Embeddings model using a proxy.

model: str | None¶

tiktoken_model_name: str | None¶

The model name to pass to tiktoken when using this class.

Tiktoken is used to count the number of tokens in documents to constrain them to be under a certain limit.

chunk_size: int¶: Maximum number of texts to embed in each batch

openai_api_version: str | None¶

Version of the OpenAI API to use.

Automatically inferred from env var OPENAI_API_VERSION if not provided.

input_type: str | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'populate_by_name': True, 'protected_namespaces': (), 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, **kwargs)¶: Initialize the OpenAIEmbeddings object.

classmethod validate_environment(values: Dict) → Dict¶

Validates the environment.

Parameters:: values (Dict) – The input values
Returns:: The validated values
Return type:: Dict

deployment_id: str | None¶

config_name: str | None¶

config_id: str | None¶

proxy_model_name: str | None¶

client: Any¶

async_client: Any¶

dimensions: int | None¶

The number of dimensions the resulting output embeddings should have.

Only supported in ‘text-embedding-3’ and later models.

deployment: str | None¶

openai_api_base: str | None¶

Base URL path for API requests, leave blank if not using a proxy or service emulator.

Resolution order (first match wins):

Explicit base_url (or openai_api_base) kwarg.
Env var OPENAI_API_BASE (read by LangChain at init).
Env var OPENAI_BASE_URL (read by the underlying openai SDK client).

openai_api_type: str | None¶

openai_proxy: str | None¶

embedding_ctx_length: int¶: The maximum number of tokens to embed at once.

openai_api_key: SecretStr | None | Callable[[], str] | Callable[[], Awaitable[str]]¶

API key to use for API calls.

Automatically inferred from env var OPENAI_API_KEY if not provided.

openai_organization: str | None¶

OpenAI organization ID to use for API calls.

Automatically inferred from env var OPENAI_ORG_ID if not provided.

allowed_special: Literal['all'] | set[str] | None¶

disallowed_special: Literal['all'] | set[str] | Sequence[str] | None¶

max_retries: int¶: Maximum number of retries to make when generating.

request_timeout: float | tuple[float, float] | Any | None¶

Timeout for requests to OpenAI completion API.

Can be float, httpx.Timeout or None.

headers: Any¶

tiktoken_enabled: bool¶

Set this to False to use HuggingFace transformers tokenization.

For non-OpenAI providers (OpenRouter, Ollama, vLLM, etc.), consider setting check_embedding_ctx_length=False instead, as it bypasses tokenization entirely.

show_progress_bar: bool¶: Whether to show a progress bar when embedding.

model_kwargs: dict[str, Any]¶: Holds any model parameters valid for create call not explicitly specified.

skip_empty: bool¶: Whether to skip empty strings when embedding or raise an error.

default_headers: Mapping[str, str] | None¶

default_query: Mapping[str, object] | None¶

retry_min_seconds: int¶: Min number of seconds to wait between retries

retry_max_seconds: int¶: Max number of seconds to wait between retries

http_client: Any | None¶

Optional httpx.Client.

Only used for sync invocations. Must specify http_async_client as well if you’d like a custom client for async invocations.

http_async_client: Any | None¶

Optional httpx.AsyncClient.

Only used for async invocations. Must specify http_client as well if you’d like a custom client for sync invocations.

check_embedding_ctx_length: bool¶

Whether to check the token length of inputs and automatically split inputs longer than embedding_ctx_length.

Set to False to send raw text strings directly to the API instead of tokenizing. Useful for many non-OpenAI providers (e.g. OpenRouter, Ollama, vLLM).

gen_ai_hub.proxy.langchain.openai.init_chat_model(proxy_client: BaseProxyClient, deployment: BaseDeployment, temperature: float = 0.0, max_tokens: int = 256, top_k: int | None = None, top_p: float = 1.0)¶

Initialize the ChatOpenAI model.

Parameters:

proxy_client (BaseProxyClient) – the proxy client
deployment (BaseDeployment) – the deployment
temperature (float, optional) – the temperature, defaults to 0.0
max_tokens (int, optional) – the maximum tokens, defaults to 256
top_k (Optional[int], optional) – the top k, defaults to None
top_p (float, optional) – the top p, defaults to 1.0

Returns:

the ChatOpenAI model

Return type:

ChatOpenAI

gen_ai_hub.proxy.langchain.openai.init_embedding_model(proxy_client: BaseProxyClient, deployment: BaseDeployment)¶

Initialize the OpenAIEmbeddings model.

Parameters:

proxy_client (BaseProxyClient) – the proxy client
deployment (BaseDeployment) – the deployment

Returns:

the OpenAIEmbeddings model

Return type:

OpenAIEmbeddings