gen_ai_hub.proxy.langchain.google_genai module

Drop-in replacements for langchain_google_genai models with SAP AI Core integration.

class gen_ai_hub.proxy.langchain.google_genai.ChatGoogleGenerativeAI(model: str = '', proxy_model_name: str = '', model_id: str = '', deployment_id: str = '', config_id: str = '', config_name: str = '', proxy_client: ~gen_ai_hub.proxy.core.base.BaseProxyClient | None = None, *, name: str | None = None, cache: ~langchain_core.caches.BaseCache | bool | None = None, verbose: bool = <factory>, callbacks: list[~langchain_core.callbacks.base.BaseCallbackHandler] | ~langchain_core.callbacks.base.BaseCallbackManager | None = None, tags: list[str] | None = None, metadata: dict[str, ~typing.Any] | None = None, custom_get_token_ids: ~collections.abc.Callable[[str], list[int]] | None = None, rate_limiter: ~langchain_core.rate_limiters.BaseRateLimiter | None = None, disable_streaming: bool | ~typing.Literal['tool_calling'] = False, output_version: str | None = <factory>, profile: ~langchain_core.language_models.model_profile.ModelProfile | None = None, api_key: ~pydantic.types.SecretStr | None = <factory>, credentials: ~typing.Any = None, vertexai: bool | None = None, project: str | None = None, location: str | None = <factory>, client_options: str | dict | None = None, additional_headers: dict[str, str] | None = None, client_args: dict[str, ~typing.Any] | None = None, api_version: str | None = None, temperature: float | None = 0.7, frequency_penalty: float | None = None, presence_penalty: float | None = None, top_p: float | None = None, top_k: int | None = None, max_tokens: int | None = None, candidate_count: int = 1, retries: int = 6, request_timeout: float | None = None, response_modalities: list[~google.genai.types.Modality] | None = None, media_resolution: ~google.genai.types.MediaResolution | None = None, image_config: dict[str, ~typing.Any] | None = None, thinking_budget: int | None = None, include_thoughts: bool | None = None, safety_settings: dict[~google.genai.types.HarmCategory, ~google.genai.types.HarmBlockThreshold] | None = None, seed: int | None = None, labels: dict[str, str] | None = None, client: ~google.genai.client.Client | None = None, default_metadata_input: ~collections.abc.Sequence[tuple[str, str]] | None = None, model_kwargs: dict[str, ~typing.Any] = <factory>, streaming: bool | None = None, convert_system_message_to_human: bool = False, stop_sequences: list[str] | None = None, response_mime_type: str | None = None, response_schema: dict[str, ~typing.Any] | None = None, thinking_level: ~typing.Literal['minimal', 'low', 'medium', 'high'] | None = None, thinking_config: dict[str, ~typing.Any] | ~google.genai.types.ThinkingConfig | None = None, cached_content: str | None = None)

Bases: _BaseGoogleGenerativeAI, ChatGoogleGenerativeAI

Drop-in replacement for langchain_google_genai.ChatGoogleGenerativeAI.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'ignore', 'populate_by_name': True, 'protected_namespaces': (), 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

client: Client | None

default_metadata: Sequence[tuple[str, str]] | None

model_kwargs: dict[str, Any]

Holds any unexpected initialization parameters.

streaming: bool | None

Whether to stream responses from the model.

convert_system_message_to_human: bool

Whether to merge any leading SystemMessage into the following HumanMessage.

Gemini does not support system messages; any unsupported messages will raise an error.

stop: list[str] | None

Stop sequences for the model.

response_mime_type: str | None

Output response MIME type of the generated candidate text.

Supported MIME types:

• ‘text/plain’: (default) Text output.

• ‘application/json’: JSON response in the candidates.

• ‘text/x.enum’: Enum in plain text. (legacy; use JSON schema output instead)

!!! note

The model also needs to be prompted to output the appropriate response type, otherwise the behavior is undefined.

(In other words, simply setting this param doesn’t force the model to comply; it only tells the model the kind of output expected. You still need to prompt it correctly.)

response_schema: dict[str, Any] | None

Enforce a schema to the output.

The format of the dictionary should follow JSON Schema specification.

!!! note “Schema Transformation”

The Google GenAI SDK automatically transforms schemas for Gemini compatibility:

• Inlines $defs definitions (enables Union types with anyOf)

• Resolves $ref pointers for nested/recursive schemas

• Preserves property ordering

• Supports constraints like minimum/maximum, minItems/maxItems

!!! tip “Using Union Types”

Union types in Pydantic models (e.g., field: Union[TypeA, TypeB]) are automatically converted to anyOf schemas and work correctly with the json_schema method.

Refer to the Gemini API [docs](https://ai.google.dev/gemini-api/docs/structured-output) for more details on supported JSON Schema features.

reasoning_effort: Literal['minimal', 'low', 'medium', 'high'] | None

Indicates the thinking level.

Possible values (support varies by model):

• ‘minimal’: Lowest available reasoning depth.

• ‘low’: Minimizes latency and cost.

• ‘medium’: Balances latency/cost with reasoning depth.

• ‘high’: Maximizes reasoning depth.

Check the model profile’s reasoning_effort_levels and reasoning_effort_default fields for model-specific support. If those fields are unavailable, consult the upstream [Gemini API docs](https://ai.google.dev/gemini-api/docs/generate-content/thinking#thinking-levels-gemini-3).

!!! note “Replaces thinking_budget”

thinking_budget is deprecated for Gemini 3+ models. If both parameters are provided, this field takes precedence.

If left unspecified, the model’s default thinking level is used.

!!! note “thinking_level alias”

thinking_level – Gemini’s own native name for this setting – is also accepted as an alias for this field, at both construction and call time. If both thinking_level and reasoning_effort are set, thinking_level wins (Pydantic’s alias-resolution precedence). Use reasoning_effort or thinking_level interchangeably to read the value back.

thinking_config: dict[str, Any] | ThinkingConfig | None

Raw Google GenAI thinking configuration.

Accepts the same fields as google.genai.types.ThinkingConfig, including thinking_level, thinking_budget, and include_thoughts.

!!! note “Precedence”

If thinking_config is provided together with flat thinking arguments, the flat arguments take precedence for matching fields. After merging, thinking_level takes precedence over thinking_budget for Gemini 3+ models.

cached_content: str | None

The name of the cached content used as context to serve the prediction.

!!! note

Only used in explicit caching, where users can have control over caching (e.g. what content to cache) and enjoy guaranteed cost savings. Format: cachedContents/{cachedContent}.

google_api_key: SecretStr | None

API key for authentication.

If not specified, will check the env vars GOOGLE_API_KEY and GEMINI_API_KEY with precedence given to GOOGLE_API_KEY.

• Gemini Developer API: API key is required (default when no project is set)

• Vertex AI: API key is optional (set vertexai=True or provide project)

  • If provided, uses API key for authentication

  • If not provided, uses [Application Default Credentials (ADC)](https://docs.cloud.google.com/docs/authentication/application-default-credentials)

    or credentials parameter

!!! tip “Vertex AI with API key”

You can now use Vertex AI with API key authentication instead of service account credentials. Set GOOGLE_GENAI_USE_VERTEXAI=true or vertexai=True along with your API key and project.

credentials: Any

Custom credentials for Vertex AI authentication.

When provided, forces Vertex AI backend (regardless of API key presence in google_api_key/api_key).

Accepts a [google.auth.credentials.Credentials](https://googleapis.dev/python/google-auth/latest/reference/google.auth.credentials.html#google.auth.credentials.Credentials) object.

If omitted and no API key is found, the SDK uses [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials).

!!! example “Service account credentials”

```python from google.oauth2 import service_account

credentials = service_account.Credentials.from_service_account_file(

“path/to/service-account.json”, scopes=[”https://www.googleapis.com/auth/cloud-platform”],

)

llm = ChatGoogleGenerativeAI(

model=”gemini-3.5-flash”, credentials=credentials, project=”my-project-id”,

vertexai: bool | None

Whether to use Vertex AI backend.

If None (default), backend is automatically determined as follows:

1. If the GOOGLE_GENAI_USE_VERTEXAI env var is set, uses Vertex AI

2. If the credentials parameter is provided, uses Vertex AI

3. If the project parameter is provided, uses Vertex AI

4. Otherwise, uses Gemini Developer API

Set explicitly to True or False to override auto-detection.

!!! tip “Vertex AI with API key”

You can use Vertex AI with API key authentication by setting:

`bash export GEMINI_API_KEY='your-api-key' export GOOGLE_GENAI_USE_VERTEXAI=true export GOOGLE_CLOUD_PROJECT='your-project-id' `

Or programmatically:

```python llm = ChatGoogleGenerativeAI(

model=”gemini-3.1-pro-preview”, api_key=”your-api-key”, project=”your-project-id”, vertexai=True,

This allows for simpler authentication compared to service account JSON files.

project: str | None

Google Cloud project ID (Vertex AI only).

Required when using Vertex AI.

Falls back to GOOGLE_CLOUD_PROJECT env var if not provided.

location: str | None

Google Cloud region (Vertex AI only).

If not provided, falls back to the GOOGLE_CLOUD_LOCATION env var, then ‘global’.

base_url: str | dict | None

Custom base URL for the API client.

If not provided, defaults depend on the API being used:

• Gemini Developer API (api_key/google_api_key):

  https://generativelanguage.googleapis.com/

• Vertex AI (credentials):

  https://{location}-aiplatform.googleapis.com/

!!! note “Backwards compatibility”

Typed to accept dict to support backwards compatibility for the (now removed) client_options param.

If a dict is passed in, it will only extract the ‘api_endpoint’ key.

additional_headers: dict[str, str] | None

Additional HTTP headers to include in API requests.

Passed as headers to HttpOptions when creating the client.

!!! example

```python llm = ChatGoogleGenerativeAI(

model=”gemini-3.5-flash”, additional_headers={

“X-Custom-Header”: “value”,

},

client_args: dict[str, Any] | None

Additional arguments to pass to the underlying HTTP client.

Applied to both sync and async clients.

!!! example “SOCKS5 proxy”

```python llm = ChatGoogleGenerativeAI(

model=”gemini-3.5-flash”, client_args={“proxy”: “socks5://user:pass@host:port”},

api_version: str | None

Override the API version path segment in request URLs.

By default, the underlying google-genai SDK currently uses v1beta1 for Vertex AI and v1beta for the Gemini Developer API. Set this when targeting a proxy or gateway that expects a different API version segment (e.g. ‘v1’).

!!! example “Custom API gateway”

```python llm = ChatGoogleGenerativeAI(

model=”gemini-3.5-flash”, vertexai=True, base_url=”https://my-gateway.example.com/api/gemini”, api_version=”v1”, additional_headers={“Authorization”: “Bearer <token>”},

model: str

Model name to use.

temperature: float | None

Run inference with this temperature.

Must be within [0.0, 2.0].

!!! note “Automatic override for Gemini 3.0+ models”

If temperature is not explicitly set and the model is Gemini 3.0 or later, it will be automatically set to None instead of the default 0.7 per the Google GenAI API best practices, as it can cause infinite loops, degraded reasoning performance, and failure on complex tasks.

frequency_penalty: float | None

Penalize tokens proportionally to how often they have already appeared.

Scales with the count of prior appearances, so it discourages verbatim repetition more strongly than presence_penalty.

Must be within [-2.0, 2.0].

presence_penalty: float | None

Penalize tokens that have already appeared at all in the generated text.

Applied once a token has appeared, regardless of how many times, so it encourages introducing new topics rather than reducing repetition.

Must be within [-2.0, 2.0].

top_p: float | None

Decode using nucleus sampling.

Consider the smallest set of tokens whose probability sum is at least top_p.

Must be within [0.0, 1.0].

top_k: int | None

consider the set of top_k most probable tokens.

Must be positive.

Type:

Decode using top-k sampling

max_output_tokens: int | None

Maximum number of tokens to include in a candidate.

Must be greater than zero.

If unset, will use the model’s default value, which varies by model.

See [docs](https://ai.google.dev/gemini-api/docs/models) for model-specific limits.

To constrain the number of thinking tokens to use when generating a response, see the thinking_budget parameter.

n: int

Number of chat completions to generate for each prompt.

Note that the API may not return the full n completions if duplicates are generated.

max_retries: int

The maximum number of retries to make when generating.

!!! warning “Disabling retries”

To disable retries, set max_retries=1 (not 0) due to a quirk in the underlying Google SDK. max_retries=0 is interpreted as “use the (Google) default” (5 retries).

Setting max_retries=1 means only the initial request is made with no retries.

!!! warning “Handling rate limits (429 errors)”

When you exceed quota limits, the API returns a 429 error with a suggested retry_delay. The SDK’s built-in retry logic ignores this value and uses fixed exponential backoff instead. This is a known issue in Google’s SDK and an issue has been [raised upstream](https://github.com/googleapis/python-genai/issues/1875). We plan to implement proper handling once it’s supported.

If you need to respect the server’s suggested retry delay, disable SDK retries with max_retries=1 and implement custom retry logic:

```python import re import time

from langchain_google_genai import ChatGoogleGenerativeAI from langchain_google_genai.chat_models import ChatGoogleGenerativeAIError

llm = ChatGoogleGenerativeAI(model=”gemini-2.0-flash”, max_retries=1)

try:

response = llm.invoke(“Hello”)

except ChatGoogleGenerativeAIError as e:

if “429” in str(e):

# Parse retry_delay from error: “[retry_delay { seconds: N }]” match = re.search(r”retry_delays*{s*seconds:s*(d+)”, str(e)) delay = int(match.group(1)) if match else 60 time.sleep(delay) # Retry…

```

timeout: float | None

The maximum number of seconds to wait for a response.

response_modalities: list[Modality] | None

A list of modalities of the response

media_resolution: MediaResolution | None

Media resolution for the input media.

May be defined at the individual part level, allowing for mixed-resolution requests (e.g., images and videos of different resolutions in the same request).

May be ‘low’, ‘medium’, or ‘high’.

Can be set either per-part or globally for all media inputs in the request. To set globally, set in the generation_config.

!!! warning “Model compatibility”

Setting per-part media resolution requests to Gemini 2.5 models is not supported.

image_config: dict[str, Any] | None

Configuration for image generation.

Provides control over generated image dimensions and quality for image generation models.

See [genai.types.ImageConfig](https://googleapis.github.io/python-genai/genai.html#genai.types.ImageConfig) for a list of supported fields and their values.

!!! note “Model compatibility”

This parameter only applies to image generation models. Supported parameters vary by model and backend (Gemini Developer API and Vertex AI each support different subsets of parameters and models).

See [the docs](https://docs.langchain.com/oss/python/integrations/chat/google_generative_ai#image-generation) for more details and examples.

thinking_budget: int | None

Indicates the thinking budget in tokens.

Used to disable thinking for supported models (when set to 0) or to constrain the number of tokens used for thinking.

Dynamic thinking (allowing the model to decide how many tokens to use) is enabled when set to -1.

More information, including per-model limits, can be found in the [Gemini API docs](https://ai.google.dev/gemini-api/docs/thinking#set-budget).

include_thoughts: bool | None

Indicates whether to include thoughts in the response.

!!! note

This parameter is only applicable for models that support thinking.

This does not disable thinking; to disable thinking, set thinking_budget to 0. for supported models. See the thinking_budget parameter for more details.

safety_settings: SafetySettingDict | None

Default safety settings to use for all generations.

!!! example

```python from google.genai.types import HarmBlockThreshold, HarmCategory

safety_settings = {

HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_ONLY_HIGH, HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,

seed: int | None

Seed used in decoding for reproducible generations.

By default, a random number is used.

!!! note

Using the same seed does not guarantee identical outputs, but makes them more deterministic. Reproducibility is “best effort” based on the model and infrastructure.

labels: dict[str, str] | None

User-defined key-value metadata for organizing and filtering billing reports.

Attach labels to categorize API usage by team, environment, or feature.

Can be overridden per-request via invoke kwargs.

See: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/add-labels-to-api-calls

rate_limiter: BaseRateLimiter | None

An optional rate limiter to use for limiting the number of requests.

disable_streaming: bool | Literal['tool_calling']

Whether to disable streaming for this model.

If streaming is bypassed, then stream/astream/astream_events will defer to invoke/ainvoke.

• If True, will always bypass streaming case.

• If ‘tool_calling’, will bypass streaming case only when the model is called

  with a tools keyword argument. In other words, LangChain will automatically switch to non-streaming behavior (invoke) only when the tools argument is provided. This offers the best of both worlds.

• If False (Default), will always use streaming case if available.

The main reason for this flag is that code might be written using stream and a user may want to swap out a given model for another model whose implementation does not properly support streaming.

output_version: str | None

Version of AIMessage output format to store in message content.

AIMessage.content_blocks will lazily parse the contents of content into a standard format. This flag can be used to additionally store the standard format in message content, e.g., for serialization purposes.

Supported values:

• ‘v0’: provider-specific format in content (can lazily-parse with

  content_blocks)

• ‘v1’: standardized format in content (consistent with content_blocks)

Partner packages (e.g., [langchain-openai](https://pypi.org/project/langchain-openai)) can also use this field to roll out new content formats in a backward-compatible way.

!!! version-added “Added in langchain-core 1.0.0”

profile: ModelProfile | None

Profile detailing model capabilities.

!!! warning “Beta feature”

This is a beta feature. The format of model profiles is subject to change.

If not specified, automatically loaded from the provider package on initialization if data is available.

Example profile data includes context window sizes, supported modalities, or support for tool calling, structured output, and other features.

!!! version-added “Added in langchain-core 1.1.0”

cache: BaseCache | bool | None

Whether to cache the response.

• If True, will use the global cache.

• If False, will not use a cache

• If None, will use the global cache if it’s set, otherwise no cache.

• If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

verbose: bool

Whether to print out response text.

callbacks: Callbacks

Callbacks to add to the run trace.

tags: list[str] | None

Tags to add to the run trace.

metadata: builtins.dict[str, Any] | None

Metadata to add to the run trace.

custom_get_token_ids: Callable[[str], list[int]] | None

Optional encoder to use for counting tokens.

name: str | None

The name of the Runnable.

Used for debugging and tracing.

class gen_ai_hub.proxy.langchain.google_genai.GoogleGenerativeAIEmbeddings(model: str = '', proxy_model_name: str = '', model_id: str = '', deployment_id: str = '', config_id: str = '', config_name: str = '', proxy_client: ~gen_ai_hub.proxy.core.base.BaseProxyClient | None = None, *, client: ~typing.Any = None, task_type: str | None = None, api_key: ~pydantic.types.SecretStr | None = <factory>, credentials: ~typing.Any = None, vertexai: bool | None = None, project: str | None = None, location: str | None = <factory>, base_url: str | None = None, additional_headers: dict[str, str] | None = None, client_args: dict[str, ~typing.Any] | None = None, api_version: str | None = None, request_options: dict | None = None, output_dimensionality: int | None = None, **kwargs)

Bases: _BaseGoogleGenerativeAI, GoogleGenerativeAIEmbeddings

Drop-in replacement for langchain_google_genai.GoogleGenerativeAIEmbeddings.

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

client: Any

The Google GenAI client instance.

model: str

The name of the embedding model to use.

task_type: str | None

The task type.

Valid options include:

• ‘TASK_TYPE_UNSPECIFIED’

• ‘RETRIEVAL_QUERY’

• ‘RETRIEVAL_DOCUMENT’

• ‘SEMANTIC_SIMILARITY’

• ‘CLASSIFICATION’

• ‘CLUSTERING’

• ‘QUESTION_ANSWERING’

• ‘FACT_VERIFICATION’

• ‘CODE_RETRIEVAL_QUERY’

See [TaskType](https://ai.google.dev/api/embeddings#tasktype) for details.

google_api_key: SecretStr | None

The Google API key to use.

If not provided, will check the env vars GOOGLE_API_KEY and GEMINI_API_KEY.

credentials: Any

Custom credentials for Vertex AI authentication.

When provided, forces Vertex AI backend.

Accepts a google.auth.credentials.Credentials object.

vertexai: bool | None

Whether to use Vertex AI backend.

If None (default), backend is automatically determined:

1. If GOOGLE_GENAI_USE_VERTEXAI env var is set, uses that value

2. If credentials parameter is provided, uses Vertex AI

3. If project parameter is provided, uses Vertex AI

4. Otherwise, uses Gemini Developer API

project: str | None

Google Cloud project ID (Vertex AI only).

Falls back to GOOGLE_CLOUD_PROJECT env var if not provided.

location: str | None

Google Cloud region (Vertex AI only).

Defaults to GOOGLE_CLOUD_LOCATION env var, then ‘us-central1’.

base_url: str | None

The base URL to use for the API client.

additional_headers: dict[str, str] | None

Additional HTTP headers to include in API requests.

client_args: dict[str, Any] | None

Additional arguments to pass to the underlying HTTP client.

Applied to both sync and async clients.

api_version: str | None

Override the API version path segment in request URLs.

By default, the underlying google-genai SDK currently uses v1beta1 for Vertex AI and v1beta for the Gemini Developer API. Set this when targeting a proxy or gateway that expects a different API version segment (e.g. ‘v1’).

request_options: dict | None

A dictionary of request options to pass to the Google API client.

Example: {‘timeout’: 10}

output_dimensionality: int | None

Default output dimensionality for embeddings.

If set, all embed calls use this dimension unless explicitly overridden.

gen_ai_hub.proxy.langchain.google_genai.init_chat_model(proxy_client: BaseProxyClient, deployment: Deployment, temperature: float = 0.0, max_tokens: int = 256, top_k: int | None = None, top_p: float = 1.0)

Initialize a ChatGoogleGenerativeAI model with the given parameters.

Parameters:

• proxy_client (BaseProxyClient) – proxy client to use for the model

• deployment (Deployment) – deployment information for the model

• temperature (float, optional) – sampling temperature, defaults to 0.0

• max_tokens (int, optional) – maximum number of tokens to generate, defaults to 256

• top_k (Optional[int], optional) – k for top-k sampling, defaults to None

• top_p (float, optional) – p for nucleus sampling, defaults to 1.0

Returns:

initialized ChatGoogleGenerativeAI model

Return type:

ChatGoogleGenerativeAI

gen_ai_hub.proxy.langchain.google_genai.init_embedding_model(proxy_client: BaseProxyClient, deployment: Deployment)