gen_ai_hub.proxy.langchain.amazon module¶

class gen_ai_hub.proxy.langchain.amazon.AICoreBedrockBaseModel(*args, model_id: str = '', deployment_id: str = '', model_name: str = '', config_id: str = '', config_name: str = '', proxy_client: BaseProxyClient | None = None, **kwargs)¶

Bases: BaseModel

AICoreBedrockBaseModel provides all adjustments to boto3 based LangChain classes to enable communication with SAP AI Core.

model_config: ClassVar[ConfigDict] = {'extra': 'allow'}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, model_id: str = '', deployment_id: str = '', model_name: str = '', config_id: str = '', config_name: str = '', proxy_client: BaseProxyClient | None = None, **kwargs)¶

Initializes the AICoreBedrockBaseModel with AICore specific parameters.: Extends the constructor of the base class with aicore specific parameters

Parameters:

model_id (str, optional) – the model identifier, defaults to “”
deployment_id (str, optional) – the deployment identifier, defaults to “”
model_name (str, optional) – the model name, defaults to “”
config_id (str, optional) – the configuration identifier, defaults to “”
config_name (str, optional) – the configuration name, defaults to “”
proxy_client (Optional[BaseProxyClient], optional) – the proxy client to use, defaults to None

static get_corresponding_model_id(full_model_name, model_version='latest')¶: Gets the corresponding model ID for a given model name. :param full_model_name: the model name :type full_model_name: str :param model_version: the model version :type model_version: str :return: the corresponding model ID :rtype: str

classmethod validate_environment(values: Dict) → Dict¶

Validates and sets up the environment for the model.

Parameters:: values (Dict) – the input values
Returns:: the validated values
Return type:: Dict

class gen_ai_hub.proxy.langchain.amazon.ChatBedrock(*args, name: str | None = None, cache: ~langchain_core.caches.BaseCache | bool | None = None, verbose: bool = <factory>, callbacks: list[~langchain_core.callbacks.base.BaseCallbackHandler] | ~langchain_core.callbacks.base.BaseCallbackManager | None = None, tags: list[str] | None = None, metadata: dict[str, ~typing.Any] | None = None, custom_get_token_ids: ~collections.abc.Callable[[str], list[int]] | None = None, client: ~typing.Any = None, bedrock_client: ~typing.Any = None, region: str | None = None, credentials_profile_name: str | None = None, aws_access_key_id: ~pydantic.types.SecretStr | None = <factory>, aws_secret_access_key: ~pydantic.types.SecretStr | None = <factory>, aws_session_token: ~pydantic.types.SecretStr | None = <factory>, api_key: ~pydantic.types.SecretStr | None = <factory>, config: ~typing.Any = None, timeout: int | None = None, max_retries: int | None = None, provider: str | None = None, model: str, base_model: str | None = None, model_kwargs: ~typing.Dict[str, ~typing.Any] | None = None, endpoint_url: str | None = None, streaming: bool = False, provider_stop_sequence_key_name_map: ~typing.Mapping[str, str] = {'ai21': 'stop_sequences', 'amazon': 'stopSequences', 'anthropic': 'stop_sequences', 'cohere': 'stop_sequences', 'mistral': 'stop_sequences'}, provider_stop_reason_key_map: ~typing.Mapping[str, str] = {'ai21': 'finishReason', 'amazon': 'completionReason', 'anthropic': 'stop_reason', 'cohere': 'finish_reason', 'mistral': 'stop_reason'}, guardrails: ~typing.Mapping[str, ~typing.Any] | None = {'guardrailIdentifier': None, 'guardrailVersion': None, 'trace': None}, temperature: float | None = None, max_tokens: int | None = None, service_tier: ~typing.Literal['priority', 'default', 'flex', 'reserved'] | None = None, rate_limiter: ~langchain_core.rate_limiters.BaseRateLimiter | None = None, disable_streaming: bool | ~typing.Literal['tool_calling'] = False, output_version: str | None = <factory>, profile: ~langchain_core.language_models.model_profile.ModelProfile | None = None, system_prompt_with_tools: str = '', beta_use_converse_api: bool = False, stop: ~typing.List[str] | None = None, **kwargs)¶

Bases: AICoreBedrockBaseModel, ChatBedrock

Drop-in replacement for LangChain ChatBedrock.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'populate_by_name': True, 'protected_namespaces': (), 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, **kwargs)¶

Initializes the AICoreBedrockBaseModel with AICore specific parameters.: Extends the constructor of the base class with aicore specific parameters

Parameters:

model_id (str, optional) – the model identifier, defaults to “”
deployment_id (str, optional) – the deployment identifier, defaults to “”
model_name (str, optional) – the model name, defaults to “”
config_id (str, optional) – the configuration identifier, defaults to “”
config_name (str, optional) – the configuration name, defaults to “”
proxy_client (Optional[BaseProxyClient], optional) – the proxy client to use, defaults to None

system_prompt_with_tools: str¶

beta_use_converse_api: bool¶: Use the new Bedrock converse API which provides a standardized interface to all Bedrock models. Support still in beta. See ChatBedrockConverse docs for more.

stop_sequences: List[str] | None¶: Stop sequence inference parameter from new Bedrock converse API providing a sequence of characters that causes a model to stop generating a response. See https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_InferenceConfiguration.html for more.

rate_limiter: BaseRateLimiter | None¶: An optional rate limiter to use for limiting the number of requests.

disable_streaming: bool | Literal['tool_calling']¶

Whether to disable streaming for this model.

If streaming is bypassed, then stream/astream/astream_events will defer to invoke/ainvoke.

If True, will always bypass streaming case.
If ‘tool_calling’, will bypass streaming case only when the model is called
with a tools keyword argument. In other words, LangChain will automatically switch to non-streaming behavior (invoke) only when the tools argument is provided. This offers the best of both worlds.
If False (Default), will always use streaming case if available.

The main reason for this flag is that code might be written using stream and a user may want to swap out a given model for another model whose implementation does not properly support streaming.

output_version: str | None¶

Version of AIMessage output format to store in message content.

AIMessage.content_blocks will lazily parse the contents of content into a standard format. This flag can be used to additionally store the standard format in message content, e.g., for serialization purposes.

Supported values:

‘v0’: provider-specific format in content (can lazily-parse with
content_blocks)
‘v1’: standardized format in content (consistent with content_blocks)

Partner packages (e.g., [langchain-openai](https://pypi.org/project/langchain-openai)) can also use this field to roll out new content formats in a backward-compatible way.

!!! version-added “Added in langchain-core 1.0.0”

profile: ModelProfile | None¶

Profile detailing model capabilities.

!!! warning “Beta feature”

This is a beta feature. The format of model profiles is subject to change.

If not specified, automatically loaded from the provider package on initialization if data is available.

Example profile data includes context window sizes, supported modalities, or support for tool calling, structured output, and other features.

!!! version-added “Added in langchain-core 1.1.0”

client: Any¶: The bedrock runtime client for making data plane API calls

bedrock_client: Any¶: The bedrock client for making control plane API calls

region_name: str | None¶: The aws region e.g., us-west-2. Falls back to AWS_REGION or AWS_DEFAULT_REGION env variable or region specified in ~/.aws/config in case it is not provided here.

credentials_profile_name: str | None¶

The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which has either access keys or role information specified.

If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used.

See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

aws_access_key_id: SecretStr | None¶

AWS access key id.

If provided, aws_secret_access_key must also be provided.

If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used.

See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

If not provided, will be read from AWS_ACCESS_KEY_ID environment variable.

aws_secret_access_key: SecretStr | None¶

AWS secret_access_key.

If provided, aws_access_key_id must also be provided.

If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used.

See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

If not provided, will be read from AWS_SECRET_ACCESS_KEY environment variable.

aws_session_token: SecretStr | None¶

AWS session token.

If provided, aws_access_key_id and aws_secret_access_key must also be provided.

Not required unless using temporary credentials.

See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

If not provided, will be read from AWS_SESSION_TOKEN environment variable.

bedrock_api_key: SecretStr | None¶

Bedrock API key.

Enables authentication using Bedrock API keys instead of standard AWS credentials. When provided, the key is set as the AWS_BEARER_TOKEN_BEDROCK environment variable.

Warning

Because this sets a process-wide environment variable, using api_key is not compatible with multi-tenant deployments where different model instances in the same process need different API keys. Each new client creation overwrites the previous value. Use standard AWS credentials (IAM roles, profiles, etc.) for multi-tenant scenarios.

See: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-use.html

If not provided, will be read from AWS_BEARER_TOKEN_BEDROCK environment variable (if it exists).

If both an API key and AWS credentials are present, the API key takes precedence.

config: Any¶: An optional botocore.config.Config instance to pass to the client.

timeout: int | None¶: Request timeout in seconds. Sets both connect_timeout and read_timeout on the botocore Config. If config is also provided, these values are merged on top of it.

max_retries: int | None¶: Maximum number of retry attempts. Sets retries.max_attempts on the botocore Config. If config is also provided, these values are merged on top of it.

provider: str | None¶: The model provider, e.g., ‘amazon’, ‘cohere’, ‘ai21’, etc. When not supplied, provider is extracted from the first part of the model_id e.g. ‘amazon’ in ‘amazon.titan-text-express-v1’. This value should be provided for model IDs that do not have the provider in them, e.g., custom and provisioned models that have an ARN associated with them.

model_id: str¶: Id of the model to call, e.g., ‘amazon.titan-text-express-v1’, this is equivalent to the modelId property in the list-foundation-models api. For custom and provisioned models, an ARN value is expected.

base_model_id: str | None¶: An optional field to pass the base model id. If provided, this will be used over the value of model_id to identify the base model.

model_kwargs: Dict[str, Any] | None¶: Keyword arguments to pass to the model.

endpoint_url: str | None¶: Needed if you don’t want to default to ‘us-east-1’ endpoint

streaming: bool¶: Whether to stream the results.

provider_stop_sequence_key_name_map: Mapping[str, str]¶

provider_stop_reason_key_map: Mapping[str, str]¶

guardrails: Mapping[str, Any] | None¶

An optional dictionary to configure guardrails for Bedrock.

This field guardrails consists of two keys: ‘guardrailId’ and ‘guardrailVersion’, which should be strings, but are initialized to None.

It’s used to determine if specific guardrails are enabled and properly set.

Type:: Optional[Mapping[str, str]]: A mapping with ‘guardrailId’ and ‘guardrailVersion’ keys.

Example

```python llm = BedrockLLM(model_id=”<model_id>”, client=<bedrock_client>,

model_kwargs={}, guardrails={

“guardrailId”: “<guardrail_id>”, “guardrailVersion”: “<guardrail_version>”})

```

To enable tracing for guardrails, set the ‘trace’ key to True and pass a callback handler to the ‘run_manager’ parameter of the ‘generate’, ‘_call’ methods.

Example

```python llm = BedrockLLM(model_id=”<model_id>”, client=<bedrock_client>,

model_kwargs={}, guardrails={

“guardrailId”: “<guardrail_id>”, “guardrailVersion”: “<guardrail_version>”, “trace”: True},

callbacks=[BedrockAsyncCallbackHandler()])

```

https://python.langchain.com/docs/concepts/callbacks/ for more information on callback handlers.

class BedrockAsyncCallbackHandler(AsyncCallbackHandler):

async def on_llm_error(: self, error: BaseException, **kwargs: Any,
) -> Any:: reason = kwargs.get(“reason”) if reason == “GUARDRAIL_INTERVENED”:

…Logic to handle guardrail intervention…

temperature: float | None¶

max_tokens: int | None¶

Maximum number of tokens to generate.

When using Anthropic models with InvokeModel API, if not set, defaults to 1024.

service_tier: Literal['priority', 'default', 'flex', 'reserved'] | None¶

Service tier for model invocation.

Specifies the processing tier type used for serving the request. Supported values are ‘priority’, ‘default’, ‘flex’, and ‘reserved’.

‘priority’: Prioritized processing for lower latency
‘default’: Standard processing tier
‘flex’: Flexible processing tier with lower cost
‘reserved’: Reserved capacity for consistent performance

If not provided, AWS uses the default tier.

cache: BaseCache | bool | None¶

Whether to cache the response.

If True, will use the global cache.
If False, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

verbose: bool¶: Whether to print out response text.

callbacks: Callbacks¶: Callbacks to add to the run trace.

tags: list[str] | None¶: Tags to add to the run trace.

metadata: builtins.dict[str, Any] | None¶: Metadata to add to the run trace.

custom_get_token_ids: Callable[[str], list[int]] | None¶: Optional encoder to use for counting tokens.

name: str | None¶

The name of the Runnable.

Used for debugging and tracing.

class gen_ai_hub.proxy.langchain.amazon.ChatBedrockConverse(*args, name: str | None = None, cache: ~langchain_core.caches.BaseCache | bool | None = None, verbose: bool = <factory>, callbacks: list[~langchain_core.callbacks.base.BaseCallbackHandler] | ~langchain_core.callbacks.base.BaseCallbackManager | None = None, tags: list[str] | None = None, metadata: dict[str, ~typing.Any] | None = None, custom_get_token_ids: ~collections.abc.Callable[[str], list[int]] | None = None, rate_limiter: ~langchain_core.rate_limiters.BaseRateLimiter | None = None, disable_streaming: bool | ~typing.Literal['tool_calling'] = False, output_version: str | None = <factory>, profile: ~langchain_core.language_models.model_profile.ModelProfile | None = None, client: ~typing.Any = None, bedrock_client: ~typing.Any = None, model: str, base_model: str | None = None, system: ~typing.List[str | ~typing.Dict[str, ~typing.Any]] | None = None, max_tokens: int | None = None, stop: ~typing.List[str] | None = None, temperature: float | None = None, top_p: float | None = None, region_name: str | None = None, credentials_profile_name: str | None = None, aws_access_key_id: ~pydantic.types.SecretStr | None = <factory>, aws_secret_access_key: ~pydantic.types.SecretStr | None = <factory>, aws_session_token: ~pydantic.types.SecretStr | None = <factory>, api_key: ~pydantic.types.SecretStr | None = <factory>, provider: str = '', streaming: bool = False, base_url: str | None = None, default_headers: ~typing.Mapping[str, str] | None = None, config: ~typing.Any = None, timeout: int | None = None, max_retries: int | None = None, guardrails: ~typing.Dict[str, ~typing.Any] | None = None, additional_model_request_fields: ~typing.Dict[str, ~typing.Any] | None = None, reasoning_effort: ~typing.Literal['low', 'medium', 'high', 'xhigh', 'max'] | None = None, additional_model_response_field_paths: ~typing.List[str] | None = None, supports_tool_choice_values: ~typing.Sequence[~typing.Literal['auto', 'any', 'tool']] | None = None, performance_config: ~typing.Mapping[str, ~typing.Any] | None = None, service_tier: ~typing.Literal['priority', 'default', 'flex', 'reserved'] | None = None, output_config: ~typing.Dict[str, ~typing.Any] | None = None, request_metadata: ~typing.Dict[str, str] | None = None, guard_last_turn_only: bool = False, raw_blocks: ~typing.List[~typing.Dict[str, ~typing.Any]] | None = None, **kwargs)¶

Bases: AICoreBedrockBaseModel, ChatBedrockConverse

Drop-in replacement for LangChain ChatBedrockConverse.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'populate_by_name': True, 'protected_namespaces': (), 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, **kwargs)¶

Initializes the AICoreBedrockBaseModel with AICore specific parameters.: Extends the constructor of the base class with aicore specific parameters

Parameters:

model_id (str, optional) – the model identifier, defaults to “”
deployment_id (str, optional) – the deployment identifier, defaults to “”
model_name (str, optional) – the model name, defaults to “”
config_id (str, optional) – the configuration identifier, defaults to “”
config_name (str, optional) – the configuration name, defaults to “”
proxy_client (Optional[BaseProxyClient], optional) – the proxy client to use, defaults to None

extract_model_kwargs_parameters(kwargs)¶

Extracts specific parameters from model_kwargs and moves them to the top level of kwargs.

Parameters:: kwargs (Dict) – the input keyword arguments

client: Any¶: The bedrock runtime client for making data plane API calls

bedrock_client: Any¶: The bedrock client for making control plane API calls

model_id: str¶

ID of the model to call.

e.g., “anthropic.claude-3-sonnet-20240229-v1:0”. This is equivalent to the modelID property in the list-foundation-models api. For custom and provisioned models, an ARN value is expected. See https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns for a list of all supported built-in models.

base_model_id: str | None¶: An optional field to pass the base model id. If provided, this will be used over the value of model_id to identify the base model.

system: List[str | Dict[str, Any]] | None¶

Optional list of system prompts for the LLM.

Each entry can be either:

a simple string (for straightforward text-based system prompts), or
a dictionary matching the Converse API system message schema, allowing inclusion of additional fields like guardContent, cachePoint, etc.

Example

system = [

“a simple system prompt”, {

“text”: “another system prompt”, “guardContent”: {“text”: {“text”: “string”}}, “cachePoint”: {“type”: “default”}

]

String inputs will be internally converted to the appropriate message format, while dict entries will be passed through as-is. Any invalid formats will be rejected by the Converse API.

max_tokens: int | None¶: Max tokens to generate.

stop_sequences: List[str] | None¶: Stop generation if any of these substrings occurs.

temperature: float | None¶: Sampling temperature. Must be 0 to 1.

top_p: float | None¶

The percentage of most-likely candidates that are considered for the next token.

Must be 0 to 1.

For example, if you choose a value of 0.8 for topP, the model selects from the top 80% of the probability distribution of tokens that could be next in the sequence.

region_name: str | None¶

The aws region, e.g., us-west-2.

Falls back to AWS_REGION or AWS_DEFAULT_REGION env variable or region specified in ~/.aws/config in case it is not provided here.

credentials_profile_name: str | None¶

The name of the profile in the ~/.aws/credentials or ~/.aws/config files.

Profile should either have access keys or role information specified. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

aws_access_key_id: SecretStr | None¶

AWS access key id.

If provided, aws_secret_access_key must also be provided. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

If not provided, will be read from ‘AWS_ACCESS_KEY_ID’ environment variable.

aws_secret_access_key: SecretStr | None¶

AWS secret_access_key.

If provided, aws_access_key_id must also be provided. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

If not provided, will be read from ‘AWS_SECRET_ACCESS_KEY’ environment variable.

aws_session_token: SecretStr | None¶

AWS session token.

If provided, aws_access_key_id and aws_secret_access_key must also be provided. Not required unless using temporary credentials. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

If not provided, will be read from ‘AWS_SESSION_TOKEN’ environment variable.

bedrock_api_key: SecretStr | None¶

Bedrock API key.

Enables authentication using Bedrock API keys instead of standard AWS credentials. When provided, the key is set as the AWS_BEARER_TOKEN_BEDROCK environment variable.

Warning

See: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-use.html

If not provided, will be read from AWS_BEARER_TOKEN_BEDROCK environment variable (if it exists).

If both an API key and AWS credentials are present, the API key takes precedence.

provider: str¶

The model provider, e.g., amazon, cohere, ai21, etc.

When not supplied, provider is extracted from the first part of the model_id, e.g. ‘amazon’ in ‘amazon.titan-text-express-v1’. This value should be provided for model IDs that do not have the provider in them, like custom and provisioned models that have an ARN associated with them.

streaming: bool¶: Whether to stream the results or not.

endpoint_url: str | None¶: Needed if you don’t want to default to us-east-1 endpoint

default_headers: Mapping[str, str] | None¶: Headers to pass to the Anthropic clients, will be used for every API call.

config: Any¶: An optional botocore.config.Config instance to pass to the client.

timeout: int | None¶: Request timeout in seconds. Sets both connect_timeout and read_timeout on the botocore Config. If config is also provided, these values are merged on top of it.

max_retries: int | None¶: Maximum number of retry attempts. Sets retries.max_attempts on the botocore Config. If config is also provided, these values are merged on top of it.

guardrail_config: Dict[str, Any] | None¶: Configuration information for a guardrail that you want to use in the request.

additional_model_request_fields: Dict[str, Any] | None¶

Additional inference parameters that the model supports.

Parameters beyond the base set of inference parameters that Converse supports in the additionalModelRequestFields field. Keys must match the exact format expected by the target model (e.g., inferenceConfig, not inference_config). Refer to the model’s AWS documentation for supported parameters.

reasoning_effort: Literal['low', 'medium', 'high', 'xhigh', 'max'] | None¶

Reasoning effort level for models that support configurable reasoning.

Translated into the appropriate provider-specific request format based on the model family. Only applied to models whose profile declares supported reasoning_effort_levels; unsupported values raise ValueError.

Explicit values already present in additional_model_request_fields take precedence. If the model has no known reasoning-effort translation, a warning is emitted and the value is ignored.

additional_model_response_field_paths: List[str] | None¶

Additional model parameters field paths to return in the response.

Converse returns the requested fields as a JSON Pointer object in the additionalModelResponseFields field. The following is example JSON for additionalModelResponseFieldPaths.

supports_tool_choice_values: Sequence[Literal['auto', 'any', 'tool']] | None¶

Which types of tool_choice values the model supports.

Inferred if not specified. Inferred as (‘auto’, ‘any’, ‘tool’) if a ‘claude-3’ model is used, (‘auto’, ‘any’) if a ‘mistral-large’ model is used, (‘auto’) if a ‘nova’ model is used, empty otherwise.

performance_config: Mapping[str, Any] | None¶

service_tier: Literal['priority', 'default', 'flex', 'reserved'] | None¶

output_config: Dict[str, Any] | None¶

Output configuration for structured model responses.

Configures native JSON schema output format via the Bedrock outputConfig parameter. Only supported on select models (Claude 4.5+, select open-weight). See https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html

request_metadata: Dict[str, str] | None¶: Key-Value pairs that you can use to filter invocation logs.

guard_last_turn_only: bool¶: Boolean flag for applying the guardrail to only the last turn.

raw_blocks: List[Dict[str, Any]] | None¶

Raw Bedrock message blocks that can be passed in.

LangChain will relay them unchanged, enabling any combination of content block types. This is useful for custom guardrail wrapping.

rate_limiter: BaseRateLimiter | None¶: An optional rate limiter to use for limiting the number of requests.

disable_streaming: bool | Literal['tool_calling']¶

Whether to disable streaming for this model.

If streaming is bypassed, then stream/astream/astream_events will defer to invoke/ainvoke.

If True, will always bypass streaming case.
If ‘tool_calling’, will bypass streaming case only when the model is called
with a tools keyword argument. In other words, LangChain will automatically switch to non-streaming behavior (invoke) only when the tools argument is provided. This offers the best of both worlds.
If False (Default), will always use streaming case if available.

The main reason for this flag is that code might be written using stream and a user may want to swap out a given model for another model whose implementation does not properly support streaming.

output_version: str | None¶

Version of AIMessage output format to store in message content.

Supported values:

‘v0’: provider-specific format in content (can lazily-parse with
content_blocks)
‘v1’: standardized format in content (consistent with content_blocks)

Partner packages (e.g., [langchain-openai](https://pypi.org/project/langchain-openai)) can also use this field to roll out new content formats in a backward-compatible way.

!!! version-added “Added in langchain-core 1.0.0”

profile: ModelProfile | None¶

Profile detailing model capabilities.

!!! warning “Beta feature”

This is a beta feature. The format of model profiles is subject to change.

If not specified, automatically loaded from the provider package on initialization if data is available.

Example profile data includes context window sizes, supported modalities, or support for tool calling, structured output, and other features.

!!! version-added “Added in langchain-core 1.1.0”

cache: BaseCache | bool | None¶

Whether to cache the response.

If True, will use the global cache.
If False, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

verbose: bool¶: Whether to print out response text.

callbacks: Callbacks¶: Callbacks to add to the run trace.

tags: list[str] | None¶: Tags to add to the run trace.

metadata: builtins.dict[str, Any] | None¶: Metadata to add to the run trace.

custom_get_token_ids: Callable[[str], list[int]] | None¶: Optional encoder to use for counting tokens.

name: str | None¶

The name of the Runnable.

Used for debugging and tracing.

class gen_ai_hub.proxy.langchain.amazon.BedrockEmbeddings(*args, client: Any = None, region_name: str | None = None, credentials_profile_name: str | None = None, model_id: str = 'amazon.titan-embed-text-v1', model_kwargs: Dict | None = None, endpoint_url: str | None = None, normalize: bool = False, **kwargs)¶

Bases: AICoreBedrockBaseModel, BedrockEmbeddings

Drop-in replacement for LangChain BedrockEmbeddings.

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'protected_namespaces': ()}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

__init__(*args, **kwargs)¶

Initializes the AICoreBedrockBaseModel with AICore specific parameters.: Extends the constructor of the base class with aicore specific parameters

Parameters:

model_id (str, optional) – the model identifier, defaults to “”
deployment_id (str, optional) – the deployment identifier, defaults to “”
model_name (str, optional) – the model name, defaults to “”
config_id (str, optional) – the configuration identifier, defaults to “”
config_name (str, optional) – the configuration name, defaults to “”
proxy_client (Optional[BaseProxyClient], optional) – the proxy client to use, defaults to None

client: Any¶: Bedrock client.

region_name: str | None¶: The aws region e.g., us-west-2. Fallsback to AWS_DEFAULT_REGION env variable or region specified in ~/.aws/config in case it is not provided here.

credentials_profile_name: str | None¶: The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which has either access keys or role information specified. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

model_id: str¶: Id of the model to call, e.g., amazon.titan-embed-text-v1, this is equivalent to the modelId property in the list-foundation-models api

model_kwargs: Dict | None¶: Keyword arguments to pass to the model.

endpoint_url: str | None¶: Needed if you don’t want to default to us-east-1 endpoint

normalize: bool¶: Whether the embeddings should be normalized to unit vectors

gen_ai_hub.proxy.langchain.amazon.init_chat_model(proxy_client: BaseProxyClient, deployment: Deployment, temperature: float = 0.0, max_tokens: int = 256, top_k: int | None = None, top_p: float = 1.0, stop_sequences: List[str] = None, model_id: str | None = '', config: Config | None = None)¶

Initializes a chat model using the legacy Bedrock Invoke API (ChatBedrock).

Parameters:

proxy_client (BaseProxyClient) – the proxy client to use
deployment (Deployment) – the deployment information
temperature (float, optional) – the temperature for the model, defaults to 0.0
max_tokens (int, optional) – the maximum number of tokens to generate, defaults to 256
top_k (Optional[int], optional) – the top-k sampling parameter, defaults to None
top_p (float, optional) – the top-p sampling parameter, defaults to 1.0
stop_sequences (List[str], optional) – the stop sequences for the model, defaults to None
model_id (Optional[str], optional) – the model identifier, defaults to ‘’
config (Optional[Config], optional) – the botocore configuration, defaults to None

Returns:

the initialized chat model

Return type:

ChatBedrock

gen_ai_hub.proxy.langchain.amazon.init_chat_converse_model(proxy_client: BaseProxyClient, deployment: Deployment, temperature: float = 0.0, max_tokens: int = 256, top_k: int | None = None, top_p: float = 1.0, stop_sequences: List[str] = None, model_id: str | None = '', config: Config | None = None)¶

Initializes a chat model using the newer Bedrock Converse API (ChatBedrockConverse). The Converse API offers several advantages over the older Invoke API:

Unified interface for different models and modalities.
Native support for tool use (function calling).
Standardized request/response structure.

Parameters:

proxy_client (BaseProxyClient) – the proxy client to use
deployment (Deployment) – the deployment information
temperature (float, optional) – the temperature for the model, defaults to 0.0
max_tokens (int, optional) – the maximum number of tokens to generate, defaults to 256
top_k (Optional[int], optional) – the top-k sampling parameter, defaults to None
top_p (float, optional) – the top-p sampling parameter, defaults to 1.0
stop_sequences (List[str], optional) – the stop sequences for the model, defaults to None
model_id (Optional[str], optional) – the model identifier, defaults to ‘’
config (Optional[Config], optional) – the botocore configuration, defaults to None

Returns:

the initialized chat model

Return type:

ChatBedrockConverse

gen_ai_hub.proxy.langchain.amazon.init_embedding_model(proxy_client: BaseProxyClient, deployment: Deployment, model_id: str | None = '')¶

Initializes an embedding model using BedrockEmbeddings.

Parameters:

proxy_client (BaseProxyClient) – the proxy client to use
deployment (Deployment) – the deployment information
model_id (Optional[str], optional) – the model identifier, defaults to ‘’

Returns:

the initialized embedding model

Return type:

BedrockEmbeddings