gen_ai_hub.orchestration_v2 package¶

Bases: ABCBaseModel

Specific filter configuration for Azure Content Safety.

This class configures content filtering based on Azure’s categories and severity levels. It allows setting thresholds for hate speech, sexual content, violence, and self-harm content.

Parameters:

hate – Threshold for hate speech content.
sexual – Threshold for sexual content.
violence – Threshold for violent content.
self_harm – Threshold for self-harm content.
prompt_shield – A flag to use prompt shield

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

hate: AzureThreshold | Literal[0, 2, 4, 6] | None¶

sexual: AzureThreshold | Literal[0, 2, 4, 6] | None¶

violence: AzureThreshold | Literal[0, 2, 4, 6] | None¶

self_harm: AzureThreshold | Literal[0, 2, 4, 6] | None¶

Bases: AzureContentFilter

Filter configuration for Azure Content Safety Input

Parameters:

hate – Threshold for hate speech content.
sexual – Threshold for sexual content.
violence – Threshold for violent content.
self_harm – Threshold for self-harm content.
prompt_shield – A flag to use prompt shield

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_shield: bool | None¶

Bases: AzureContentFilter

Filter configuration for Azure Content Safety Output

Parameters:

hate – Threshold for hate speech content.
sexual – Threshold for sexual content.
violence – Threshold for violent content.
self_harm – Threshold for self-harm content.
protected_material_code – Detect protected code content from known GitHub repositories. The scan includes software libraries, source code, algorithms, and other proprietary programming content.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

protected_material_code: bool | None¶

class gen_ai_hub.orchestration_v2.AzureThreshold(*values)¶

Bases: int, Enum

Enumerates the threshold levels for the Azure Content Safety service.

This enum defines the various threshold levels that can be used to filter content based on its safety score. Each threshold value represents a specific level of content moderation.

Values:: ALLOW_SAFE: Allows only Safe content. ALLOW_SAFE_LOW: Allows Safe and Low content. ALLOW_SAFE_LOW_MEDIUM: Allows Safe, Low, and Medium content. ALLOW_ALL: Allows all content (Safe, Low, Medium, and High).

ALLOW_SAFE = 0¶

ALLOW_SAFE_LOW = 2¶

ALLOW_SAFE_LOW_MEDIUM = 4¶

ALLOW_ALL = 6¶

class gen_ai_hub.orchestration_v2.ModuleConfig(*, prompt_templating: PromptTemplatingModuleConfig, filtering: FilteringModuleConfig | None = None, masking: MaskingModuleConfig | None = None, grounding: GroundingModuleConfig | None = None, translation: TranslationModuleConfig | None = None)¶

Bases: ABCBaseModel

Configuration for the Orchestration Service’s content generation process.

Defines modules for a harmonized API that combines LLM-based content generation with additional processing functionalities.

The orchestration service allows for advanced content generation by processing inputs through a series of steps: template rendering, text generation via LLMs, and optional input/output transformations such as data masking or filtering.

Parameters:

prompt_templating – Template object for rendering input prompts and language model for text generation.
filtering – Module for filtering and validating input/output content.
masking – Module for anonymizing or pseudonymizing sensitive information.
grounding – Module for document grounding.
translation – Module for translating input and output content.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_templating: PromptTemplatingModuleConfig¶

filtering: FilteringModuleConfig | None¶

masking: MaskingModuleConfig | None¶

grounding: GroundingModuleConfig | None¶

translation: TranslationModuleConfig | None¶

class gen_ai_hub.orchestration_v2.OrchestrationConfig(*, modules: ModuleConfig | Annotated[List[ModuleConfig], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])], stream: GlobalStreamOptions | None = None)¶

Bases: ABCBaseModel

Configuration for the Orchestration Service’s content generation process.

Parameters:

modules – Either a single ModuleConfig or a list of ModuleConfigs. When a list is provided,
succeeds. (the orchestration service will try each configuration in order until one)
stream – Optional streaming configuration.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

modules: ModuleConfig | Annotated[List[ModuleConfig], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]¶

stream: GlobalStreamOptions | None¶

class gen_ai_hub.orchestration_v2.CompletionRequestConfigurationReferenceByIdConfigRef(*, id: str)¶

Bases: ABCBaseModel

Represents a reference to an orchestration config identified by a unique ID.

Parameters:: id (str) – The unique identifier for the configuration.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

id: str¶

class gen_ai_hub.orchestration_v2.CompletionRequestConfigurationReferenceByNameScenarioVersionConfigRef(*, scenario: str, name: str, version: str)¶

Bases: ABCBaseModel

Represents a reference to aan orchestration config identified by name, scenario, and version.

Parameters:

scenario (str) – Scenario name
name (str) – Name of config
version (str) – Version of config

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

scenario: str¶

name: str¶

version: str¶

class gen_ai_hub.orchestration_v2.ContentFilterProvider(*values)¶

Bases: str, Enum

Enumerates supported content filter providers.

This enum defines the available content filtering services that can be used for content moderation tasks. Each enum value represents a specific provider.

Values:

AZURE: Represents the Azure Content Safety service.

LLAMA_GUARD_3_8B: Represents the Llama Guard 3 based on Llama-3.1-8B pretrained model.

AZURE = 'azure_content_safety'¶

LLAMA_GUARD_3_8B = 'llama_guard_3_8b'¶

class gen_ai_hub.orchestration_v2.ContentFilter(*, type: ContentFilterProvider, config: AzureContentFilter | LlamaGuard38bFilter | None = None)¶

Bases: ABCBaseModel

Base class for content filtering configurations.

This class provides a generic structure for defining content filters from various providers. It allows for specifying the provider and associated configuration parameters.

Parameters:

type – The name of the content filter provider.
config – A dictionary containing the configuration parameters for the content filter.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider¶

config: AzureContentFilter | LlamaGuard38bFilter | None¶

class gen_ai_hub.orchestration_v2.LlamaGuard38bFilterConfig(*, type: ContentFilterProvider = ContentFilterProvider.LLAMA_GUARD_3_8B, config: LlamaGuard38bFilter)¶

Bases: ContentFilter

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider¶

config: LlamaGuard38bFilter¶

class gen_ai_hub.orchestration_v2.AzureContentSafetyInputFilterConfig(*, type: ContentFilterProvider = ContentFilterProvider.AZURE, config: AzureContentSafetyInput | None = None)¶

Bases: ContentFilter

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider¶

config: AzureContentSafetyInput | None¶

class gen_ai_hub.orchestration_v2.AzureContentSafetyOutputFilterConfig(*, type: ContentFilterProvider = ContentFilterProvider.AZURE, config: AzureContentSafetyOutput | None = None)¶

Bases: ContentFilter

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider¶

config: AzureContentSafetyOutput | None¶

class gen_ai_hub.orchestration_v2.FilteringStreamOptions(*, overlap: Annotated[int | None, Ge(ge=0), Le(le=10000)] = 0)¶

Bases: ABCBaseModel

overlap: Number of characters that should be additionally sent to content filtering services from previous chunks as additional context.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

overlap: int | None¶

class gen_ai_hub.orchestration_v2.InputFiltering(*, filters: Annotated[List[AzureContentSafetyInputFilterConfig | LlamaGuard38bFilterConfig | ContentFilter], MinLen(min_length=1)])¶

Bases: ABCBaseModel

Module for managing and applying input content filters.

Parameters:: filters – List of ContentFilter objects to be applied to input content.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

filters: List[AzureContentSafetyInputFilterConfig | LlamaGuard38bFilterConfig | ContentFilter]¶

class gen_ai_hub.orchestration_v2.OutputFiltering(*, filters: Annotated[List[AzureContentSafetyOutputFilterConfig | LlamaGuard38bFilterConfig | ContentFilter], MinLen(min_length=1)], stream_options: FilteringStreamOptions | None = None)¶

Bases: ABCBaseModel

Module for managing and applying output content filters.

Parameters:

filters – List of ContentFilter objects to be applied to output content.
stream_options – Module-specific streaming options.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

filters: List[AzureContentSafetyOutputFilterConfig | LlamaGuard38bFilterConfig | ContentFilter]¶

stream_options: FilteringStreamOptions | None¶

class gen_ai_hub.orchestration_v2.FilteringModuleConfig(*, input: InputFiltering | None = None, output: OutputFiltering | None = None)¶

Bases: ABCBaseModel

Module for managing and applying content filters.

Parameters:

input – Module for filtering and validating input content before processing.
output – Module for filtering and validating output content after generation.

classmethod enforce_min_properties(values)¶: Ensure at least one of input or output filtering is provided.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

input: InputFiltering | None¶

output: OutputFiltering | None¶

class gen_ai_hub.orchestration_v2.DataMaskingProviderName(*values)¶

Bases: str, Enum

Enumerates the available data masking providers.

This enum defines the supported providers for masking sensitive data in the LLM module.

Values: SAP_DATA_PRIVACY_INTEGRATION: Refers to the SAP Data Privacy Integration service, which offers anonymization and pseudonymization capabilities for sensitive data.

SAP_DATA_PRIVACY_INTEGRATION = 'sap_data_privacy_integration'¶

class gen_ai_hub.orchestration_v2.MaskingMethod(*values)¶

Bases: str, Enum

Enumerates the supported masking methods.

This enum defines the two main methods for masking sensitive information: anonymization and pseudonymization. Anonymization irreversibly removes sensitive data, while pseudonymization allows the original data to be recovered.

Values:

ANONYMIZATION: Irreversibly replaces sensitive data with placeholders (e.g., MASKED_ENTITY).

PSEUDONYMIZATION: Replaces sensitive data with reversible placeholders (e.g., MASKED_ENTITY_ID).

ANONYMIZATION = 'anonymization'¶

PSEUDONYMIZATION = 'pseudonymization'¶

class gen_ai_hub.orchestration_v2.ProfileEntity(*values)¶

Bases: str, Enum

Enumerates the entity categories that can be masked by the SAP Data Privacy Integration service.

This enum lists different types of personal or sensitive information (PII) that can be detected and masked by the data masking module, such as personal details, organizational data, contact information, and identifiers.

Values:

PERSON: Represents personal names.

ORG: Represents organizational names.

UNIVERSITY: Represents educational institutions.

LOCATION: Represents geographical locations.

EMAIL: Represents email addresses.

PHONE: Represents phone numbers.

ADDRESS: Represents physical addresses.

SAP_IDS_INTERNAL: Represents internal SAP identifiers.

SAP_IDS_PUBLIC: Represents public SAP identifiers.

URL: Represents URLs.

USERNAME_PASSWORD: Represents usernames and passwords.

NATIONAL_ID: Represents national identification numbers.

IBAN: Represents International Bank Account Numbers.

SSN: Represents Social Security Numbers.

CREDIT_CARD_NUMBER: Represents credit card numbers.

PASSPORT: Represents passport numbers.

DRIVING_LICENSE: Represents driving license numbers.

NATIONALITY: Represents nationality information.

RELIGIOUS_GROUP: Represents religious group affiliation.

POLITICAL_GROUP: Represents political group affiliation.

PRONOUNS_GENDER: Represents pronouns and gender identity.

GENDER: Represents gender information.

SEXUAL_ORIENTATION: Represents sexual orientation.

TRADE_UNION: Represents trade union membership.

SENSITIVE_DATA: Represents any other sensitive information.

PERSON = 'profile-person'¶

ORG = 'profile-org'¶

UNIVERSITY = 'profile-university'¶

LOCATION = 'profile-location'¶

EMAIL = 'profile-email'¶

PHONE = 'profile-phone'¶

ADDRESS = 'profile-address'¶

SAP_IDS_INTERNAL = 'profile-sapids-internal'¶

SAP_IDS_PUBLIC = 'profile-sapids-public'¶

URL = 'profile-url'¶

USERNAME_PASSWORD = 'profile-username-password'¶

NATIONAL_ID = 'profile-nationalid'¶

IBAN = 'profile-iban'¶

SSN = 'profile-ssn'¶

CREDIT_CARD_NUMBER = 'profile-credit-card-number'¶

PASSPORT = 'profile-passport'¶

DRIVING_LICENSE = 'profile-driverlicense'¶

NATIONALITY = 'profile-nationality'¶

RELIGIOUS_GROUP = 'profile-religious-group'¶

POLITICAL_GROUP = 'profile-political-group'¶

PRONOUNS_GENDER = 'profile-pronouns-gender'¶

GENDER = 'profile-gender'¶

SEXUAL_ORIENTATION = 'profile-sexual-orientation'¶

TRADE_UNION = 'profile-trade-union'¶

SENSITIVE_DATA = 'profile-sensitive-data'¶

ETHNICITY = 'profile-ethnicity'¶

class gen_ai_hub.orchestration_v2.DPIMethodConstant(*, method: str = 'constant', value: str)¶

Bases: ABCBaseModel

Replaces the entity with the specified value followed by an incrementing number

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

method: str¶

value: str¶

class gen_ai_hub.orchestration_v2.DPIMethodFabricatedData(*, method: str = 'fabricated_data')¶

Bases: ABCBaseModel

Replaces the entity with a randomly generated value appropriate to its type.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

method: str¶

class gen_ai_hub.orchestration_v2.DPICustomEntity(*, regex: str, replacement_strategy: DPIMethodConstant)¶

Bases: ABCBaseModel

regex: Regular expression to match the entity replacement_strategy: Replacement strategy to be used for the entity

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

regex: str¶

replacement_strategy: DPIMethodConstant¶

class gen_ai_hub.orchestration_v2.DPIStandardEntity(*, type: ProfileEntity, replacement_strategy: DPIMethodConstant | DPIMethodFabricatedData | None = None)¶

Bases: ABCBaseModel

type: Standard entity type to be masked replacement_strategy: Replacement strategy to be used for the entity

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ProfileEntity¶

replacement_strategy: DPIMethodConstant | DPIMethodFabricatedData | None¶

class gen_ai_hub.orchestration_v2.MaskGroundingInput(*, enabled: bool = False)¶

Bases: ABCBaseModel

Controls whether the input to the grounding module will be masked with the configuration supplied in the masking module

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

enabled: bool¶

class gen_ai_hub.orchestration_v2.MaskingProviderConfig(*, type: DataMaskingProviderName = DataMaskingProviderName.SAP_DATA_PRIVACY_INTEGRATION, method: MaskingMethod, entities: List[DPIStandardEntity | DPICustomEntity], allowlist: List[str] | None = None, mask_grounding_input: MaskGroundingInput | None = None)¶

Bases: ABCBaseModel

SAP Data Privacy Integration provider for data masking.

This class implements the SAP Data Privacy Integration service, which can anonymize or pseudonymize specified entity categories in the input data. It supports masking sensitive information like personal names, contact details, and identifiers.

Parameters:

method – The method of masking to apply (anonymization or pseudonymization).
entities – A list of entity categories to be masked, such as names, locations, or emails.
allowlist – A list of strings that should not be masked.
mask_grounding_input – A flag indicating whether to mask input to the grounding module.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: DataMaskingProviderName¶

method: MaskingMethod¶

entities: List[DPIStandardEntity | DPICustomEntity]¶

allowlist: List[str] | None¶

mask_grounding_input: MaskGroundingInput | None¶

class gen_ai_hub.orchestration_v2.MaskingModuleConfig(*, providers: Annotated[List[MaskingProviderConfig] | None, MinLen(min_length=1)] = None, masking_providers: Annotated[List[MaskingProviderConfig] | None, MinLen(min_length=1)] = None)¶

Bases: ABCBaseModel

Configuration for the data masking module.

Parameters:

providers – list of masking service provider configurations
masking_providers – list of masking provider configurations

IMPORTANT: use exactly one of the parameters to set the list of masking provider configurations. DEPRECATED: parameter ‘masking_providers’ will be removed Sept 15, 2026. Use ‘providers’ instead.

enforce_exactly_one_provider_list()¶

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

providers: List[MaskingProviderConfig] | None¶

masking_providers: List[MaskingProviderConfig] | None¶

class gen_ai_hub.orchestration_v2.GroundingType(*values)¶

Bases: str, Enum

Enumerates supported grounding types.

DOCUMENT_GROUNDING_SERVICE = 'document_grounding_service'¶

class gen_ai_hub.orchestration_v2.DataRepositoryType(*values)¶

Bases: str, Enum

Enumerates data repository types.

VECTOR = 'vector'¶

URL = 'help.sap.com'¶

class gen_ai_hub.orchestration_v2.DocumentGroundingFilter(*, id: str | None = None, data_repository_type: DataRepositoryType | Literal['vector', 'help.sap.com'], search_config: GroundingSearchConfig | None = None, data_repositories: List[str] | None = None, data_repository_metadata: List[KeyValueListPair] | None = None, document_metadata: List[DocumentMetadataKeyValueListPairs] | None = None, chunk_metadata: List[KeyValueListPair] | None = None)¶

Bases: ABCBaseModel

Module for configuring document grounding filters.

Parameters:

id – The unique identifier for the grounding filter.
search_config – GroundingSearchConfig object.
data_repository_type – Only include DataRepositories with the given type: vector, help.sap.com.
data_repositories – list of data repositories to search. Specify [‘*’] to search across all DataRepositories or give a specific list of DataRepository ids.
data_repository_metadata – The metadata for the data repository. Restrict DataRepositories considered during search to those annotated with the given metadata. Useful when combined with dataRepositories=[‘*’]
document_metadata – DocumentMetadata object.
chunk_metadata – Restrict chunks considered during search to those with the given metadata.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

id: str | None¶

data_repository_type: DataRepositoryType | Literal['vector', 'help.sap.com']¶

search_config: GroundingSearchConfig | None¶

data_repositories: List[str] | None¶

data_repository_metadata: List[KeyValueListPair] | None¶

document_metadata: List[DocumentMetadataKeyValueListPairs] | None¶

chunk_metadata: List[KeyValueListPair] | None¶

class gen_ai_hub.orchestration_v2.DocumentGroundingPlaceholders(*, input: Annotated[List[str], MinLen(min_length=1)], output: str)¶

Bases: ABCBaseModel

input: The list of input parameters used for grounding input questions (minItems: 1). output: Parameter name used for grounding output.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

input: List[str]¶

output: str¶

class gen_ai_hub.orchestration_v2.DocumentGroundingConfig(*, filters: List[DocumentGroundingFilter] | None = None, placeholders: DocumentGroundingPlaceholders, metadata_params: list[str] | None = None)¶

Bases: ABCBaseModel

defines the detailed configuration for the Grounding module.

Parameters:

filters – List of DocumentGroundingFilter objects.
placeholders – Placeholders to be used for grounding input questions and output.
metadata_params – Parameter name used for specifying metadata parameters.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

filters: List[DocumentGroundingFilter] | None¶

placeholders: DocumentGroundingPlaceholders¶

metadata_params: list[str] | None¶

class gen_ai_hub.orchestration_v2.GroundingModuleConfig(*, type: GroundingType = GroundingType.DOCUMENT_GROUNDING_SERVICE, config: DocumentGroundingConfig)¶

Bases: ABCBaseModel

Module for managing and applying grounding aka RAG configurations.

Parameters:

type – The type of the grounding module.
config – Configuration dictionary for the grounding module.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type: GroundingType¶

config: DocumentGroundingConfig¶

class gen_ai_hub.orchestration_v2.KeyValueListPair(*, key: str, value: List[str])¶

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

key: str¶

value: List[str]¶

class gen_ai_hub.orchestration_v2.DocumentMetadataKeyValueListPairs(*, key: str, value: List[str], select_mode: List[Literal['ignoreIfKeyAbsent']] | None = None)¶

Bases: KeyValueListPair

Restrict documents considered during search to those annotated with the given metadata.

Parameters:

key – The key for the metadata.
value – The list of values for the metadata.
select_mode – Select mode for search filters.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

select_mode: List[Literal['ignoreIfKeyAbsent']] | None¶

class gen_ai_hub.orchestration_v2.GroundingSearchConfig(*, max_chunk_count: Annotated[int | None, Gt(gt=0)] = None, max_document_count: Annotated[int | None, Gt(gt=0)] = None)¶

Bases: ABCBaseModel

Search configuration for the data repository.

Parameters:

max_chunk_count(int – 0, exclusiveMinimum: true): Maximum number of chunks to be returned.
minimum – 0, exclusiveMinimum: true): Maximum number of chunks to be returned.
'maxDocumentCount'. (Cannot be used with)
max_document_count(int – 0, exclusiveMinimum: true): [Only supports ‘vector’ dataRepositoryType]
minimum – 0, exclusiveMinimum: true): [Only supports ‘vector’ dataRepositoryType]
'maxChunkCount'. (- Maximum number of documents to be returned. Cannot be used with)
given (If maxDocumentCount is)
returned. (then only one chunk per document is)

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

validate_max_chunk_count_and_max_document_count()¶

max_chunk_count: int | None¶

max_document_count: int | None¶

class gen_ai_hub.orchestration_v2.EmbeddingsEncodingFormat(*values)¶

Bases: str, Enum

Encoding format for the embeddings output.

Values:: FLOAT: Returns embeddings as an array of floats. BASE64: Returns embeddings as a base64 encoded string. BINARY: Returns embeddings in binary format.

FLOAT = 'float'¶

BASE64 = 'base64'¶

BINARY = 'binary'¶

class gen_ai_hub.orchestration_v2.EmbeddingsInputType(*values)¶

Bases: str, Enum

Type hint for the embedding model about the purpose of the text.

Some models use asymmetric embeddings for better search performance.

Values:: TEXT: General purpose text (default). DOCUMENT: Content to be searched/retrieved. QUERY: Short search queries.

TEXT = 'text'¶

DOCUMENT = 'document'¶

QUERY = 'query'¶

class gen_ai_hub.orchestration_v2.EmbeddingsModelParams(*, dimensions: int | None = None, encoding_format: EmbeddingsEncodingFormat | None = None, normalize: bool | None = None)¶

Bases: ABCBaseModel

Additional parameters for generating embeddings.

Parameters:

dimensions – The number of dimensions for the output embeddings.
encoding_format – The format for the embeddings output (float, base64, or binary).
normalize – Whether to normalize the embeddings.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

dimensions: int | None¶

encoding_format: EmbeddingsEncodingFormat | None¶

normalize: bool | None¶

class gen_ai_hub.orchestration_v2.EmbeddingsModelDetails(*, name: str, version: str | None = 'latest', params: EmbeddingsModelParams | None = None, timeout: Annotated[int | None, Ge(ge=1), Le(le=600)] = 600, max_retries: Annotated[int | None, Ge(ge=0), Le(le=5)] = 2)¶

Bases: ABCBaseModel

The model and parameters to be used for generating embeddings.

Parameters:

name – Name of the embedding model.
version – Version of the model to be used. Defaults to “latest”.
params – Additional parameters for the model (dimensions, encoding_format, normalize).
timeout – Timeout for the embeddings request in seconds. Ignored for Vertex AI models.
max_retries – Maximum number of retries. Ignored for Vertex AI models.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str¶

version: str | None¶

params: EmbeddingsModelParams | None¶

timeout: int | None¶

max_retries: int | None¶

class gen_ai_hub.orchestration_v2.EmbeddingsModelConfig(*, model: EmbeddingsModelDetails)¶

Bases: ABCBaseModel

Configuration for the embeddings model.

Parameters:: model – The embedding model details.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model: EmbeddingsModelDetails¶

class gen_ai_hub.orchestration_v2.EmbeddingsModuleConfigs(*, embeddings: EmbeddingsModelConfig, masking: MaskingModuleConfig | None = None)¶

Bases: ABCBaseModel

Module configurations for the embeddings endpoint.

Parameters:

embeddings – Required configuration for the embeddings model.
masking – Optional configuration for data masking before embedding.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

embeddings: EmbeddingsModelConfig¶

masking: MaskingModuleConfig | None¶

class gen_ai_hub.orchestration_v2.EmbeddingsOrchestrationConfig(*, modules: EmbeddingsModuleConfigs)¶

Bases: ABCBaseModel

Configuration for the Embeddings Orchestration endpoint.

Parameters:: modules – The module configurations including embeddings model and optional masking.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

modules: EmbeddingsModuleConfigs¶

class gen_ai_hub.orchestration_v2.EmbeddingsInput(*, text: str | List[str], type: EmbeddingsInputType | None = None)¶

Bases: ABCBaseModel

Input for the embeddings endpoint.

Parameters:

text – The text to embed. Can be a single string or a list of strings.
type – Optional type hint for the embedding model (text, document, or query).

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

text: str | List[str]¶

type_: EmbeddingsInputType | None¶

class gen_ai_hub.orchestration_v2.EmbeddingsUsage(*, prompt_tokens: int, total_tokens: int)¶

Bases: ABCBaseModel

Token usage information for the embeddings request.

Parameters:

prompt_tokens – The number of tokens used by the prompt.
total_tokens – The total number of tokens used by the request.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_tokens: int¶

total_tokens: int¶

class gen_ai_hub.orchestration_v2.EmbeddingResult(*, object: str = 'embedding', embedding: List[float] | str, index: int)¶

Bases: ABCBaseModel

A single embedding result.

Parameters:

object – The object type, always “embedding”.
embedding – The embedding vector (array of floats) or base64 string.
index – The index of this embedding in the list.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

object: str¶

embedding: List[float] | str¶

index: int¶

class gen_ai_hub.orchestration_v2.EmbeddingsResponse(*, object: str = 'list', data: List[EmbeddingResult], model: str, usage: EmbeddingsUsage)¶

Bases: ABCBaseModel

The response from the embedding model, following OpenAI specification.

Parameters:

object – The object type, always “list”.
data – The list of embeddings generated by the model.
model – The name of the model used to generate the embeddings.
usage – Token usage information.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

object: str¶

data: List[EmbeddingResult]¶

model: str¶

usage: EmbeddingsUsage¶

class gen_ai_hub.orchestration_v2.EmbeddingsPostResponse(*, request_id: str, intermediate_results: Dict | None = None, final_result: EmbeddingsResponse)¶

Bases: ABCBaseModel

Response for an embeddings POST request.

Parameters:

request_id – Unique identifier for the request.
intermediate_results – Optional results from intermediate modules (e.g., masking).
final_result – The embeddings response from the model.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str¶

intermediate_results: Dict | None¶

final_result: EmbeddingsResponse¶

class gen_ai_hub.orchestration_v2.EmbeddingsRequest(*, config: EmbeddingsOrchestrationConfig, input: EmbeddingsInput)¶

Bases: ABCBaseModel

Request body for the embeddings endpoint.

Parameters:

config – The embeddings orchestration configuration.
input – The input text to embed.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

config: EmbeddingsOrchestrationConfig¶

input: EmbeddingsInput¶

class gen_ai_hub.orchestration_v2.LlamaGuard38bFilter(*, violent_crimes: bool = False, non_violent_crimes: bool = False, sex_crimes: bool = False, child_exploitation: bool = False, defamation: bool = False, specialized_advice: bool = False, privacy: bool = False, intellectual_property: bool = False, indiscriminate_weapons: bool = False, hate: bool = False, self_harm: bool = False, sexual_content: bool = False, elections: bool = False, code_interpreter_abuse: bool = False)¶

Bases: ABCBaseModel

Specific implementation of ContentFilter for Llama Guard 3. Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification.

Parameters:

violent_crimes – Responses that enable, encourage, or endorse the commission of violent crimes.
non_violent_crimes – Responses that enable, encourage, or endorse the commission of non-violent crimes.
sex_crimes – Responses that enable, encourage, or endorse the commission of sex-related crimes.
child_exploitation – Responses that contain, describe, enable or endorse the sexual abuse of children.
defamation – Responses that are both verifiably false and likely to injure a living person’s reputation.
specialized_advice – Responses that contain specialized financial, medical or legal advice.
privacy – Responses that contain sensitive or nonpublic personal information.
intellectual_property – Responses that may violate the intellectual property rights of any third party.
indiscriminate_weapons – Responses that enable, encourage, or endorse the creation of indiscriminate weapons.
hate – Responses that demean or dehumanize people on the basis of their sensitive, personal characteristics.
self_harm – Responses that enable, encourage, or endorse acts of intentional self-harm.
sexual_content – Responses that contain erotica.
elections – Responses that contain factually incorrect information about electoral systems and processes.
code_interpreter_abuse – Responses that seek to abuse code interpreters.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

violent_crimes: bool¶

non_violent_crimes: bool¶

sex_crimes: bool¶

child_exploitation: bool¶

defamation: bool¶

specialized_advice: bool¶

privacy: bool¶

intellectual_property: bool¶

indiscriminate_weapons: bool¶

hate: bool¶

self_harm: bool¶

sexual_content: bool¶

elections: bool¶

code_interpreter_abuse: bool¶

class gen_ai_hub.orchestration_v2.LLMModelDetails(*, name: str, version: str | None = 'latest', params: Dict | None = None, timeout: Annotated[int | None, Ge(ge=1), Le(le=600)] = 600, max_retries: Annotated[int | None, Ge(ge=0), Le(le=5)] = 2)¶

Bases: ABCBaseModel

The model and parameters to be used for the prompt templating. This is the model that will be used to generate the response.

Parameters:

name – Name of the model as in LLM Access configuration.
version – Version of the model to be used. Defaults to “latest”.
params – Additional parameters for the model. Default values are used for mandatory parameters.
timeout – Timeout for the LLM request in seconds. This parameter is currently ignored for Vertex AI models.
max_retries – Maximum number of retries for the LLM request. This parameter is currently ignored for Vertex AI models.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str¶

version: str | None¶

params: Dict | None¶

timeout: int | None¶

max_retries: int | None¶

class gen_ai_hub.orchestration_v2.SystemMessage(*, role: Role = Role.SYSTEM, content: str | List[TextPart])¶

Bases: ABCBaseModel

Represents a system message in a prompt or conversation template.

System messages typically provide context or instructions to the AI model.

Parameters:

role – The role of the entity sending the message.
content – The text content of the system message.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role¶

content: str | List[TextPart]¶

Bases: ABCBaseModel

Represents a user message in a prompt or conversation template.

User messages typically contain queries or inputs from the user.

Parameters:

role – The role of the entity sending the message.
content – The message content, which may be plain text or a sequence of text and images.

classmethod content_validation(content)¶: Validates and maps the content field to the appropriate types.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role¶

content: str | TextPart | ImagePart | List[str | TextPart | ImagePart | ImageItem]¶

class gen_ai_hub.orchestration_v2.AssistantMessage(*, role: Role = Role.ASSISTANT, content: str | List[TextPart] | None = None, refusal: str | None = None, tool_calls: List[MessageToolCall] | None = None)¶

Bases: ABCBaseModel

Represents an assistant message in a prompt or conversation template.

Assistant messages typically contain responses or outputs from the AI model.

Parameters:

role – The role of the entity sending the message.
content – The text content of the assistant message.
refusal – A string indicating refusal reason.
tool_calls – A list of tool call objects.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role¶

content: str | List[TextPart] | None¶

refusal: str | None¶

tool_calls: List[MessageToolCall] | None¶

class gen_ai_hub.orchestration_v2.ToolChatMessage(*, role: Role = Role.TOOL, tool_call_id: str, content: str | List[TextPart])¶

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role¶

tool_call_id: str¶

content: str | List[TextPart]¶

class gen_ai_hub.orchestration_v2.DeveloperChatMessage(*, role: Role = Role.DEVELOPER, content: str | List[TextPart])¶

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role¶

content: str | List[TextPart]¶

class gen_ai_hub.orchestration_v2.ResponseChatMessage(*, role: Role = Role.ASSISTANT, content: str, refusal: str | None = None, tool_calls: List[MessageToolCall] | None = None)¶

Bases: ABCBaseModel

Represents a response message in a conversation.

Parameters:

role – The role of the entity sending the message.
content – The text content of the assistant message.
refusal – A string indicating refusal reason.
tool_calls – A list of tool call objects.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role¶

content: str¶

refusal: str | None¶

tool_calls: List[MessageToolCall] | None¶

class gen_ai_hub.orchestration_v2.FunctionCall(*, name: str, arguments: str)¶

Bases: ABCBaseModel

Represents a function call with its name and arguments.

name¶

str The name of the function to call.

Type:: str

arguments¶

str The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.

Type:: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

parse_arguments() → dict¶

Parses the arguments string as JSON.

Returns:: A dictionary representing the parsed arguments.
Return type:: dict

name: str¶

arguments: str¶

class gen_ai_hub.orchestration_v2.MessageToolCall(*, id: str, type: Literal['function'] = 'function', function: FunctionCall)¶

Bases: ABCBaseModel

The tool calls generated by the model, such as function calls.

id¶

The ID of the tool call.

Type:: str

type¶

The type of the tool. Currently, only function is supported.

Type:: Literal[‘function’]

function¶

The function that the model called.

Type:: gen_ai_hub.orchestration_v2.models.message.FunctionCall

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

id: str¶

type: Literal['function']¶

function: FunctionCall¶

class gen_ai_hub.orchestration_v2.ImageDetailLevel(*values)¶

Bases: Enum

Controls the resolution and detail level for image analysis.

AUTO¶: The model determines the detail level automatically.

LOW¶: The model uses a low-fidelity, faster version of the image.

HIGH¶: The model uses a high-fidelity version of the image.

AUTO = 'auto'¶

LOW = 'low'¶

HIGH = 'high'¶

class gen_ai_hub.orchestration_v2.TextPart(*, text: str, type: Literal['text'] = 'text')¶

Bases: ABCBaseModel

Represents a text segment within a multimodal content block.

Parameters:

text – The string content of the text part.
type – The type identifier, defaulting to “text”.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

text: str¶

type_: Literal['text']¶

class gen_ai_hub.orchestration_v2.ImageUrl(*, url: str, detail: ImageDetailLevel | None = None)¶

Bases: ABCBaseModel

A data structure holding the URL and detail level for an image.

Parameters:

url – The location of the image, as a standard or data URL.
detail – The processing detail level for the image.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

url: str¶

detail: ImageDetailLevel | None¶

class gen_ai_hub.orchestration_v2.ImagePart(*, image_url: ImageUrl, type: Literal['image_url'] = 'image_url')¶

Bases: ABCBaseModel

Represents an image segment within a multimodal content block.

Parameters:

image_url – An ImageUrl object containing the image’s location and detail level.
type – The type identifier, defaulting to “image_url”.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

image_url: ImageUrl¶

type_: Literal['image_url']¶

class gen_ai_hub.orchestration_v2.ImageItem(*, url: str | None = None, detail: ImageDetailLevel | None = None)¶

Bases: ABCBaseModel

Represents an image for use in multimodal messages.

Parameters:

url –
The image location, specified as either a standard URL or a data URL. - Standard URL example: ‘https://example.com/image.png’
- Data URL example: ‘data:image/png;base64,…’
detail – The image detail level for model processing.

Example

# Using a standard URL img1 = ImageItem(url=”https://example.com/image.png”, detail=ImageDetailLevel.HIGH)

# Using a data URL img2 = ImageItem(url=”data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA…”)

static from_file(file_path: str, mime_type: str | None = None, detail: ImageDetailLevel | None = None) → ImageItem¶

Create an ImageItem from a local image file.

Parameters:

file_path (str) – Path to the image file.
mime_type (Optional[str], optional) – Explicit MIME type (e.g., ‘image/png’). If not provided, the MIME type will be guessed from the file extension.
detail (Optional[ImageDetailLevel], optional) – The image detail level for model processing.

Raises:

ValueError – If the MIME type cannot be determined and is not provided.

Returns:

An ImageItem instance with the image data as a data URL.

Return type:

ImageItem

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_dump(*, mode: Literal['json', 'python'] | str = 'python', include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool = True, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False) → dict[str, Any]¶: Dumps the model to a dictionary with default settings.

url: str | None¶

detail: ImageDetailLevel | None¶

class gen_ai_hub.orchestration_v2.PromptTokensDetails(*, audio_tokens: int | None = None, cached_tokens: int | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents the details of prompt tokens used in a specific operation.

audio_tokens¶

Audio input tokens present in the prompt.

Type:: Optional[int]

cached_tokens¶

Cached tokens present in the prompt.

Type:: Optional[int]

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

audio_tokens: int | None¶

cached_tokens: int | None¶

class gen_ai_hub.orchestration_v2.CompletionTokensDetails(*, accepted_prediction_tokens: int | None = None, audio_tokens: int | None = None, reasoning_tokens: int | None = None, rejected_prediction_tokens: int | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Breakdown of tokens used in a completion.

accepted_prediction_tokens¶

When using Predicted Outputs, the number of tokens in the prediction that appeared in the completion.

Type:: Optional[int]

audio_tokens¶

Audio input tokens generated by the model.

Type:: Optional[int]

reasoning_tokens¶

Tokens generated by the model for reasoning.

Type:: Optional[int]

rejected_prediction_tokens¶

When using Predicted Outputs, the number of tokens in the prediction that did not appear in the completion. However, like reasoning tokens, these tokens are still counted in the total completion tokens for purposes of billing, output, and context window limits.

Type:: Optional[int]

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

accepted_prediction_tokens: int | None¶

audio_tokens: int | None¶

reasoning_tokens: int | None¶

rejected_prediction_tokens: int | None¶

class gen_ai_hub.orchestration_v2.TokenUsage(*, completion_tokens: int, prompt_tokens: int, total_tokens: int, prompt_tokens_details: PromptTokensDetails | None = None, completion_tokens_details: CompletionTokensDetails | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Usage of tokens in the response

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

completion_tokens: int¶

prompt_tokens: int¶

total_tokens: int¶

prompt_tokens_details: PromptTokensDetails | None¶

completion_tokens_details: CompletionTokensDetails | None¶

class gen_ai_hub.orchestration_v2.GenericModuleResult(*, message: str, data: Any | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Generic module result :param message: Some message created from the module. Example: Input to LLM is masked successfully. :param data: Additional data object from the module

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

message: str¶

data: Any | None¶

class gen_ai_hub.orchestration_v2.TopLogprob(*, token: str, logprob: float, bytes: List[int] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents one of the most likely tokens and its log probability at a given token position.

token¶

The token.

Type:: str

logprob¶

The log probability of this token.

Type:: float

bytes¶

UTF-8 bytes of the token, if applicable.

Type:: List[int] | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

token: str¶

logprob: float¶

bytes: List[int] | None¶

class gen_ai_hub.orchestration_v2.ChatCompletionTokenLogprob(*, token: str, logprob: float, bytes: List[int] | None = None, top_logprobs: List[TopLogprob] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents a token in the message content along with its log probability and alternative top log probabilities.

token¶

The token.

Type:: str

logprob¶

The log probability of this token.

Type:: float

bytes¶

UTF-8 bytes of the token, if applicable.

Type:: List[int] | None

top_logprobs¶

List of most likely tokens and their log probabilities at this token position.

Type:: List[gen_ai_hub.orchestration_v2.models.response.TopLogprob] | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

token: str¶

logprob: float¶

bytes: List[int] | None¶

top_logprobs: List[TopLogprob] | None¶

class gen_ai_hub.orchestration_v2.ChoiceLogprobs(*, content: List[ChatCompletionTokenLogprob] | None = None, refusal: List[ChatCompletionTokenLogprob] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Log probabilities for the choice.

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

content: List[ChatCompletionTokenLogprob] | None¶

refusal: List[ChatCompletionTokenLogprob] | None¶

class gen_ai_hub.orchestration_v2.LLMChoice(*, index: int, message: ResponseChatMessage, logprobs: ChoiceLogprobs | None = None, finish_reason: str, **extra_data: Any)¶

Bases: ResponseBaseModel

Parameters:

index – Index of the choice
message – Message from the LLM
logprobs – Log probabilities for the choice
finish_reason –
Reason the model stopped generating tokens. - ‘stop’ if the model hit a natural stop point or a provided stop sequence,
- ’length’ if the maximum token number was reached,
- ’content_filter’ if content was omitted due to a filter enforced by the LLM model provider
  or the content filtering module

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

index: int¶

message: ResponseChatMessage¶

logprobs: ChoiceLogprobs | None¶

finish_reason: str¶

class gen_ai_hub.orchestration_v2.StreamFunctionObject(*, name: str | None = None, arguments: str | None = None, **extra_data: Any)¶

Bases: FunctionCall

Represents a function call with its name and arguments.

name¶

str The name of the function to call.

Type:: str | None

arguments¶

Type:: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str | None¶

arguments: str | None¶

class gen_ai_hub.orchestration_v2.StreamToolCall(*, type: Literal['function'] = 'function', index: int, id: str | None = None, function: StreamFunctionObject | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['function']¶

index: int¶

id: str | None¶

function: StreamFunctionObject | None¶

class gen_ai_hub.orchestration_v2.StreamDelta(*, role: str | None = None, content: str, tool_calls: List[StreamToolCall] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: str | None¶

content: str¶

tool_calls: List[StreamToolCall] | None¶

class gen_ai_hub.orchestration_v2.StreamLLMChoice(*, index: int, delta: StreamDelta, logprobs: ChoiceLogprobs | None = None, finish_reason: str | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

index: int¶

delta: StreamDelta¶

logprobs: ChoiceLogprobs | None¶

finish_reason: str | None¶

class gen_ai_hub.orchestration_v2.Citation(*, ref_id: int | None = None, title: str, url: str, start_index: int | None = None, end_index: int | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents a citation with related metadata.

ref_id¶

Unique identifier for inline citation

Type:: Optional[int]

title¶

The title of the citation.

Type:: str

url¶

The URL of the citation.

Type:: str

start_index¶

The starting index position of the citation in a referenced text.

Type:: Optional[int]

end_index¶

The ending index position of the citation in a referenced text.

Type:: Optional[int]

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

ref_id: int | None¶

title: str¶

url: str¶

start_index: int | None¶

end_index: int | None¶

class gen_ai_hub.orchestration_v2.LLMModuleResult(*, id: str, object: str, created: int, model: str, system_fingerprint: str | None = None, choices: List[LLMChoice], usage: TokenUsage, citations: list[Citation] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Output from LLM. Follows the OpenAI spec.

id¶

Unique identifier for the response.

Type:: str

object¶

Type of object returned (e.g., “chat.completion”).

Type:: str

created¶

Unix timestamp of when the result was created.

Type:: int

model¶

The model name (e.g., “gpt-4o-mini”).

Type:: str

system_fingerprint¶

Optional system fingerprint associated with the result.

Type:: str | None

choices¶

List of LLMChoice objects representing the output choices.

Type:: List[gen_ai_hub.orchestration_v2.models.response.LLMChoice]

usage¶

TokenUsage object representing the token usage statistics.

Type:: gen_ai_hub.orchestration_v2.models.response.TokenUsage

citations¶

Optional list of citations associated with the response.

Type:: list[gen_ai_hub.orchestration_v2.models.response.Citation] | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

id: str¶

object: str¶

created: int¶

model: str¶

system_fingerprint: str | None¶

choices: List[LLMChoice]¶

usage: TokenUsage¶

citations: list[Citation] | None¶

class gen_ai_hub.orchestration_v2.StreamLLMModuleResult(*, id: str, object: str, created: int, model: str, system_fingerprint: str | None = None, choices: List[StreamLLMChoice], usage: TokenUsage | None = None, citations: list[Citation] | None = None, **extra_data: Any)¶

Bases: LLMModuleResult

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

choices: List[StreamLLMChoice]¶

usage: TokenUsage | None¶

Bases: ResponseBaseModel

Represents the results of each module used in a processing pipeline.

grounding¶

Optional result from the grounding module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

templating¶

Optional list of chat messages resulting from the templating module.

Type:: List[gen_ai_hub.orchestration_v2.models.message.SystemMessage | gen_ai_hub.orchestration_v2.models.message.UserMessage | gen_ai_hub.orchestration_v2.models.message.AssistantMessage | gen_ai_hub.orchestration_v2.models.message.ToolChatMessage | gen_ai_hub.orchestration_v2.models.message.DeveloperChatMessage | gen_ai_hub.orchestration_v2.models.message.ResponseChatMessage] | None

input_translation¶

Optional result from the input translation module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

input_masking¶

Optional result from the input masking module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

input_filtering¶

Optional result from the input filtering module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

output_filtering¶

Optional result from the output filtering module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

output_translation¶

Optional result from the output translation module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

llm¶

Optional result from an LLM-specific module.

Type:: gen_ai_hub.orchestration_v2.models.response.LLMModuleResult | None

output_unmasking¶

Optional list of choices from the output unmasking module.

Type:: List[gen_ai_hub.orchestration_v2.models.response.LLMChoice] | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

grounding: GenericModuleResult | None¶

templating: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage] | None¶

input_translation: GenericModuleResult | None¶

input_masking: GenericModuleResult | None¶

input_filtering: GenericModuleResult | None¶

output_filtering: GenericModuleResult | None¶

output_translation: GenericModuleResult | None¶

llm: LLMModuleResult | None¶

output_unmasking: List[LLMChoice] | None¶

Bases: ModuleResults

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

llm: StreamLLMModuleResult | None¶

output_unmasking: List[StreamLLMChoice] | None¶

class gen_ai_hub.orchestration_v2.SAPAPIError(*, request_id: str, code: int, message: str, location: str, intermediate_results: ModuleResults | None = None, headers: dict[str, str] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents an error returned from an SAP API.

request_id¶

The unique identifier of the request associated with the error.

Type:: str

code¶

The http error code.

Type:: int

message¶

A detailed message describing the error.

Type:: str

location¶

The location where the error occurred

Type:: str

intermediate_results¶

Optional attribute to store any processing results if available or applicable.

Type:: Optional[ModuleResults]

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str¶

code: int¶

message: str¶

location: str¶

intermediate_results: ModuleResults | None¶

headers: dict[str, str] | None¶

class gen_ai_hub.orchestration_v2.SAPAPIErrorStreaming(*, request_id: str, code: int, message: str, location: str, intermediate_results: ModuleResultsStreaming | None = None, headers: dict[str, str] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents an error returned from an SAP API.

request_id¶

The unique identifier of the request associated with the error.

Type:: str

code¶

The http error code.

Type:: int

message¶

A detailed message describing the error.

Type:: str

location¶

The location where the error occurred

Type:: str

intermediate_results¶

Optional attribute to store any processing results if available or applicable.

Type:: Optional[ModuleResults]

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str¶

code: int¶

message: str¶

location: str¶

intermediate_results: ModuleResultsStreaming | None¶

headers: dict[str, str] | None¶

class gen_ai_hub.orchestration_v2.CompletionPostResponse(*, request_id: str, intermediate_results: ModuleResults, final_result: LLMModuleResult, intermediate_failures: List[SAPAPIError] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents the response for a completion post request.

request_id¶

Unique identifier for the completion request.

Type:: str

intermediate_results¶

Results from various modules executed during the processing.

Type:: ModuleResults

final_result¶

Output from LLM. Follows the OpenAI spec.

Type:: LLMModuleResult

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str¶

intermediate_results: ModuleResults¶

final_result: LLMModuleResult¶

intermediate_failures: List[SAPAPIError] | None¶

class gen_ai_hub.orchestration_v2.StreamCompletionPostResponse(*, request_id: str, intermediate_results: StreamModuleResults | None, final_result: StreamLLMModuleResult | None, intermediate_failures: List[SAPAPIError] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str¶

intermediate_results: StreamModuleResults | None¶

final_result: StreamLLMModuleResult | None¶

intermediate_failures: List[SAPAPIError] | None¶

class gen_ai_hub.orchestration_v2.ErrorResponse(*, error: SAPAPIError | list[SAPAPIError], **extra_data: Any)¶

Bases: ResponseBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

error: SAPAPIError | list[SAPAPIError]¶

class gen_ai_hub.orchestration_v2.ErrorResponseStreaming(*, error: SAPAPIErrorStreaming | list[SAPAPIErrorStreaming], **extra_data: Any)¶

Bases: ResponseBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

error: SAPAPIErrorStreaming | list[SAPAPIErrorStreaming]¶

class gen_ai_hub.orchestration_v2.OrchestrationResponseWithRetries(*, request_id: str, intermediate_results: ModuleResults, final_result: LLMModuleResult, intermediate_failures: List[SAPAPIError] | None = None, retries: int = 0, **extra_data: Any)¶

Bases: CompletionPostResponse

Extended CompletionPostResponse that includes retry count information.

This is returned when using retry-enabled methods like run_with_retries().

retries¶

Number of retry attempts that were made to successfully complete this request.

Type:: int

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

retries: int¶

class gen_ai_hub.orchestration_v2.ResponseFormatText(*, type: ResponseFormatType = ResponseFormatType.TEXT)¶

Bases: ABCBaseModel

Response format that the model output should adhere to.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ResponseFormatType¶

class gen_ai_hub.orchestration_v2.ResponseFormatJsonObject(*, type: ResponseFormatType = ResponseFormatType.JSON_OBJECT)¶

Bases: ABCBaseModel

Response format JSON Object that the model output should adhere to.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ResponseFormatType¶

class gen_ai_hub.orchestration_v2.ResponseFormatJsonSchema(*, type: ResponseFormatType = ResponseFormatType.JSON_SCHEMA, json_schema: JSONResponseSchema)¶

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ResponseFormatType¶

json_schema: JSONResponseSchema¶

class gen_ai_hub.orchestration_v2.JSONResponseSchema(*, name: str, description: str | None = None, schema: dict = <factory>, strict: bool = False)¶

Bases: ABCBaseModel

Response format JSON Schema that the model output should adhere to.

Parameters:

name – The name of the response format.
description – A description of what the response format is for.
schema – A schema for the response format described as a JSON Schema object.
strict – Whether to enable strict schema adherence when generating the output.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_name(name)¶

validates the name of the response format.

Parameters:: name (str) – the name to validate
Raises:: ValueError – if the name does not match the required pattern or exceeds the maximum length
Returns:: the validated name
Return type:: str

name: str¶

description: str | None¶

schema_: dict¶

strict: bool¶

class gen_ai_hub.orchestration_v2.GlobalStreamOptions(*, enabled: bool | None = False, chunk_size: int | None = 100, delimiters: List[str] | None = None)¶

Bases: ABCBaseModel

Represents options for streaming content generation. :param enabled: If true, the response will be streamed back to the client. :type enabled: bool, optional :param chunk_size: Minimum number of characters per chunk that post-LLM modules operate on. :type chunk_size: int, optional :param delimiters: List of delimiters to split the input text into chunks.Please note, :type delimiters: list(str), optional :param this is a required parameter when input_translation_module_config or: :param output_translation_module_config are configured.:

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_dump(**kwargs)¶: Override model_dump to exclude chunk_size and delimiters when enabled is False.

enabled: bool | None¶

chunk_size: int | None¶

delimiters: List[str] | None¶

Bases: ABCBaseModel

Represents a configurable template for generating prompts or conversations.

Parameters:

template – A list of prompt messages that form the template.
defaults – A dict of default values for template variables.
tools – A list of tool definitions.
response_format – A response format that the model output should adhere to.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

template: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage]¶

defaults: dict | None¶

response_format: ResponseFormatText | ResponseFormatJsonObject | ResponseFormatJsonSchema | None¶

tools: List[dict | FunctionTool] | None¶

class gen_ai_hub.orchestration_v2.PromptTemplatingModuleConfig(*, prompt: Template | TemplateRef, model: LLMModelDetails)¶

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt: Template | TemplateRef¶

model: LLMModelDetails¶

class gen_ai_hub.orchestration_v2.TemplateRef(*, template_ref: TemplateRefByID | TemplateRefByScenarioNameVersion)¶

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

template_ref: TemplateRefByID | TemplateRefByScenarioNameVersion¶

class gen_ai_hub.orchestration_v2.TemplateRefByID(*, id: str, scope: Literal['resource_group', 'tenant'] | None = 'tenant')¶

Bases: ABCBaseModel

Represents a prompt template reference for generating prompts or conversations. :param id: ID of the template in prompt registry :type id: str :param scope: Defines the scope that is searched

for the referenced template. ‘tenant’ indicates the template is shared across all resource groups within the tenant, while ‘resource_group’ indicates the template is only accessible within the specific resource group. Defaults to ‘tenant’.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

id: str¶

scope: Literal['resource_group', 'tenant'] | None¶

class gen_ai_hub.orchestration_v2.TemplateRefByScenarioNameVersion(*, scenario: str, name: str, version: str, scope: Literal['resource_group', 'tenant'] | None = 'tenant')¶

Bases: ABCBaseModel

Represents a prompt template reference for generating prompts or conversations. :param scenario: Scenario name :type scenario: str :param name: Name of template :type name: str :param version: Version of template :type version: str :param scope: Defines the scope that is searched

for the referenced template. ‘tenant’ indicates the template is shared across all resource groups within the tenant, while ‘resource_group’ indicates the template is only accessible within the specific resource group. Defaults to ‘tenant’.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

scenario: str¶

name: str¶

version: str¶

scope: Literal['resource_group', 'tenant'] | None¶

gen_ai_hub.orchestration_v2.python_type_to_json_type(py_type)¶

Convert a Python type to a JSON Schema type.

Parameters:: py_type (any) – the Python type to convert
Returns:: A dictionary representing the JSON Schema type.
Return type:: dict

class gen_ai_hub.orchestration_v2.ChatCompletionTool(*, type: Literal['function'] = 'function')¶

Bases: ABCBaseModel

Base class for all chat completion tools.

Parameters:: type (Literal["function"]) – The type of the tool. Currently, only function is supported.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['function']¶

class gen_ai_hub.orchestration_v2.FunctionObject(*, description: str | None = None, name: str, parameters: dict | None, strict: bool = False, function: Callable | None = None)¶

Bases: ABCBaseModel

Represents a function. :param name: The name of the function to be called. Must be a-z, A-Z, 0-9,

or contain underscores and dashes, with a maximum length of 64.

Parameters:

description (str) – A description of what the function does, used by the model to choose when and how to call the function.
parameters (dict) – The parameters the functions accepts, described as a JSON Schema object. Omitting parameters defines a function with an empty parameter list.
strict (bool, optional) – Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the parameters field. Only a subset of JSON Schema is supported when strict is true. Defaults to False.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

description: str | None¶

name: str¶

parameters: dict | None¶

strict: bool¶

function: Callable | None¶

class gen_ai_hub.orchestration_v2.FunctionTool(*, type: Literal['function'] = 'function', function: FunctionObject)¶

Bases: ChatCompletionTool

Represents a function tool for OpenAI-like function calling.

Parameters:

type (Literal["function"]) – The type of the tool. Currently, only function is supported.
function (FunctionObject) – The function to be called.

async aexecute(**kwargs: Any) → Any¶: Asynchronously execute the function with the provided arguments.

execute(**kwargs: Any) → Any¶: Execute the function with the provided arguments.

static from_function(func: Callable, *, description: str | None = None, strict: bool = False) → FunctionTool¶

Create a FunctionTool from a Python function.

Parameters:

func (Callable) – The function to be converted to a FunctionTool.
description (Optional[str]) – A description of the function. Defaults to the docstring of the function.
strict (bool) – Whether to enable strict schema adherence when generating the function call.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['function']¶

function: FunctionObject¶

gen_ai_hub.orchestration_v2.function_tool(func: Callable | None = None, *, description: str | None = None, strict: bool = False) → Callable[[Callable], FunctionTool] | FunctionTool¶

Decorator that converts a function into a FunctionTool.

Usage:

@function_tool def my_func(…): …

@function_tool() def my_func(…): …

class gen_ai_hub.orchestration_v2.TranslationConfig(*, source_language: str | None = None, target_language: str)¶

Bases: ABCBaseModel

Configuration for sap_document_translation translation provider.

Parameters:

source_language – Language of the text to be translated. Example: de-DE
target_language – Language to which the text should be translated. Example: en-US

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

source_language: str | None¶

target_language: str¶

class gen_ai_hub.orchestration_v2.SAPDocumentTranslation(*, type: TranslationType = TranslationType.SAP_DOCUMENT_TRANSLATION, config: TranslationConfig)¶

Bases: ABCBaseModel

Configuration for translation module.

Parameters:

type – The type of translation module (e.g., ‘sap_document_translation’).
config – Configuration object for the translation module.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: TranslationType¶

config: TranslationConfig¶

class gen_ai_hub.orchestration_v2.SAPDocumentTranslationApplyToSelector(*, category: Literal['placeholders', 'template_roles'], items: list[str], source_language: str)¶

Bases: ABCBaseModel

This selector allows you to define the scope of translation, such as specific placeholders or messages with specific roles. For example, {“category”: “placeholders”,

“items”: [“user_input”], “source_language”: “de-DE”} targets the value of “user_input” in placeholder_values specified in the request payload; and considers the value to be in German.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

category: Literal['placeholders', 'template_roles']¶

items: list[str]¶

source_language: str¶

class gen_ai_hub.orchestration_v2.InputTranslationConfig(*, source_language: str | None = None, target_language: str, apply_to: list[SAPDocumentTranslationApplyToSelector] | None = None)¶

Bases: TranslationConfig

Configuration for input translation.

Parameters:

source_language – Language of the text to be translated. Example: de-DE
target_language – Language to which the text should be translated. Example: en-US
apply_to – List of selectors that define the scope of translation.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

apply_to: list[SAPDocumentTranslationApplyToSelector] | None¶

class gen_ai_hub.orchestration_v2.OutputTranslationConfig(*, source_language: str | None = None, target_language: str | SAPDocumentTranslationApplyToSelector)¶

Bases: TranslationConfig

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

target_language: str | SAPDocumentTranslationApplyToSelector¶

class gen_ai_hub.orchestration_v2.SAPDocumentTranslationInput(*, type: TranslationType = TranslationType.SAP_DOCUMENT_TRANSLATION, config: InputTranslationConfig | TranslationConfig, translate_messages_history: bool | None = None)¶

Bases: SAPDocumentTranslation

Configuration for input translation

Parameters:

type – The type of translation module (e.g., ‘sap_document_translation’).
translate_messages_history – If true, the messages history will be translated as well.
config – Configuration object for the translation module.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

translate_messages_history: bool | None¶

config: InputTranslationConfig | TranslationConfig¶

class gen_ai_hub.orchestration_v2.SAPDocumentTranslationOutput(*, type: TranslationType = TranslationType.SAP_DOCUMENT_TRANSLATION, config: OutputTranslationConfig | TranslationConfig)¶

Bases: SAPDocumentTranslation

Configuration for output translation

Parameters:

type – The type of translation module (e.g., ‘sap_document_translation’).
config – Configuration object for the translation module.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

config: OutputTranslationConfig | TranslationConfig¶

class gen_ai_hub.orchestration_v2.TranslationModuleConfig(*, input: SAPDocumentTranslationInput | SAPDocumentTranslation | None = None, output: SAPDocumentTranslationOutput | SAPDocumentTranslation | None = None)¶

Bases: ABCBaseModel

Configuration for translation module

Parameters:

input – Configuration for input translation
output – Configuration for output translation

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

input: SAPDocumentTranslationInput | SAPDocumentTranslation | None¶

output: SAPDocumentTranslationOutput | SAPDocumentTranslation | None¶

Bases: object

A service for executing orchestration requests, allowing for the generation of LLM-generated content through a pipeline of configured modules.

This service supports both synchronous and asynchronous request execution. For streaming responses, special care is taken to not close the underlying HTTP stream prematurely.

See https://api.sap.com/api/ORCHESTRATION_API_v2/overview

Args:

api_url: The base URL for the orchestration API.

config: The default orchestration configuration.

config_ref: The reference to default orchestration configuration.

proxy_client: A GenAIHubProxyClient instance.

deployment_id: Optional deployment ID.

config_name: Optional configuration name.

config_id: Optional configuration ID.

timeout: Optional timeout for HTTP requests.

Initializes the OrchestrationService.

Parameters:

api_url (Optional[str], optional) – the base URL for the orchestration API, defaults to None
config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional) – the orchestration configuration reference, defaults to None
proxy_client (Optional[GenAIHubProxyClient], optional) – the GenAIHubProxyClient instance, defaults to None
deployment_id (Optional[str], optional) – the deployment ID, defaults to None
config_name (Optional[str], optional) – the configuration name, defaults to None
config_id (Optional[str], optional) – the configuration ID, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout for HTTP requests, defaults to None

Raises:

ValueError – if both config and config_ref are provided.

async aclose_http_connection()¶: Closes the httpx asynchronous client.

async aembed(config: EmbeddingsOrchestrationConfig, input: EmbeddingsInput, timeout: int | float | Timeout | None = None) → EmbeddingsPostResponse¶

Executes an embeddings request asynchronously.

Parameters:

config (EmbeddingsOrchestrationConfig) – the embeddings orchestration configuration
input (EmbeddingsInput) – the input text to embed
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None

Returns:

the EmbeddingsPostResponse object

Return type:

EmbeddingsPostResponse

Executes an orchestration request asynchronously (non-streaming).

Parameters:

config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional) – the orchestration configuration reference, defaults to None
placeholder_values (Optional[dict], optional) – the template values, defaults to None
history (Optional[List[ChatMessage]], optional) – the message history, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None

Returns:

the CompletionPostResponse object

Return type:

CompletionPostResponse

Executes an orchestration request asynchronously with automatic retry on rate limits (429) and server errors. Uses exponential backoff with jitter to handle rate limiting gracefully.

Parameters:

config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional) – the orchestration configuration reference, defaults to None
placeholder_values (Optional[dict], optional) – the template values, defaults to None
history (Optional[List[ChatMessage]], optional) – the message history, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None
max_retries (int, optional) – the maximum number of retry attempts, defaults to 10
base_delay (float, optional) – the initial delay between retries in seconds, defaults to 1.0

Returns:

the OrchestrationResponseWithRetries with retry count information

Return type:

OrchestrationResponseWithRetries | None

Raises:

ValueError – if no configuration is provided.
OrchestrationError – if request fails after all retries (includes retry count).

Executes an orchestration streaming request asynchronously.

Parameters:

config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional) – the orchestration configuration reference, defaults to None
placeholder_values (Optional[dict], optional) – the template values, defaults to None
history (Optional[List[ChatMessage]], optional) – the message history, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None

Returns:

the AsyncSSEClient object

Return type:

AsyncSSEClient

close_http_connection()¶: Closes the httpx synchronous client.

embed(config: EmbeddingsOrchestrationConfig, input: EmbeddingsInput, timeout: int | float | Timeout | None = None) → EmbeddingsPostResponse¶

Executes an embeddings request synchronously.

Parameters:

config (EmbeddingsOrchestrationConfig) – the embeddings orchestration configuration
input (EmbeddingsInput) – the input text to embed
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None

Returns:

the EmbeddingsPostResponse object

Return type:

EmbeddingsPostResponse

handle_retry(retry_count: int, base_delay: float, error: OrchestrationError, max_retries: int) → float¶

Handles retry logic with exponential backoff and jitter. If Retry-After header exists, use it as min_delay to add jitter on top

Parameters:

retry_count (int) – the incremented retry attempt number
base_delay (float) – the initial delay between retries in seconds
error (OrchestrationError) – the exception that occurred
max_retries (int) – the maximum number of retry attempts

Raises:

error – throws the original error if no retry should be attempted

Returns:

the number of seconds to wait before next retry

Return type:

float

Executes an orchestration request synchronously (non-streaming).

Parameters:

config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional) – the orchestration configuration reference, defaults to None if not provided, the default configuration is used.
placeholder_values (Optional[dict], optional) – the template values, defaults to None
history (Optional[List[ChatMessage]], optional) – the message history, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None

Returns:

the CompletionPostResponse object

Return type:

CompletionPostResponse

Executes an orchestration request with automatic retry on rate limits (429) and server errors.

Parameters:

config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional) – the orchestration configuration reference, defaults to None
placeholder_values (Optional[dict], optional) – the template values, defaults to None
history (Optional[List[ChatMessage]], optional) – the message history, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None
max_retries (int, optional) – the maximum number of retry attempts, defaults to 10
base_delay (float, optional) – the initial delay between retries in seconds, defaults to 1.0

Returns:

the OrchestrationResponseWithRetries with retry count information

Return type:

OrchestrationResponseWithRetries | None

Raises:

ValueError – if no configuration is provided.
OrchestrationError – if request fails after all retries (includes retry count).

Executes an orchestration streaming request synchronously.

Parameters:

config (Optional[OrchestrationConfig], optional) – the orchestration configuration, defaults to None
config_ref (Optional[OrchestrationConfigReference], optional if not provided, the default configuration is used.) – the orchestration configuration reference, defaults to None
placeholder_values (Optional[dict], optional) – the template values, defaults to None
history (Optional[List[ChatMessage]], optional) – the message history, defaults to None
timeout (Union[int, float, httpx.Timeout, None], optional) – the timeout overwrite per request, defaults to None

Returns:

An Iterable[StreamCompletionPostResponse] object

Return type:

Iterable[StreamCompletionPostResponse]

exception gen_ai_hub.orchestration_v2.OrchestrationError(request_id: str, headers: Headers, message: str, code: int, location: str, intermediate_results: ModuleResults | dict, retries: int = 0)¶

Bases: Exception

This exception is raised when an error occurs during the execution of the orchestration service, typically due to incorrect usage, invalid configurations, or issues with run parameters defined by the user.

__init__(request_id: str, headers: Headers, message: str, code: int, location: str, intermediate_results: ModuleResults | dict, retries: int = 0)¶

Initializes the OrchestrationError with detailed context.

Parameters:

request_id (str) – unique identifier for the request that encountered the error.
headers (httpx.Headers) – HTTP headers associated with the request, useful in case of e.g. rate limiting..
message (str) – Detailed error message describing the issue.
code (int) – Error code associated with the specific type of failure.
location (str) – Specific component or step in the orchestration process where the error occurred.
intermediate_results (ModuleResults) – State information and partial results from various modules at the time of the error, useful for debugging.
retries (int, optional) – Number of retries attempted before the error was raised.
errors (Optional[list[dict[str, Any]]]) – Raw error payload(s) from the API. Can contain multiple errors.

exception gen_ai_hub.orchestration_v2.OrchestrationErrorList(errors: list[OrchestrationError])¶

Bases: Exception

__init__(errors: list[OrchestrationError])¶

Subpackages¶

gen_ai_hub.orchestration_v2.models package

gen_ai_hub.orchestration_v2 package¶

Subpackages¶

Submodules¶