gen_ai_hub.orchestration_v2.models.response module¶

Response models for orchestration v2

class gen_ai_hub.orchestration_v2.models.response.PromptTokensDetails(*, audio_tokens: int | None = None, cached_tokens: int | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents the details of prompt tokens used in a specific operation.

audio_tokens¶

Audio input tokens present in the prompt.

Type:: Optional[int]

cached_tokens¶

Cached tokens present in the prompt.

Type:: Optional[int]

audio_tokens: int | None¶

cached_tokens: int | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.CompletionTokensDetails(*, accepted_prediction_tokens: int | None = None, audio_tokens: int | None = None, reasoning_tokens: int | None = None, rejected_prediction_tokens: int | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Breakdown of tokens used in a completion.

accepted_prediction_tokens¶

When using Predicted Outputs, the number of tokens in the prediction that appeared in the completion.

Type:: Optional[int]

audio_tokens¶

Audio input tokens generated by the model.

Type:: Optional[int]

reasoning_tokens¶

Tokens generated by the model for reasoning.

Type:: Optional[int]

rejected_prediction_tokens¶

When using Predicted Outputs, the number of tokens in the prediction that did not appear in the completion. However, like reasoning tokens, these tokens are still counted in the total completion tokens for purposes of billing, output, and context window limits.

Type:: Optional[int]

accepted_prediction_tokens: int | None¶

audio_tokens: int | None¶

reasoning_tokens: int | None¶

rejected_prediction_tokens: int | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.TokenUsage(*, completion_tokens: int, prompt_tokens: int, total_tokens: int, prompt_tokens_details: PromptTokensDetails | None = None, completion_tokens_details: CompletionTokensDetails | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Usage of tokens in the response

completion_tokens: int¶

prompt_tokens: int¶

total_tokens: int¶

prompt_tokens_details: PromptTokensDetails | None¶

completion_tokens_details: CompletionTokensDetails | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.GenericModuleResult(*, message: str, data: Any | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Generic module result :param message: Some message created from the module. Example: Input to LLM is masked successfully. :param data: Additional data object from the module

message: str¶

data: Any | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.TopLogprob(*, token: str, logprob: float, bytes: List[int] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents one of the most likely tokens and its log probability at a given token position.

token¶

The token.

Type:: str

logprob¶

The log probability of this token.

Type:: float

bytes¶

UTF-8 bytes of the token, if applicable.

Type:: List[int] | None

token: str¶

logprob: float¶

bytes: List[int] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.ChatCompletionTokenLogprob(*, token: str, logprob: float, bytes: List[int] | None = None, top_logprobs: List[TopLogprob] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents a token in the message content along with its log probability and alternative top log probabilities.

token¶

The token.

Type:: str

logprob¶

The log probability of this token.

Type:: float

bytes¶

UTF-8 bytes of the token, if applicable.

Type:: List[int] | None

top_logprobs¶

List of most likely tokens and their log probabilities at this token position.

Type:: List[gen_ai_hub.orchestration_v2.models.response.TopLogprob] | None

token: str¶

logprob: float¶

bytes: List[int] | None¶

top_logprobs: List[TopLogprob] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.ChoiceLogprobs(*, content: List[ChatCompletionTokenLogprob] | None = None, refusal: List[ChatCompletionTokenLogprob] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Log probabilities for the choice.

content: List[ChatCompletionTokenLogprob] | None¶

refusal: List[ChatCompletionTokenLogprob] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.LLMChoice(*, index: int, message: ResponseChatMessage, logprobs: ChoiceLogprobs | None = None, finish_reason: str, **extra_data: Any)¶

Bases: ResponseBaseModel

Parameters:

index – Index of the choice
message – Message from the LLM
logprobs – Log probabilities for the choice
finish_reason –
Reason the model stopped generating tokens. - ‘stop’ if the model hit a natural stop point or a provided stop sequence,
- ’length’ if the maximum token number was reached,
- ’content_filter’ if content was omitted due to a filter enforced by the LLM model provider
  or the content filtering module

index: int¶

message: ResponseChatMessage¶

logprobs: ChoiceLogprobs | None¶

finish_reason: str¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.StreamFunctionObject(*, name: str | None = None, arguments: str | None = None, **extra_data: Any)¶

Bases: FunctionCall

Represents a function call with its name and arguments.

name¶

str The name of the function to call.

Type:: str | None

arguments¶

str The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.

Type:: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str | None¶

arguments: str | None¶

class gen_ai_hub.orchestration_v2.models.response.StreamToolCall(*, type: Literal['function'] = 'function', index: int, id: str | None = None, function: StreamFunctionObject | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

type_: Literal['function']¶

index: int¶

id: str | None¶

function: StreamFunctionObject | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.StreamDelta(*, role: str | None = None, content: str, tool_calls: List[StreamToolCall] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

role: str | None¶

content: str¶

tool_calls: List[StreamToolCall] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.StreamLLMChoice(*, index: int, delta: StreamDelta, logprobs: ChoiceLogprobs | None = None, finish_reason: str | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

index: int¶

delta: StreamDelta¶

logprobs: ChoiceLogprobs | None¶

finish_reason: str | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.Citation(*, ref_id: int | None = None, title: str, url: str, start_index: int | None = None, end_index: int | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents a citation with related metadata.

ref_id¶

Unique identifier for inline citation

Type:: Optional[int]

title¶

The title of the citation.

Type:: str

url¶

The URL of the citation.

Type:: str

start_index¶

The starting index position of the citation in a referenced text.

Type:: Optional[int]

end_index¶

The ending index position of the citation in a referenced text.

Type:: Optional[int]

ref_id: int | None¶

title: str¶

url: str¶

start_index: int | None¶

end_index: int | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.LLMModuleResult(*, id: str, object: str, created: int, model: str, system_fingerprint: str | None = None, choices: List[LLMChoice], usage: TokenUsage, citations: list[Citation] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Output from LLM. Follows the OpenAI spec.

id¶

Unique identifier for the response.

Type:: str

object¶

Type of object returned (e.g., “chat.completion”).

Type:: str

created¶

Unix timestamp of when the result was created.

Type:: int

model¶

The model name (e.g., “gpt-4o-mini”).

Type:: str

system_fingerprint¶

Optional system fingerprint associated with the result.

Type:: str | None

choices¶

List of LLMChoice objects representing the output choices.

Type:: List[gen_ai_hub.orchestration_v2.models.response.LLMChoice]

usage¶

TokenUsage object representing the token usage statistics.

Type:: gen_ai_hub.orchestration_v2.models.response.TokenUsage

citations¶

Optional list of citations associated with the response.

Type:: list[gen_ai_hub.orchestration_v2.models.response.Citation] | None

id: str¶

object: str¶

created: int¶

model: str¶

system_fingerprint: str | None¶

choices: List[LLMChoice]¶

usage: TokenUsage¶

citations: list[Citation] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.StreamLLMModuleResult(*, id: str, object: str, created: int, model: str, system_fingerprint: str | None = None, choices: List[StreamLLMChoice], usage: TokenUsage | None = None, citations: list[Citation] | None = None, **extra_data: Any)¶

Bases: LLMModuleResult

choices: List[StreamLLMChoice]¶

usage: TokenUsage | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

id: str¶

object: str¶

created: int¶

model: str¶

system_fingerprint: str | None¶

citations: list[Citation] | None¶

Bases: ResponseBaseModel

Represents the results of each module used in a processing pipeline.

grounding¶

Optional result from the grounding module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

templating¶

Optional list of chat messages resulting from the templating module.

Type:: List[gen_ai_hub.orchestration_v2.models.message.SystemMessage | gen_ai_hub.orchestration_v2.models.message.UserMessage | gen_ai_hub.orchestration_v2.models.message.AssistantMessage | gen_ai_hub.orchestration_v2.models.message.ToolChatMessage | gen_ai_hub.orchestration_v2.models.message.DeveloperChatMessage | gen_ai_hub.orchestration_v2.models.message.ResponseChatMessage] | None

input_translation¶

Optional result from the input translation module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

input_masking¶

Optional result from the input masking module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

input_filtering¶

Optional result from the input filtering module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

output_filtering¶

Optional result from the output filtering module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

output_translation¶

Optional result from the output translation module.

Type:: gen_ai_hub.orchestration_v2.models.response.GenericModuleResult | None

llm¶

Optional result from an LLM-specific module.

Type:: gen_ai_hub.orchestration_v2.models.response.LLMModuleResult | None

output_unmasking¶

Optional list of choices from the output unmasking module.

Type:: List[gen_ai_hub.orchestration_v2.models.response.LLMChoice] | None

grounding: GenericModuleResult | None¶

templating: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage] | None¶

input_translation: GenericModuleResult | None¶

input_masking: GenericModuleResult | None¶

input_filtering: GenericModuleResult | None¶

output_filtering: GenericModuleResult | None¶

output_translation: GenericModuleResult | None¶

llm: LLMModuleResult | None¶

output_unmasking: List[LLMChoice] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.StreamModuleResults(*, grounding: GenericModuleResult | None = None, templating: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage] | None = None, input_translation: GenericModuleResult | None = None, input_masking: GenericModuleResult | None = None, input_filtering: GenericModuleResult | None = None, output_filtering: GenericModuleResult | None = None, output_translation: GenericModuleResult | None = None, llm: StreamLLMModuleResult | None = None, output_unmasking: List[StreamLLMChoice] | None = None, **extra_data: Any)¶

Bases: ModuleResults

llm: StreamLLMModuleResult | None¶

output_unmasking: List[StreamLLMChoice] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

grounding: GenericModuleResult | None¶

templating: List[ChatMessage] | None¶

input_translation: GenericModuleResult | None¶

input_masking: GenericModuleResult | None¶

input_filtering: GenericModuleResult | None¶

output_filtering: GenericModuleResult | None¶

output_translation: GenericModuleResult | None¶

class gen_ai_hub.orchestration_v2.models.response.SAPAPIError(*, request_id: str, code: int, message: str, location: str, intermediate_results: ModuleResults | None = None, headers: dict[str, str] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents an error returned from an SAP API.

request_id¶

The unique identifier of the request associated with the error.

Type:: str

code¶

The http error code.

Type:: int

message¶

A detailed message describing the error.

Type:: str

location¶

The location where the error occurred

Type:: str

intermediate_results¶

Optional attribute to store any processing results if available or applicable.

Type:: Optional[ModuleResults]

request_id: str¶

code: int¶

message: str¶

location: str¶

intermediate_results: ModuleResults | None¶

headers: dict[str, str] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.SAPAPIErrorStreaming(*, request_id: str, code: int, message: str, location: str, intermediate_results: ModuleResultsStreaming | None = None, headers: dict[str, str] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents an error returned from an SAP API.

request_id¶

The unique identifier of the request associated with the error.

Type:: str

code¶

The http error code.

Type:: int

message¶

A detailed message describing the error.

Type:: str

location¶

The location where the error occurred

Type:: str

intermediate_results¶

Optional attribute to store any processing results if available or applicable.

Type:: Optional[ModuleResults]

request_id: str¶

code: int¶

message: str¶

location: str¶

intermediate_results: ModuleResultsStreaming | None¶

headers: dict[str, str] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.CompletionPostResponse(*, request_id: str, intermediate_results: ModuleResults, final_result: LLMModuleResult, intermediate_failures: List[SAPAPIError] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

Represents the response for a completion post request.

request_id¶

Unique identifier for the completion request.

Type:: str

intermediate_results¶

Results from various modules executed during the processing.

Type:: ModuleResults

final_result¶

Output from LLM. Follows the OpenAI spec.

Type:: LLMModuleResult

request_id: str¶

intermediate_results: ModuleResults¶

final_result: LLMModuleResult¶

intermediate_failures: List[SAPAPIError] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.StreamCompletionPostResponse(*, request_id: str, intermediate_results: StreamModuleResults | None, final_result: StreamLLMModuleResult | None, intermediate_failures: List[SAPAPIError] | None = None, **extra_data: Any)¶

Bases: ResponseBaseModel

request_id: str¶

intermediate_results: StreamModuleResults | None¶

final_result: StreamLLMModuleResult | None¶

intermediate_failures: List[SAPAPIError] | None¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.ErrorResponse(*, error: SAPAPIError | list[SAPAPIError], **extra_data: Any)¶

Bases: ResponseBaseModel

error: SAPAPIError | list[SAPAPIError]¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.ErrorResponseStreaming(*, error: SAPAPIErrorStreaming | list[SAPAPIErrorStreaming], **extra_data: Any)¶

Bases: ResponseBaseModel

error: SAPAPIErrorStreaming | list[SAPAPIErrorStreaming]¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.orchestration_v2.models.response.OrchestrationResponseWithRetries(*, request_id: str, intermediate_results: ModuleResults, final_result: LLMModuleResult, intermediate_failures: List[SAPAPIError] | None = None, retries: int = 0, **extra_data: Any)¶

Bases: CompletionPostResponse

Extended CompletionPostResponse that includes retry count information.

This is returned when using retry-enabled methods like run_with_retries().

retries¶

Number of retry attempts that were made to successfully complete this request.

Type:: int

retries: int¶

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str¶

intermediate_results: ModuleResults¶

final_result: LLMModuleResult¶

intermediate_failures: List[SAPAPIError] | None¶