gen_ai_hub.batch_service.models.request module

Request models for the LLM Batch Service API.

class gen_ai_hub.batch_service.models.request.BatchInput(*, uri: str)

Bases: ABCBaseModel

Input configuration for a batch job.

Points to the .jsonl file in an object store that contains the individual LLM requests to be processed.

Parameters:

uri (str) – Fully qualified object-store URI of the input file. Must point to a .jsonl file (e.g. ai://my-store/input/requests.jsonl).

uri: str
model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.batch_service.models.request.BatchOutput(*, uri: str)

Bases: ABCBaseModel

Output configuration for a batch job.

Points to the directory in an object store where results will be written once the job completes.

Parameters:

uri (str) – Fully qualified object-store URI of the output directory (e.g. ai://my-store/output/).

uri: str
model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.batch_service.models.request.BatchSpec(*, provider: str, model: str)

Bases: ABCBaseModel

Specification of the LLM to use for a batch job.

Parameters:

  • provider (str) – LLM provider name as registered in SAP AI Core (e.g. "azure-openai").

  • model (str) – Model name to use for inference (e.g. "gpt-4.1-mini").

provider: str
model: str
model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class gen_ai_hub.batch_service.models.request.BatchCreateRequest(*, type: Literal['llm-native'], input: BatchInput, output: BatchOutput, spec: BatchSpec)

Bases: ABCBaseModel

Request body sent to POST /llm-batch-service/v1/batches.

Describes a new batch processing job: where to read input from, where to write output, and which model to use.

Parameters:

  • type (Literal["llm-native"]) – Batch processing type. Currently only "llm-native" is supported.

  • input (BatchInput) – Input file configuration.

  • output (BatchOutput) – Output directory configuration.

  • spec (BatchSpec) – LLM provider and model specification.

type: Literal['llm-native']
input: BatchInput
output: BatchOutput
spec: BatchSpec
model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].