gen_ai_hub.orchestration_v2.models.llm_model_details module

Module defining the LLM (Large Language Model) configuration model.

class gen_ai_hub.orchestration_v2.models.llm_model_details.LLMModelDetails(*, name: str, version: str | None = 'latest', params: Dict | None = None, timeout: Annotated[int | None, Ge(ge=1), Le(le=600)] = 600, max_retries: Annotated[int | None, Ge(ge=0), Le(le=5)] = 2)

Bases: ABCBaseModel

The model and parameters to be used for the prompt templating. This is the model that will be used to generate the response.

Parameters:

  • name – Name of the model as in LLM Access configuration.

  • version – Version of the model to be used. Defaults to “latest”.

  • params – Additional parameters for the model. Default values are used for mandatory parameters.

  • timeout – Timeout for the LLM request in seconds. This parameter is currently ignored for Vertex AI models.

  • max_retries – Maximum number of retries for the LLM request. This parameter is currently ignored for Vertex AI models.

name: str
version: str | None
params: Dict | None
timeout: int | None
max_retries: int | None
model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].