Type Alias LLMModelDetails

The model and parameters to be used for the prompt templating. This is the model that will be used to generate the response.

type LLMModelDetails = {
    max_retries?: number;
    name: string;
    params?: Record<string, any>;
    timeout?: number;
    version?: string;
}

Index

Properties

max_retries? name params? timeout? version?

Properties

`Optional`max_retries

max_retries?: number

Maximum number of retries for the LLM request. This parameter is currently ignored for Vertex AI models. Default: 2. Maximum: 5.

name

name: string

Name of the model as in LLM Access configuration

Example

"gpt-4o-mini"

`Optional`params

params?: Record<string, any>

Additional parameters for the model. Default values are used for mandatory parameters.

Example

{
     *   "max_tokens": 300,
     *   "temperature": 0.1,
     *   "frequency_penalty": 0,
     *   "presence_penalty": 0,
     *   "n": 2,
     *   "stream_options": {
     *     "include_usage": true
     *   }
     * }

`Optional`timeout

timeout?: number

Timeout for the LLM request in seconds. This parameter is currently ignored for Vertex AI models. Default: 600. Maximum: 600. Minimum: 1.

`Optional`version

version?: string

Version of the model to be used Default: "latest".

Type Alias LLMModelDetails

Index

Properties

Properties

`Optional`max_retries

name

Example

`Optional`params

Example

`Optional`timeout

`Optional`version

Settings

On This Page

Type Alias LLMModelDetails

Index

Properties

Properties

Optionalmax_retries

name

Example

Optionalparams

Example

Optionaltimeout

Optionalversion

Settings

On This Page

`Optional`max_retries

`Optional`params

`Optional`timeout

`Optional`version