agent_inspect.clients package

Submodules

agent_inspect.clients.azure_openai_client module

agent_inspect.clients.llm_client module

class agent_inspect.clients.llm_client.LLMClient[source]

Bases: ABC

This is a base abstract class that should be extended for actual implementations to connect to llm-as-a-judge model.

abstract async make_llm_request(prompt)[source]

This is an abstract method and should be implemented for concrete class to make LLM request to the LLM model.

Parameters:: prompt (str) – the user provided prompt to send to the model.
Return type:: LLMResponse
Returns:: LLMResponse object containing status code, completion and error message.

abstract async make_llm_requests(prompts)[source]

This is an abstract method and should be implemented for concrete class to make multiple LLM requests to the LLM model.

Parameters:: prompts (list[str]) – the user provided prompts to send to the model.
Return type:: list[LLMResponse]
Returns:: a List [LLMResponse] object containing status codes, completions and error messages.

abstract async make_request_with_payload(payload)[source]

This is an abstract method and should be implemented for concrete class to make LLM request to the LLM model with LLMPayload.

Parameters:: payload (LLMPayload) – the user provided LLMPayload to send to the model.
Return type:: LLMResponse
Returns:: LLMResponse object containing status code, completion and error message.

Module contents

Client subpackage.

LLMClient (the abstract base) is always importable — it has no optional dependencies. The concrete implementations are imported lazily so that import agent_inspect.clients succeeds even when the optional extras are not installed:

AzureOpenAIClient requires the azure-openai extra (openai, backoff).

LiteLLMClient requires litellm (and openai).

If you reference one of those names without the corresponding dependencies installed, you will get an ImportError pointing you at the right pip extra.

class agent_inspect.clients.LLMClient[source]