Package com.sap.ai.sdk.orchestration

Class OrchestrationClient

java.lang.Object

com.sap.ai.sdk.orchestration.OrchestrationClient

public class OrchestrationClient
extends Object

Client to execute requests to the orchestration service.

 chatCompletion()
 │
 ├── config(config, fallbackConfigs...)
 │   │
 │   └── promptTemplate(templateOrReference)
 │       ├── execute()
 │       ├── executeStream()
 │       └── executeStreamDeltas()
 │
 ├── configReference(configRef)
 │   ├── execute()
 │   ├── executeStream()
 │   └── executeStreamDeltas()
 │
 └── lowLevel()
     └── request(completionPostRequest)
         ├── execute()
         ├── executeStream()
         └── executeStreamDeltas()

 embeddings()
 │
 ├── request(orchestrationEmbeddingRequest)
 │   └── execute()
 │
 └── lowLevel()
     └── request(embeddingsPostRequest)
         └── execute()
 

High-level chat execution returns OrchestrationChatResponse; low-level execution returns CompletionPostResponse. Embedding execution similarly returns OrchestrationEmbeddingResponse or EmbeddingsPostResponse. Use lowLevel() only when supplying a custom generated request object.

Both chat APIs support text streaming and detailed deltas. Text streaming throws on an output content-filter finish reason; detailed streaming exposes that reason in the deltas. Close returned streams with try-with-resources. Streaming does not modify the configured request or shared stream options, so the same execution client can also be used for non-streaming calls.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	OrchestrationClient.ChatCompletionBuilder	Builder to select a local or referenced orchestration configuration for a chat completion.
class	OrchestrationClient.ChatCompletionExecutor	Client to execute a configured chat completion with a regular or streaming response.
class	OrchestrationClient.ChatCompletionWithConfig	Intermediate builder to select a prompt template for a local orchestration configuration.
class	OrchestrationClient.EmbeddingsBuilder	Builder to select a high-level or low-level embedding request.
class	OrchestrationClient.EmbeddingsExecutor	Client to execute a high-level embedding request.
class	OrchestrationClient.LowLevelChatCompletionBuilder	Builder to supply a custom low-level chat completion request.
class	OrchestrationClient.LowLevelChatCompletionExecutor	Client to execute a custom low-level chat completion request.
class	OrchestrationClient.LowLevelEmbeddingsBuilder	Builder to supply a custom low-level embedding request.
class	OrchestrationClient.LowLevelEmbeddingsExecutor	Client to execute a custom low-level embedding request.

Constructor Summary

Constructors

Constructor	Description
OrchestrationClient()	Default constructor.
OrchestrationClient(com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)	Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination.

Method Summary

Modifier and Type	Method	Description
OrchestrationClient.ChatCompletionBuilder	chatCompletion()	Create a builder to configure and execute a chat completion.
OrchestrationClient.EmbeddingsBuilder	embeddings()	Create a builder to configure and execute an embedding request.
static CompletionRequestConfiguration	toCompletionPostRequest(PromptTemplate template, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)	Convert the given prompt and config into a low-level request data object.
OrchestrationClient	withHeader(String key, String value)	Create a new orchestration client with a custom header added to every call made with this client.
OrchestrationClient	withHeaders(Map<String,String> headers)	Create a new orchestration client with multiple custom headers added to every call made with this client.
OrchestrationClient	withResourceGroup(String resourceGroup, String scenario)	Create a new orchestration client for the given resource group and scenario.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OrchestrationClient

public OrchestrationClient()

Default constructor.

OrchestrationClient

public OrchestrationClient(@Nonnull
 com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)

Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination. The destination needs to be configured with a URL pointing to an orchestration service deployment. Typically, such a destination should be obtained using AiCoreService.getInferenceDestination(String).

Example:

 new OrchestrationClient(new AiCoreService().getInferenceDestination("custom-rg").forScenario("orchestration"));
 

Parameters:

destination - the specific HttpDestination to use.

See Also:

• AiCoreService.getInferenceDestination(String)

Method Details

toCompletionPostRequest

@Nonnull
public static CompletionRequestConfiguration toCompletionPostRequest(@Nonnull
 PromptTemplate template,
 @Nonnull
 OrchestrationModuleConfig config,
 @Nonnull
 OrchestrationModuleConfig... fallbackConfigs)

Convert the given prompt and config into a low-level request data object. The data object allows for further customization before sending the request.

Parameters:

template - the PromptTemplate to generate a completion for.

config - the OrchestrationConfig configuration to use for the completion.

fallbackConfigs - fallback configurations to use.

Returns:

the low-level request data object to send to orchestration.

chatCompletion

@Nonnull
public OrchestrationClient.ChatCompletionBuilder chatCompletion()

Create a builder to configure and execute a chat completion.

Returns:

a builder to configure the chat completion

embeddings

@Nonnull
public OrchestrationClient.EmbeddingsBuilder embeddings()

Create a builder to configure and execute an embedding request.

Returns:

a builder to supply an embedding request or select the low-level API.

withHeader

@Nonnull
public OrchestrationClient withHeader(@Nonnull
 String key,
 @Nonnull
 String value)

Create a new orchestration client with a custom header added to every call made with this client.

Parameters:

key - the key of the custom header to add.

value - the value of the custom header to add.

Returns:

a new client.

Since:

1.11.0

withHeaders

@Nonnull
public OrchestrationClient withHeaders(@Nonnull
 Map<String,String> headers)

Create a new orchestration client with multiple custom headers added to every call made with this client.

Parameters:

headers - a map of key value pairs for the custom headers to add.

Returns:

a new client.

Since:

1.22.0

withResourceGroup

@Nonnull
public OrchestrationClient withResourceGroup(@Nonnull
 String resourceGroup,
 @Nonnull
 String scenario)

Create a new orchestration client for the given resource group and scenario.

Parameters:

resourceGroup - the resource group, usually "default".

scenario - the scenario.

Returns:

a new client configured with the resolved destination.