Package com.sap.ai.sdk.orchestration

Class OrchestrationClient

java.lang.Object

com.sap.ai.sdk.orchestration.OrchestrationClient

public class OrchestrationClient
extends Object

Client to execute requests to the orchestration service.

Constructor Summary

Constructors

Constructor	Description
OrchestrationClient()	Default constructor.
OrchestrationClient(com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)	Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination.

Method Summary

Modifier and Type	Method	Description
OrchestrationChatResponse	chatCompletion(OrchestrationPrompt prompt, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)	Generate a completion for the given prompt.
OrchestrationChatResponse	chatCompletionUsingReference(OrchestrationConfigReference reference)	Generate a completion using a referenced Orchestration config.
EmbeddingsPostResponse	embed(EmbeddingsPostRequest request)	Generates embeddings using the low-level API request.
OrchestrationEmbeddingResponse	embed(OrchestrationEmbeddingRequest request)	Generate embeddings for a OrchestrationEmbeddingRequest request.
CompletionPostResponse	executeRequest(CompletionPostRequest request)	Serializes the given request, executes it and deserializes the response.
OrchestrationChatResponse	executeRequestFromJsonModuleConfig(OrchestrationPrompt prompt, String moduleConfig)	Perform a request to the orchestration service using a module configuration provided as JSON string.
Stream<String>	streamChatCompletion(OrchestrationPrompt prompt, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)	Generate a completion for the given prompt.
Stream<OrchestrationChatCompletionDelta>	streamChatCompletionDeltas(CompletionRequestConfiguration request)	Generate a completion for the given prompt.
static CompletionRequestConfiguration	toCompletionPostRequest(OrchestrationPrompt prompt, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)	Convert the given prompt and config into a low-level request data object.
OrchestrationClient	withHeader(String key, String value)	Create a new orchestration client with a custom header added to every call made with this client

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OrchestrationClient

public OrchestrationClient()

Default constructor.

OrchestrationClient

public OrchestrationClient(@Nonnull
 com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)

Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination. The destination needs to be configured with a URL pointing to an orchestration service deployment. Typically, such a destination should be obtained using AiCoreService.getInferenceDestination(String).

Example:

 new OrchestrationClient(new AiCoreService().getInferenceDestination("custom-rg").forScenario("orchestration"));
 

Parameters:

destination - The specific HttpDestination to use.

See Also:

• AiCoreService.getInferenceDestination(String)

Method Details

toCompletionPostRequest

@Nonnull
public static CompletionRequestConfiguration toCompletionPostRequest(@Nonnull
 OrchestrationPrompt prompt,
 @Nonnull
 OrchestrationModuleConfig config,
 @Nonnull
 OrchestrationModuleConfig... fallbackConfigs)

Convert the given prompt and config into a low-level request data object. The data object allows for further customization before sending the request.

Parameters:

prompt - The OrchestrationPrompt to generate a completion for.

config - The OrchestrationConfig configuration to use for the completion.

fallbackConfigs - Fallback configurations to use.

Returns:

The low-level request data object to send to orchestration.

chatCompletion

@Nonnull
public OrchestrationChatResponse chatCompletion(@Nonnull
 OrchestrationPrompt prompt,
 @Nonnull
 OrchestrationModuleConfig config,
 @Nonnull
 OrchestrationModuleConfig... fallbackConfigs)
                                         throws OrchestrationClientException

Generate a completion for the given prompt.

Parameters:

prompt - The OrchestrationPrompt to send to orchestration.

config - the configuration to use

fallbackConfigs - fallback configurations

Returns:

the completion output

Throws:

OrchestrationClientException - if the request fails.

streamChatCompletion

@Nonnull
public Stream<String> streamChatCompletion(@Nonnull
 OrchestrationPrompt prompt,
 @Nonnull
 OrchestrationModuleConfig config,
 @Nonnull
 OrchestrationModuleConfig... fallbackConfigs)
                                    throws OrchestrationClientException

Generate a completion for the given prompt.

Parameters:

prompt - a text message.

config - the configuration to use

fallbackConfigs - fallback configurations

Returns:

a stream of message deltas

Throws:

OrchestrationClientException - if the request fails or if the finish reason is content_filter

Since:

1.1.0

executeRequest

@Nonnull
public CompletionPostResponse executeRequest(@Nonnull
 CompletionPostRequest request)
                                      throws OrchestrationClientException

Serializes the given request, executes it and deserializes the response.

Override this method to customize the request execution. For example, to modify the request object before it is sent, use:

 @Override
 protected CompletionPostResponse executeRequest(@Nonnull CompletionPostRequest request) {
   request.setCustomField("myField", "myValue");
   return super.executeRequest(request);
 }
 

Alternatively, you can call this method directly with a fully custom request object.

Parameters:

request - The request data object to send to orchestration.

Returns:

The response data object from orchestration.

Throws:

OrchestrationClientException - If the request fails.

chatCompletionUsingReference

@Beta
@Nonnull
public OrchestrationChatResponse chatCompletionUsingReference(@Nonnull
 OrchestrationConfigReference reference)

Generate a completion using a referenced Orchestration config.

Parameters:

reference - A reference to an Orchestration config stored in prompt registry

Returns:

The completion output

Since:

1.15.0

executeRequestFromJsonModuleConfig

@Beta
@Nonnull
public OrchestrationChatResponse executeRequestFromJsonModuleConfig(@Nonnull
 OrchestrationPrompt prompt,
 @Nonnull
 String moduleConfig)
                                                             throws OrchestrationClientException

Perform a request to the orchestration service using a module configuration provided as JSON string. This can be useful when building a configuration in the AI Launchpad UI and exporting it as JSON. Furthermore, this allows for using features that are not yet supported natively by the API.

NOTE: This method does not support streaming.

Parameters:

prompt - The input parameters and optionally message history to use for prompt execution.

moduleConfig - The module configuration in JSON format.

Returns:

The completion response.

Throws:

OrchestrationClientException - If the request fails.

streamChatCompletionDeltas

@Nonnull
public Stream<OrchestrationChatCompletionDelta> streamChatCompletionDeltas(@Nonnull
 CompletionRequestConfiguration request)
                                                                    throws OrchestrationClientException

Generate a completion for the given prompt.

Parameters:

request - the prompt, including messages and other parameters.

Returns:

A stream of chat completion delta elements.

Throws:

OrchestrationClientException - if the request fails

Since:

1.1.0

embed

@Nonnull
public OrchestrationEmbeddingResponse embed(@Nonnull
 OrchestrationEmbeddingRequest request)
                                     throws OrchestrationClientException

Generate embeddings for a OrchestrationEmbeddingRequest request.

Parameters:

request - the request containing the input text and other parameters.

Returns:

the response containing the embeddings.

Throws:

OrchestrationClientException - if the request fails

Since:

1.12.0

embed

@Nonnull
public EmbeddingsPostResponse embed(@Nonnull
 EmbeddingsPostRequest request)
                             throws OrchestrationClientException

Generates embeddings using the low-level API request.

This method provides direct access to the underlying API for advanced use cases. For most scenarios, prefer embed(OrchestrationEmbeddingRequest).

Parameters:

request - the low-level API request

Returns:

the low level response object

Throws:

OrchestrationClientException - if the request fails

Since:

1.12.0

See Also:

• embed(OrchestrationEmbeddingRequest)

withHeader

@Beta
@Nonnull
public OrchestrationClient withHeader(@Nonnull
 String key,
 @Nonnull
 String value)

Create a new orchestration client with a custom header added to every call made with this client

Parameters:

key - the key of the custom header to add

value - the value of the custom header to add

Returns:

a new client.

Since:

1.11.0