Package com.sap.ai.sdk.orchestration

Class OrchestrationClient

java.lang.Object
com.sap.ai.sdk.orchestration.OrchestrationClient
public class OrchestrationClient extends Object
Client to execute requests to the orchestration service.

  • Constructor Details

    • OrchestrationClient

      public OrchestrationClient()
      Default constructor.

    • OrchestrationClient

      public OrchestrationClient(@Nonnull com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)
      Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination. The destination needs to be configured with a URL pointing to an orchestration service deployment. Typically, such a destination should be obtained using AiCoreService.getInferenceDestination(String).

      Example: 

      
 new OrchestrationClient(new AiCoreService().getInferenceDestination("custom-rg").forScenario("orchestration"));
      Parameters:
      destination - The specific HttpDestination to use.
      See Also:

  • Method Details

    • toCompletionPostRequest

      @Nonnull public static CompletionRequestConfiguration toCompletionPostRequest(@Nonnull OrchestrationPrompt prompt, @Nonnull OrchestrationModuleConfig config, @Nonnull OrchestrationModuleConfig... fallbackConfigs)
      Convert the given prompt and config into a low-level request data object. The data object allows for further customization before sending the request.
      Parameters:
      prompt - The OrchestrationPrompt to generate a completion for.
      config - The OrchestrationConfig configuration to use for the completion.
      fallbackConfigs - Fallback configurations to use.
      Returns:
      The low-level request data object to send to orchestration.

    • chatCompletion

      @Nonnull public OrchestrationChatResponse chatCompletion(@Nonnull OrchestrationPrompt prompt, @Nonnull OrchestrationModuleConfig config, @Nonnull OrchestrationModuleConfig... fallbackConfigs) throws OrchestrationClientException
      Generate a completion for the given prompt.
      Parameters:
      prompt - The OrchestrationPrompt to send to orchestration.
      config - the configuration to use
      fallbackConfigs - fallback configurations
      Returns:
      the completion output
      Throws:
      OrchestrationClientException - if the request fails.

    • streamChatCompletion

      @Nonnull public Stream<String> streamChatCompletion(@Nonnull OrchestrationPrompt prompt, @Nonnull OrchestrationModuleConfig config, @Nonnull OrchestrationModuleConfig... fallbackConfigs) throws OrchestrationClientException
      Generate a completion for the given prompt.
      Parameters:
      prompt - a text message.
      config - the configuration to use
      fallbackConfigs - fallback configurations
      Returns:
      a stream of message deltas
      Throws:
      OrchestrationClientException - if the request fails or if the finish reason is content_filter
      Since:
      1.1.0

    • executeRequest

      @Nonnull public CompletionPostResponse executeRequest(@Nonnull CompletionPostRequest request) throws OrchestrationClientException
      Serializes the given request, executes it and deserializes the response.

      Override this method to customize the request execution. For example, to modify the request object before it is sent, use: 

      
 @Override
 protected CompletionPostResponse executeRequest(@Nonnull CompletionPostRequest request) {
   request.setCustomField("myField", "myValue");
   return super.executeRequest(request);
 }

      Alternatively, you can call this method directly with a fully custom request object.

      Parameters:
      request - The request data object to send to orchestration.
      Returns:
      The response data object from orchestration.
      Throws:
      OrchestrationClientException - If the request fails.

    • chatCompletionUsingReference

      @Nonnull public OrchestrationChatResponse chatCompletionUsingReference(@Nonnull OrchestrationConfigReference reference)
      Generate a completion using a referenced Orchestration config.
      Parameters:
      reference - A reference to an Orchestration config stored in prompt registry
      Returns:
      The completion output
      Since:
      1.15.0

    • executeRequestFromJsonModuleConfig

      @Beta @Nonnull public OrchestrationChatResponse executeRequestFromJsonModuleConfig(@Nonnull OrchestrationPrompt prompt, @Nonnull String moduleConfig) throws OrchestrationClientException
      Perform a request to the orchestration service using a module configuration provided as JSON string. This can be useful when building a configuration in the AI Launchpad UI and exporting it as JSON. Furthermore, this allows for using features that are not yet supported natively by the API.

      NOTE: This method does not support streaming.

      Parameters:
      prompt - The input parameters and optionally message history to use for prompt execution.
      moduleConfig - The module configuration in JSON format.
      Returns:
      The completion response.
      Throws:
      OrchestrationClientException - If the request fails.

    • streamChatCompletionDeltas

      @Nonnull public Stream<OrchestrationChatCompletionDelta> streamChatCompletionDeltas(@Nonnull CompletionPostRequest request) throws OrchestrationClientException
      Generate a completion for the given request object.

      Note, that streaming will get enabled on the request object as a side effect.

      Parameters:
      request - the prompt, including messages and other parameters.
      Returns:
      A stream of chat completion delta elements.
      Throws:
      OrchestrationClientException - if the request fails
      Since:
      1.1.0

    • embed

      @Nonnull public OrchestrationEmbeddingResponse embed(@Nonnull OrchestrationEmbeddingRequest request) throws OrchestrationClientException
      Generate embeddings for a OrchestrationEmbeddingRequest request.
      Parameters:
      request - the request containing the input text and other parameters.
      Returns:
      the response containing the embeddings.
      Throws:
      OrchestrationClientException - if the request fails
      Since:
      1.12.0

    • embed

      @Nonnull public EmbeddingsPostResponse embed(@Nonnull EmbeddingsPostRequest request) throws OrchestrationClientException
      Generates embeddings using the low-level API request.

      This method provides direct access to the underlying API for advanced use cases. For most scenarios, prefer embed(OrchestrationEmbeddingRequest).

      Parameters:
      request - the low-level API request
      Returns:
      the low level response object
      Throws:
      OrchestrationClientException - if the request fails
      Since:
      1.12.0
      See Also:

    • withHeader

      @Nonnull public OrchestrationClient withHeader(@Nonnull String key, @Nonnull String value)
      Create a new orchestration client with a custom header added to every call made with this client
      Parameters:
      key - the key of the custom header to add
      value - the value of the custom header to add
      Returns:
      a new client.
      Since:
      1.11.0

    • withHeaders

      @Nonnull public OrchestrationClient withHeaders(@Nonnull Map<String,String> headers)
      Create a new orchestration client with multiple custom headers added to every call made with this client
      Parameters:
      headers - a map of key value pairs for the custom headers to add
      Returns:
      a new client.
      Since:
      1.22.0

    • withResourceGroup

      @Nonnull public OrchestrationClient withResourceGroup(@Nonnull String resourceGroup, @Nonnull String scenario)
      Create a new orchestration client for the given resource group and scenario.
      Parameters:
      resourceGroup - the resource group
      scenario - the scenario
      Returns:
      a new client configured with the resolved destination.