Foundation Models

The @sap-ai-sdk/foundation-models package incorporates generative AI foundation models into your AI activities in SAP AI Core and SAP AI Launchpad.

Installation

npm install @sap-ai-sdk/foundation-models

Prerequisites

Refer to prerequisites outlined here.

Client Initialization

Initialize a client with the model name. If the model version is not specified, it is set to latest.

The current available clients are: - AzureOpenAiChatClient - AzureOpenAiEmbeddingClient

Take the AzureOpenAiChatClient as an example, use the following code to initialize a client:

import { AzureOpenAiChatClient } from '@sap-ai-sdk/foundation-models';

const client = new AzureOpenAiChatClient('gpt-4o');

The deployment ID will be implicitly fetched from the default resource group. By default, the deployment information is cached for five minutes, including deployment ID, model name, and model version.

When using other resource groups, you can specify the resourceGroup parameter.

const client = new AzureOpenAiChatClient({
  modelName: 'gpt-4o',
  // modelVersion: '2024-08-06', // optional
  resourceGroup: 'my-resource-group'
});

Alternatively, you can also provide a deployment ID instead of model name (and model version).

const client = new AzureOpenAiChatClient({
  // resourceGroup: 'my-resource-group', // optional
  deploymentId: 'd1234'
});

tip

When initializing a client to use a foundation model, it is equivalent to providing a combination of model name and model version, or a deployment ID to identify a model within a Resource Group. If multiple deployments were created with the same model name and version, the first deployment will be used.

Custom Destination

When initializing the client, it is possible to provide a custom destination for your SAP AI Core instance.

const client = await new AzureOpenAiChatClient('gpt-4o', {
  destinationName: 'my-destination'
});

By default, the fetched destination is cached. To disable caching, set the useCache parameter to false together with the destinationName parameter.

For more information about configuring a destination, refer to the Using a Destination section.

Custom Request Configuration

Set custom request configuration in the requestConfig parameter when calling the run() method of a client.

const response = await client.run(
  {
    ...
  },
  {
    headers: {
      'x-custom-header': 'custom-value'
      // Add more headers here
    },
    params: {
      // Add more parameters here
    }
    // Add more request configuration here
  }
);

This can be useful, e.g., to overwrite the API version of Azure OpenAI by defining the api-version parameter in the params object.

{
  params: {
    'api-version': 'AZURE_OPENAI_API_VERSION'
  }
}

Custom Headers

You can attach custom headers to your request, including SAP internal metering headers (for example, premium-billing and user-context), using the requestConfig.headers property. These internal metering headers are typically used for internal cost allocation and context propagation.

const response = await client.run(
  {
    messages: [{ role: 'user', content: 'What is the time in Tokyo?' }]
  },
  {
    requestConfig: {
      headers: {
        'premium-billing': 'premium-plan',
        'user-context': 'userId=12345;role=developer'
      }
    }
  }
);
console.log(response.getContent());

To apply these headers globally for all requests, configure them on a destination or when building/initializing the client.