Our SDK offers a developer-friendly way to consume foundational models available in the SAP generative AI hub. We strive to facilitate seamless interactions with these models by providing integrations that act as drop-in replacements for the native client SDKs and LangChain. This allows developers to use familiar interfaces and workflows.

Native Client Integrations

As of now, there are integrations with three types of native client SDKs (OpenAI, Google, Amazon).

The list of the available models can be found in the overview.

Completions

OpenAI

ChatCompletions equivalent to openai.ChatCompletions:

from gen_ai_hub.proxy.native.openai import chat

messages = [{"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
            {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
            {"role": "user", "content": "Do other Azure Cognitive Services support this too?"}]

kwargs = dict(model_name='gpt-4o-mini', messages=messages)
response = chat.completions.create(**kwargs)
print(response)

With model_version parameter:

from gen_ai_hub.proxy.native.openai import chat

messages = [{"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
            {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
            {"role": "user", "content": "Do other Azure Cognitive Services support this too?"}]

response = chat.completions.create(model_name='gpt-4o-mini', model_version="latest", messages=messages)
print(response)

With deployment_id instead of model_name:

from gen_ai_hub.proxy.native.openai import chat

messages = [{"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
            {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
            {"role": "user", "content": "Do other Azure Cognitive Services support this too?"}]

response = chat.completions.create(deployment_id="dcef02e219ae4916", messages=messages)
print(response)

Responses API

Responses equivalent to openai.Responses. See the OpenAI migration guide.

from gen_ai_hub.proxy.native.openai import responses

response = responses.create(
    model="gpt-5",
    instructions="You are a helpful assistant.",
    input="What is the capital of France?",
)
print(response.output_text)

Structured model outputs

LLM output as JSON objects is a powerful feature that allows you to define the structure of the output you expect from the model. See OpenAI structured outputs.

from pydantic import BaseModel
from gen_ai_hub.proxy.native.openai import chat, responses

class Person(BaseModel):
    name: str
    age: int

response = chat.completions.parse(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Tell me about John Doe, aged 30."}],
    response_format=Person
)
person = response.choices[0].message.parsed  # Fully typed Person
print(person)

# Using responses API
response = responses.parse(
    model="gpt-5",
    input="Tell me about John Doe aged 30.",
    text_format=Person
)
print(response.output_parsed)  # Fully typed Person

Google GenAI

Generate Content:

from gen_ai_hub.proxy.native.google_genai import Client
from gen_ai_hub.proxy import get_proxy_client

proxy_client = get_proxy_client('gen-ai-hub')
client = Client(proxy_client=proxy_client)

response = client.models.generate_content(model="gemini-3.5-flash",
    contents="How many paws are there for a dog?"
)
print(response)

response = client.models.generate_content(model="gemini-3.5-flash",
    contents="Explain the theory of relativity in simple terms."
)
print(response)

Generate Content streaming:

from gen_ai_hub.proxy.native.google_genai import Client
from gen_ai_hub.proxy import get_proxy_client

proxy_client = get_proxy_client('gen-ai-hub')
client = Client(proxy_client=proxy_client)

response_stream = client.models.generate_content_stream(model="gemini-3.5-flash",
    contents="Explain singularity in short terms.")

for chunk in response_stream:
    print("Chunk: ", chunk.text)

Function Calling:

from google.genai import types
from gen_ai_hub.proxy.native.google_genai import Client
from gen_ai_hub.proxy import get_proxy_client

def get_current_weather(location: str) -> str:
    """Returns the current weather.

    Args:
        location: The city and state, e.g. San Francisco, CA
    """
    return 'sunny'

proxy_client = get_proxy_client('gen-ai-hub')
client = Client(proxy_client=proxy_client)

chat = client.chats.create(
    model='gemini-3.5-flash',
    config=types.GenerateContentConfig(tools=[get_current_weather]),
)
response = chat.send_message('What is the weather like in Boston?')
print(response)

Amazon

Converse:

from gen_ai_hub.proxy.native.amazon import Session

bedrock = Session().client(model_name="anthropic--claude-4-sonnet")
conversation = [
    {
        "role": "user",
        "content": [{"text": "Describe the purpose of a 'hello world' program in one line."}],
    }
]
response = bedrock.converse(
    messages=conversation,
    inferenceConfig={"maxTokens": 512, "temperature": 0.5, "topP": 0.9},
)
print(response)

Embeddings

OpenAI

Embeddings are equivalent to openai.Embeddings. See the examples below for how to use Embeddings in the generative AI hub SDK.

from gen_ai_hub.proxy.native.openai import embeddings

response = embeddings.create(
    input="Every decoding is another encoding.",
    model_name="text-embedding-ada-002"
)
print(response.data)

With encoding format:

from gen_ai_hub.proxy.native.openai import embeddings

response = embeddings.create(
    input="Every decoding is another encoding.",
    model_name="text-embedding-ada-002",
    encoding_format='base64'
)
print(response.data)

Amazon

import json
from gen_ai_hub.proxy.native.amazon import Session

bedrock = Session().client(model_name="amazon--titan-embed-text")
body = json.dumps({"inputText": "Please recommend books with a theme similar to the movie 'Inception'."})
response = bedrock.invoke_model(body=body)
response_body = json.loads(response.get("body").read())
print(response_body)

LangChain Integration

LangChain provides an interface that abstracts provider-specific details into a common interface. Classes like Chat and Embeddings are interchangeable.

The list of the available models can be found in the overview.

Harmonized Model Initialization

The init_llm and init_embedding_model functions allow easy initialization of LangChain model interfaces:

from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from gen_ai_hub.proxy.langchain import init_llm
from gen_ai_hub.proxy import get_proxy_client

template = """Question: {question}
    Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=['question'])
question = 'What is a supernova?'

llm = init_llm('gpt-5.4-nano', max_tokens=300, proxy_client=get_proxy_client('gen-ai-hub'))
chain = prompt | llm | StrOutputParser()
response = chain.invoke({'question': question})
print(response)

from gen_ai_hub.proxy.langchain import init_embedding_model

text = 'Every decoding is another encoding.'
embeddings = init_embedding_model('text-embedding-ada-002')
response = embeddings.embed_query(text)
print(response)

LLM

from langchain_core.prompts import PromptTemplate
from gen_ai_hub.proxy.langchain import OpenAI
from gen_ai_hub.proxy import get_proxy_client

proxy_client = get_proxy_client('gen-ai-hub')
model_name = "mistralai--mistral-small-instruct"
llm = OpenAI(proxy_model_name=model_name, proxy_client=proxy_client)

template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = prompt | llm

question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"
print(llm_chain.invoke({'question': question}))

Chat Model

from langchain_core.prompts import (
    AIMessagePromptTemplate,
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from gen_ai_hub.proxy.langchain import ChatOpenAI
from gen_ai_hub.proxy import get_proxy_client

proxy_client = get_proxy_client('gen-ai-hub')
chat_llm = ChatOpenAI(proxy_model_name='gpt-4o-mini', proxy_client=proxy_client)

template = 'You are a helpful assistant that translates english to pirate.'
system_message_prompt = SystemMessagePromptTemplate.from_template(template)
example_human = HumanMessagePromptTemplate.from_template('Hi')
example_ai = AIMessagePromptTemplate.from_template('Ahoy!')
human_message_prompt = HumanMessagePromptTemplate.from_template('{text}')
chat_prompt = ChatPromptTemplate.from_messages(
    [system_message_prompt, example_human, example_ai, human_message_prompt])

chain = chat_prompt | chat_llm
response = chain.invoke({'text': 'I love planking.'})
print(response.content)

Structured model outputs

from gen_ai_hub.proxy.langchain import ChatOpenAI
from gen_ai_hub.proxy import get_proxy_client
from langchain_core.messages import HumanMessage
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

chat_model = ChatOpenAI(proxy_model_name="gpt-4o-mini", proxy_client=get_proxy_client('gen-ai-hub'))
chat_model = chat_model.with_structured_output(method="json_schema", schema=Person, strict=True)

message = HumanMessage(content="Tell me about a person named John who is 30")
print(chat_model.invoke([message]))

Embeddings

from gen_ai_hub.proxy.langchain import OpenAIEmbeddings
from gen_ai_hub.proxy import get_proxy_client

proxy_client = get_proxy_client('gen-ai-hub')
embedding_model = OpenAIEmbeddings(proxy_model_name='text-embedding-ada-002', proxy_client=proxy_client)
response = embedding_model.embed_query('Every decoding is another encoding.')

# Without proxy_client
embedding_model = OpenAIEmbeddings(proxy_model_name='text-embedding-ada-002')
response = embedding_model.embed_query('Every decoding is another encoding.')
print(response)

SAP RPT-1 Models

SAP-RPT-1 is a relational pretrained transformer for use on relational and structured data, developed and maintained by SAP. Relational Foundation Models (RFMs) are large-scale machine learning models designed to understand, process, and make predictions on tabular and relational data.

SAP-RPT-1 is a table-native model aiming to achieve the highest prediction quality and lowest error rates for predictions on tabular business data. It's pretrained and doesn't need additional training or fine-tuning steps. It solves predictive tasks such as classification and regression out-of-the-box via in-context learning. Due to its table-native architecture, prediction quality on enterprise data is typically very high, ahead of state-of-the-art narrow AI models and LLMs employed for such tasks.

You can get predictions directly by the native SDK client.

For detailed information visit the SAP Help Portal.

Regression

This is a simple example of RPTClient usage with minimal fields in the request body, using pydantic models.

from gen_ai_hub.proxy.native.sap import RPTRequest, PredictionConfig, TargetColumn, RPTClient

rows_regression = [
    {"PRODUCT": "Couch", "PRICE": 999.99, "ORDERDATE": "28-11-2025", "ID": "35", "DISCOUNT_RATE": "[PREDICT]"},
    {"PRODUCT": "Office Chair", "PRICE": 150.80, "ORDERDATE": "02-11-2025", "ID": "44", "DISCOUNT_RATE": 0.12},
    {"PRODUCT": "Server Rack", "PRICE": 2200.00, "ORDERDATE": "01-11-2025", "ID": "104", "DISCOUNT_RATE": 0.05},
    {"PRODUCT": "Standing Desk", "PRICE": 640.00, "ORDERDATE": "05-11-2025", "ID": "205", "DISCOUNT_RATE": 0.10},
    {"PRODUCT": "Monitor 27 inch", "PRICE": 289.99, "ORDERDATE": "08-11-2025", "ID": "306", "DISCOUNT_RATE": "[PREDICT]"},
]

client = RPTClient()
body = RPTRequest(
    prediction_config=PredictionConfig(
        target_columns=[TargetColumn(name="DISCOUNT_RATE", task_type="regression")]
    ),
    rows=rows_regression
)
response = client.predict(body=body, model_name="sap-rpt-1-small")
print(response.predictions)

# With model_version
response = client.predict(body=body, model_name="sap-rpt-1-small", model_version="latest")
print(response.predictions)

Classification

This example shows that you can also use a plain dictionary for RPTClient.

from gen_ai_hub.proxy.native.sap import RPTClient

client = RPTClient()
example_request_by_columns_dict = {
    "prediction_config": {
        "target_columns": [{"name": "COSTCENTER", "prediction_placeholder": "[PREDICT]", "task_type": "classification"}]
    },
    "columns": {
        "PRODUCT": ["Couch", "Office Chair", "Server Rack"],
        "PRICE": [999.99, 150.8, 2200.00],
        "ORDERDATE": ["28-11-2025", "02-11-2025", "01-11-2025"],
        "ID": ["35", "44", "104"],
        "COSTCENTER": ["[PREDICT]", "Office Furniture", "Data Infrastructure"]
    },
    "data_schema": {
        "PRODUCT": {"dtype": "string"},
        "PRICE": {"dtype": "numeric"},
        "ORDERDATE": {"dtype": "date"},
        "ID": {"dtype": "string"},
        "COSTCENTER": {"dtype": "string"}
    }
}

response = client.predict(body=example_request_by_columns_dict, model_name="sap-rpt-1-small")
print(response.predictions)

Async

The RPTClient also supports asynchronous calls via the apredict method.

await client.apredict(body=example_request_by_columns_dict, model_name="sap-rpt-1-small")

Completions​

OpenAI​

Responses API​

Structured model outputs​

Google GenAI​

Amazon​

Embeddings​

OpenAI​

Amazon​

LangChain Integration

Harmonized Model Initialization​

LLM​

Chat Model​

Structured model outputs​

Embeddings​

SAP RPT-1 Models

Regression​

Classification​

Async​

Completions

OpenAI

Responses API

Structured model outputs

Google GenAI

Amazon

Embeddings

OpenAI

Amazon

Harmonized Model Initialization

LLM

Chat Model

Structured model outputs

Embeddings

Regression

Classification

Async