Skip to main content

Using Orchestration Templates in LangGraph Agents

Introduction​

This tutorial shows how to use the OrchestrationClient from @sap-ai-sdk/langchain inside LangGraph workflows when a prompt template or template_ref is involved.

There are two behaviors to be aware of:

  • Inline template (template) — the template messages are always prepended to every request. Reusing the same client across multiple turns causes the template to be duplicated in each call.
  • Prompt Registry reference (template_ref) — the template is stored remotely and cannot be extended inline. Any messages passed alongside a template_ref are automatically routed to messages_history instead of being merged into the template.

The recommended pattern for both cases is the two-client approach: use one client with the template for the first node, and a second client without the template for all subsequent conversational nodes.

Prerequisites​

Refer to the prerequisites outlined here.

This tutorial assumes a basic understanding of TypeScript, LangGraph, and the orchestration client.

Installation​

npm install @sap-ai-sdk/langchain langchain @langchain/langgraph @langchain/core

Inline Template: Two-Client Approach​

When a template is defined on an OrchestrationClient and that client is reused across multiple turns, the template is prepended on every call. In a LangGraph agent loop this means the system prompt repeats with each turn. The SDK logs an info message on the first call and a warn on subsequent calls.

Use two clients: one with the template for the first node, one without for all subsequent nodes.

import { OrchestrationClient } from '@sap-ai-sdk/langchain';

// First node: template is applied once for the opening turn.
const clientWithTemplate = new OrchestrationClient({
  promptTemplating: {
    model: { name: 'gpt-5' },
    prompt: {
      template: [
        {
          role: 'system',
          content:
            'You are a helpful travel assistant. Always respond in a friendly tone.'
        }
      ]
    }
  }
});

// Subsequent nodes: no template, used for all follow-up turns.
const clientWithoutTemplate = new OrchestrationClient({
  promptTemplating: {
    model: { name: 'gpt-5' }
  }
});
tip

If you only need the system prompt once, you can also use a single client without a template and pass the system message as the first entry in messagesHistory on the opening call.

Prompt Registry Reference: Two-Client Approach​

When template_ref is configured, the orchestration service fetches the template from the Prompt Registry at request time. Messages cannot be merged into a remotely stored template — they are automatically routed to messages_history. The SDK logs a warn when messages are passed alongside a template_ref.

Use the same two-client pattern: one client with template_ref for the first node, one without for subsequent nodes.

import { OrchestrationClient } from '@sap-ai-sdk/langchain';

// First node: fetches and applies the remote template once.
const clientWithTemplateRef = new OrchestrationClient({
  promptTemplating: {
    model: { name: 'gpt-5' },
    prompt: {
      template_ref: {
        name: 'my-travel-assistant-template',
        scenario: 'travel',
        version: '1.0.0'
      }
    }
  }
});

// Subsequent nodes: no template_ref, messages are used directly.
const clientForFollowUp = new OrchestrationClient({
  promptTemplating: {
    model: { name: 'gpt-5' }
  }
});

How to create a LangGraph Workflow​

This section shows how to wire the clients into a complete multi-turn LangGraph workflow using the two-client pattern.

Two common entry points exist in LangChain:

  • createAgent() — two agents sharing a MemorySaver
  • StateGraph — full control over nodes, edges, and routing.

Option 1 — createAgent() with two agents​

Create two agents backed by the same MemorySaver instance: one with the template for the first turn, one without for all follow-up turns. MemorySaver keys state by thread_id only, so both agents read and write to the same conversation history. The template is part of the HTTP request config sent to the orchestration service — it is not stored in the checkpoint — so followUpAgent never re-applies it.

import { createAgent } from 'langchain';
import { OrchestrationClient } from '@sap-ai-sdk/langchain';
import { MemorySaver } from '@langchain/langgraph';

const clientWithTemplate = new OrchestrationClient({
  promptTemplating: {
    model: { name: 'gpt-5' },
    prompt: {
      template: [
        {
          role: 'system',
          content:
            'You are a helpful travel assistant. Always respond in a friendly tone.'
        }
      ]
    }
  }
});

const clientWithoutTemplate = new OrchestrationClient({
  promptTemplating: { model: { name: 'gpt-5' } }
});

// Shared checkpointer — both agents read and write the same thread history.
const memory = new MemorySaver();

const firstAgent = createAgent({
  model: clientWithTemplate,
  tools: [],
  checkpointer: memory
});

const followUpAgent = createAgent({
  model: clientWithoutTemplate,
  tools: [],
  checkpointer: memory
});

const threadConfig = { configurable: { thread_id: 'conv-1' } };

// First turn — template is applied once.
const first = await firstAgent.invoke(
  { messages: [{ role: 'user', content: 'Plan a day trip to Paris.' }] },
  threadConfig
);
console.log(first.messages.at(-1)?.content);

// Follow-up turn — history is loaded from the shared MemorySaver; template is not re-applied.
const followUp = await followUpAgent.invoke(
  { messages: [{ role: 'user', content: 'Add more outdoor activities.' }] },
  threadConfig
);
console.log(followUp.messages.at(-1)?.content);

Option 2 — StateGraph with the two-client pattern​

Use StateGraph when you need an explicit first-turn node that applies the template exactly once, then a separate conversational node for every subsequent turn.

import { OrchestrationClient } from '@sap-ai-sdk/langchain';
import {
  StateGraph,
  MessagesAnnotation,
  START,
  END
} from '@langchain/langgraph';
import { MemorySaver } from '@langchain/langgraph';

// First-turn client: carries the system-prompt template.
const clientWithTemplate = new OrchestrationClient({
  promptTemplating: {
    model: { name: 'gpt-5' },
    prompt: {
      template: [
        {
          role: 'system',
          content:
            'You are a helpful travel assistant. Always respond in a friendly tone.'
        }
      ]
    }
  }
});

// Follow-up client: no template — messages are passed as-is.
const clientWithoutTemplate = new OrchestrationClient({
  promptTemplating: { model: { name: 'gpt-5' } }
});

// Node executed only for the opening turn.
async function firstTurnNode(state: typeof MessagesAnnotation.State) {
  const response = await clientWithTemplate.invoke(state.messages);
  return { messages: [response] };
}

// Node executed for every subsequent turn.
async function conversationNode(state: typeof MessagesAnnotation.State) {
  const response = await clientWithoutTemplate.invoke(state.messages);
  return { messages: [response] };
}

// Route: if this is the first AI response, go to firstTurnNode, otherwise conversationNode.
function routeByTurn(state: typeof MessagesAnnotation.State): string {
  const hasAiMessage = state.messages.some(m => m._getType() === 'ai');
  return hasAiMessage ? 'conversationNode' : 'firstTurnNode';
}

const graph = new StateGraph(MessagesAnnotation)
  .addNode('firstTurnNode', firstTurnNode)
  .addNode('conversationNode', conversationNode)
  .addConditionalEdges(START, routeByTurn, [
    'firstTurnNode',
    'conversationNode'
  ])
  .addEdge('firstTurnNode', END)
  .addEdge('conversationNode', END)
  .compile({ checkpointer: new MemorySaver() });

const threadConfig = { configurable: { thread_id: 'conv-1' } };

// First turn — routes to firstTurnNode, template is applied once.
const first = await graph.invoke(
  { messages: [{ role: 'user', content: 'Plan a day trip to Paris.' }] },
  threadConfig
);
console.log(first.messages.at(-1)?.content);

// Follow-up — routes to conversationNode, no template duplication.
const followUp = await graph.invoke(
  { messages: [{ role: 'user', content: 'Add more outdoor activities.' }] },
  threadConfig
);
console.log(followUp.messages.at(-1)?.content);
tip

The same StateGraph approach applies to the template_ref two-client pattern — swap clientWithTemplate for clientWithTemplateRef in the firstTurnNode.

For a more advanced example with tool use, human-in-the-loop interruption, and MCP integration, see the Getting Started with Agents tutorial.

Further Reading​