SAP AI SDK for JavaScript - v2.15.0
    Preparing search index...

    Function orchestrationPromptCachingMiddleware

    • Creates a prompt caching middleware for the Orchestration OrchestrationClient by delegating to the upstream anthropicPromptCachingMiddleware.

      When the conversation reaches the configured message threshold the upstream middleware injects cache_control into request.modelSettings. The OrchestrationClient then applies a single cache breakpoint to the last cacheable text block of the last message before the request is sent.

      Prompt caching is supported by Anthropic Claude and Amazon Nova model families served through Orchestration; other models simply ignore the directive. See the SAP AI Core prompt caching docs for the current matrix of supported models and breakpoint limits.

      Parameters

      • OptionalmiddlewareOptions: Partial<
            {
                enableCaching?: boolean;
                minMessagesToCache?: number;
                ttl?: "5m"
                | "1h";
                unsupportedModelBehavior?: "ignore" | "raise" | "warn";
            },
        >

        Configuration options forwarded verbatim to anthropicPromptCachingMiddleware. See PromptCachingMiddlewareConfig.

      Returns AgentMiddleware

      A middleware instance that can be passed to createAgent.

      This dedicated subpath depends on the optional langchain peer dependency. Importing @sap-ai-sdk/langchain/orchestration/prompt-caching-middleware fails during module resolution when langchain is not installed.

      If the upstream langchain middleware shape changes and contextSchema or wrapModelCall cannot be extracted.

      If unsupportedModelBehavior is set to 'raise' and the middleware is used with a non-Orchestration model.

      Basic usage with default settings

      import { createAgent } from 'langchain';
      import { OrchestrationClient } from '@sap-ai-sdk/langchain';
      import { orchestrationPromptCachingMiddleware } from '@sap-ai-sdk/langchain/orchestration/prompt-caching-middleware';

      const agent = createAgent({
        model: new OrchestrationClient({
          promptTemplating: { model: { name: 'anthropic--claude-4.5-haiku' } }
        }),
        middleware: [orchestrationPromptCachingMiddleware()]
      });

      Longer TTL with a higher message threshold

      orchestrationPromptCachingMiddleware({ ttl: '1h', minMessagesToCache: 5 });

    Settings
    Member Visibility