Skip to main content

Document Grounding

The Document Grounding module implements Retrieval Augmented Generation (RAG). It is a module in the Orchestration Service. It uses the SAP HANA Vector Engine to retrieve relevant document context (the "context") and generate more accurate responses.

Prerequisites​

A vector knowledge base is required. It can be created from:

  • Documents in a SharePoint folder, S3 storage, or SFTP repository
  • Text chunks fed directly via the Vector API

Another option is to use the help.sap.com elastic search endpoint.

Create a Vector Knowledge Base (S3 Example)​

In this example, an S3 data storage created by the user already has a set of documents uploaded to it. Check that Document Grounding is enabled, which allows retrieving the documents in the S3 bucket. The Pipelines API can be run via this SDK:

Make sure a Generic Secret for the S3 bucket has been created in AI Core so the documents can be retrieved from the bucket.

from gen_ai_hub.proxy import get_proxy_client
from gen_ai_hub.document_grounding import PipelineAPIClient, S3PipelineCreateRequest, CommonConfiguration

aicore_client = get_proxy_client()
pipelines_api_client = PipelineAPIClient(aicore_client)

generic_secret_s3_bucket = "<*** generic secret name for the S3 bucket ***>"
s3_config = S3PipelineCreateRequest(configuration=CommonConfiguration(destination=generic_secret_s3_bucket))
response = pipelines_api_client.create_pipeline(s3_config)
print(f"Pipeline ID: {response.pipelineId}")

After creating the pipeline, wait a few seconds before triggering or querying its status, as it may not be immediately available.

print(pipelines_api_client.get_pipeline_status(response.pipelineId))

Configuration​

Create a client for the Orchestration Service. If no api_url is provided, the service auto-discovers the deployment URL for the current resource group.

from gen_ai_hub.orchestration_v2.service import OrchestrationService
from gen_ai_hub.orchestration_v2.models.config import OrchestrationConfig, ModuleConfig
from gen_ai_hub.orchestration_v2.models.document_grounding import (
    GroundingModuleConfig, GroundingType, DataRepositoryType,
    DocumentGroundingConfig, DocumentGroundingFilter, GroundingSearchConfig,
    DocumentGroundingPlaceholders
)
from gen_ai_hub.orchestration_v2.models.llm_model_details import LLMModelDetails

orchestration_service = OrchestrationService()

llm = LLMModelDetails(name="gpt-4o-mini", params={'temperature': 0.0})

Define the Prompt​

from gen_ai_hub.orchestration_v2.models.message import SystemMessage, UserMessage
from gen_ai_hub.orchestration_v2.models.template import Template, PromptTemplatingModuleConfig

prompt = Template(template=[
    SystemMessage(content="You are an expert on SAP Product features."),
    UserMessage(content="""Context: {{ ?grounding_response }}
                   Question: What are the features of {{ ?product }}
                """),
])
filters = [DocumentGroundingFilter(id="SAPHelp", data_repository_type=DataRepositoryType.URL.value)]

grounding_config = GroundingModuleConfig(
    type=GroundingType.DOCUMENT_GROUNDING_SERVICE.value,
    config=DocumentGroundingConfig(
        placeholders=DocumentGroundingPlaceholders(input=["product"], output="grounding_response"),
        filters=filters
    )
)

config = OrchestrationConfig(
    modules=ModuleConfig(
        prompt_templating=PromptTemplatingModuleConfig(prompt=prompt, model=llm),
        grounding=grounding_config
    )
)
response = orchestration_service.run(config=config, placeholder_values={"product": "Generative AI Hub"})
print(response.final_result.choices[0].message.content)

Grounding via Custom Data Repository​

Assume the documentation for a custom product extension is vectorized and stored in the Vector knowledge base created earlier from the S3 bucket.

filters = [DocumentGroundingFilter(
    id="<*** product extension docs id ***>",
    data_repositories=["<*** data repository referencing the S3 pipeline id ***>"],
    search_config=GroundingSearchConfig(max_chunk_count=3),
    data_repository_type=DataRepositoryType.VECTOR.value
)]

grounding_config = GroundingModuleConfig(
    type=GroundingType.DOCUMENT_GROUNDING_SERVICE.value,
    config=DocumentGroundingConfig(
        placeholders=DocumentGroundingPlaceholders(input=["product"], output="grounding_response"),
        filters=filters
    )
)

config = OrchestrationConfig(
    modules=ModuleConfig(
        prompt_templating=PromptTemplatingModuleConfig(prompt=prompt, model=llm),
        grounding=grounding_config
    )
)
response = orchestration_service.run(
    config=config,
    placeholder_values={"product": "<*** custom extension name ***>"}
)
print(response.final_result.choices[0].message.content)

Show Retrieved Context​

print(response.intermediate_results.grounding.data['grounding_result'])

Data Masking of Retrieved Context​

The retrieved context can be masked in the same way as in the Orchestration Service, to avoid passing sensitive information to the LLM.

from gen_ai_hub.orchestration_v2.models.data_masking import (
    MaskingModuleConfig, MaskingProviderConfig, MaskingMethod,
    DPIStandardEntity, ProfileEntity, MaskGroundingInput
)

data_masking = MaskingModuleConfig(
    providers=[
        MaskingProviderConfig(
            method=MaskingMethod.ANONYMIZATION,
            entities=[DPIStandardEntity(type=ProfileEntity.SAP_IDS_INTERNAL)],
            mask_grounding_input=MaskGroundingInput(enabled=True)
        )
    ]
)

masking_config = OrchestrationConfig(
    modules=ModuleConfig(
        prompt_templating=PromptTemplatingModuleConfig(prompt=prompt, model=llm),
        grounding=grounding_config,
        masking=data_masking
    )
)
response = orchestration_service.run(
    config=masking_config,
    placeholder_values={"product": "<*** custom extension name ***>"}
)
print(response.final_result.choices[0].message.content)
print(response.intermediate_results.grounding.data['grounding_result'])