gen_ai_hub.evaluations.utils.metric_client_utils module

gen_ai_hub.evaluations.utils.metric_client_utils.get_custom_metric_by_id(metric_id: str, ai_core_client: AICoreV2Client, resource_group: str, error_collector: ValidationCollector) → dict | None

Fetches a specific custom metric by its ID from the GenAI metrics server.

Note: Not using rest_client.get() because it converts camelCase to snake_case, but the Metric Management Service requires exact camelCase field names like ‘additionalProperties’ (would become ‘additional_properties’ if using rest_client).

Parameters:

  • metric_id (str) – The unique ID of the metric to retrieve.

  • ai_core_client (AICoreV2Client) – AI Core client instance for API access.

  • resource_group (str) – The resource group name.

  • error_collector (ValidationCollector) – ValidationCollector instance for collecting validation errors.

Returns:

Parsed JSON response as a dictionary, or None if not found.

Return type:

dict | None

gen_ai_hub.evaluations.utils.metric_client_utils.get_metric_template_info_from_server(metric: str, ai_core_client: AICoreV2Client, resource_group: str, error_collector: ValidationCollector)

Retrieves metric template information from the server by metric name.

Parameters:

  • metric (str) – The name of the metric to retrieve.

  • ai_core_client (AICoreV2Client) – AI Core client instance for API access.

  • resource_group (str) – The resource group name.

  • error_collector (ValidationCollector) – ValidationCollector instance for collecting validation errors.

Returns:

Metric information dictionary, or None if not found.

Return type:

dict | None

gen_ai_hub.evaluations.utils.metric_client_utils.get_metric_version_history(scenario: str, metric_id: str, version: str, ai_core_client: AICoreV2Client, resource_group: str, error_collector: ValidationCollector) → dict | None

Fetches the version history for a specific evaluation metric in a scenario.

Note: Not using rest_client.get() because it converts camelCase to snake_case, but the Metric Management Service requires exact camelCase field names.

Parameters:

  • scenario (str) – The name of the scenario.

  • metric_id (str) – The unique ID of the evaluation metric.

  • version (str) – The version of the metric.

  • ai_core_client (AICoreV2Client) – AI Core client instance for API access.

  • resource_group (str) – The resource group name.

  • error_collector (ValidationCollector) – ValidationCollector instance for collecting validation errors.

Returns:

Parsed JSON response as a dictionary, or None if not found.

Return type:

dict | None

gen_ai_hub.evaluations.utils.metric_client_utils.fetch_all_system_predefined_metrics(ai_core_client: AICoreV2Client, resource_group: str, error_collector: ValidationCollector)

Fetches all system-predefined metrics from the GenAI metrics server.

Parameters:

  • ai_core_client (AICoreV2Client) – AI Core client instance for API access.

  • resource_group (str) – The resource group name.

  • error_collector (ValidationCollector) – ValidationCollector instance for collecting validation errors.

Returns:

List of system-predefined metric templates.

Return type:

list

Raises:

RuntimeError – If fetching system predefined metrics fails.