Skip to content

User Guide · Alerting

Automatically record and fire alerts when metrics go wrong.


Capabilities

CapabilityDescription
Threshold alertsFire when error rate, latency, throughput, etc. cross the line
Change detectionCatch sudden metric shifts
Scheduled evaluationRuns every minute, looking back 5 minutes
Event recordsTrack trigger, recovery, handling (auto-resolved when metrics recover)
AI analysisAsk for root cause directly from alert details

Evaluation mechanics: Architecture · Alerting.


FeaturePath
Detection rulesConfiguration → Alert Config
Alert listAlert Center → Alert List

Workflow

1. Create a Detection Rule

Configuration → Alert Config → New Rule

Configurable fields:

  • Scope: service or instance
  • Metrics: error rate, avg latency, P99 latency, request count, etc.
  • Condition: threshold (above/below) or change detection
  • Severity: Info / Warning / Critical
  • Evaluation: follows platform default (every minute)

2. View and Handle Alerts

Alert Center → Alert List

Filter by service, severity, status. Click an alert for details:

  • Abnormal metric trends
  • Related traces and logs
  • (Optional) AI root cause analysis, or Alert Center → Manual Root Cause Analysis for a time-range investigation
  • Handling log

Alerts auto-resolve when metrics recover.


Working with AI

From alert details or AI Platform:

"order-service error rate alert — help me analyze the cause"

AI queries metrics, traces, and topology automatically. For Agent integration, use MCP tool queryServiceAlarms — see Agent Integration.


FAQ

SymptomAction
No alerts after creating rulesEnsure services have metrics; evaluation runs every minute; verify rule scope (see Docker / K8s ops troubleshooting)
No alerts after Demo installCreate and enable a detection rule first; install demo app for traffic; wait 1–2 evaluation cycles
Too many alertsTune thresholds or narrow the monitoring scope

Alert Notification Channel (Webhook)

DataBuff provides a Webhook notification channel to push real-time system alerts and alarms to your custom HTTP endpoints. When an alarm event is triggered, the system automatically sends an HTTP POST request containing alert details in JSON format.

Configuration Parameters

To set up the alert notification channel, the following parameters are used within the configuration map:

Parameter KeyTypeDescriptionDefault
webhookUrlStringThe target HTTP/HTTPS URL where the JSON alert payload will be sent."" (Empty)
enabledBooleanToggles the notification channel system. Set to true to activate.false

Webhook Payload Format

The notification system transmits payloads with the header Content-Type: application/json. The timeout for connection is configured at 3 seconds, with a maximum request timeout of 5 seconds.

Supported Properties

The JSON payload contains the following alert fields:

  • alarmId (String): The unique identifier of the triggered alarm.
  • service (String): The name of the service or component generating the alert.
  • status (String): The current state of the alarm (e.g., triggered, resolved).
  • message (String): A detailed text description of the alarm event.

JSON Example

Below is an example of the JSON payload sent by the DataBuff system to your webhook endpoint:

json
{
  "alarmId": "ALARM-2026-0091",
  "service": "auth-service-vm",
  "status": "CRITICAL",
  "message": "Memory usage exceeded 92% on instance node-01."
}