SLI & SLO

The Anatomy of an SLO

Every SLO needs three components: an SLI to measure, a target to hit, and a window to measure over. Get any of these wrong and the SLO is meaningless.

Key Numbers

Defining a Good SLI

A good SLI is a ratio of good events over total events. Both numerator and denominator must be measurable from production. The user experience — not internal health — is what defines "good".

{`# Availability SLI: fraction of requests that succeed
sli_availability = sum(rate(http_requests_total{status!~"5.."}[5m]))
                 / sum(rate(http_requests_total[5m]))

# Latency SLI: fraction of requests served quickly
# (must use histogram, not average; averages lie)
sli_latency = sum(rate(http_request_duration_seconds_bucket{le="0.3"}[5m]))
            / sum(rate(http_request_duration_seconds_count[5m]))

# Joint SLI: succeeds AND is fast
sli_joint = sum(rate(http_request_duration_seconds_bucket{
              le="0.3", status!~"5.."}[5m]))
          / sum(rate(http_request_duration_seconds_count[5m]))

# Freshness SLI for a data pipeline: rows updated within deadline
sli_freshness = count(time() - max_over_time(last_updated[1h]) < 3600)
              / count(last_updated)`}

The denominator matters. "Average latency" is not an SLI — it has no good/total ratio and one outlier ruins it. Histogram-based fractions are the standard form because they roll up cleanly and survive label aggregation.

Error Budget Math

The error budget is what you're allowed to "spend" on failures. It is denominated in the same units as your SLI: bad-event-fraction * window.

{`SLO target  = 99.9%
window      = 28 days = 28 * 86400 s = 2,419,200 s
budget      = (1 - 0.999) * 2,419,200 = 2,419.2 s = ~40 minutes

# If your service does 100M requests in 28 days:
budget_in_requests = 100_000_000 * 0.001 = 100,000 bad requests allowed

# Burn rate = current_bad_event_rate / budget_burn_rate_per_unit_time
# At nominal rate, you consume 1 / 28 * 100% of budget per day = 3.57%/day
# A burn rate of 1.0 = nominal pace. A burn rate of 14.4 = 100% of monthly
# budget gone in 2 days.

# Why 14.4?
14.4 = 28 days / (2 days)        # exhausts 28d budget in 2d
2.0  = 28 days / (14 days)       # exhausts in half the window
1.0  = sustainable for a window`}

Multi-Window Multi-Burn-Rate Alerts

Page on burn rate, not raw error rate. The Google SRE workbook recommends two complementary alerts: a fast one for "we're burning the whole monthly budget in 2 days" and a slow one for "we've been slowly bleeding for the last 6 hours."

{`groups:
  - name: api_slo_alerts
    rules:
      # Fast burn: 14.4x burn rate, requires confirmation in two windows
      - alert: ApiBurnRateFast
        expr: |
          (
            (1 - sli_availability_5m) > 14.4 * (1 - 0.999)
          and
            (1 - sli_availability_1h) > 14.4 * (1 - 0.999)
          )
        for: 2m
        annotations:
          summary: "API burning 14.4x normal rate - 100% of budget gone in 2 days"
          severity: page

      # Slow burn: 6x burn rate over a longer window
      - alert: ApiBurnRateSlow
        expr: |
          (
            (1 - sli_availability_30m) > 6 * (1 - 0.999)
          and
            (1 - sli_availability_6h) > 6 * (1 - 0.999)
          )
        for: 15m
        annotations:
          summary: "API burning 6x normal rate - 100% of budget gone in ~5 days"
          severity: ticket`}

The two-window AND prevents single-spike false positives. The fast alert catches true incidents quickly (5min + 1h). The slow alert catches gradual degradations (30min + 6h). Both fire long before the budget is exhausted, leaving time to fix the underlying issue.

Translating SLO to "9s"

The relationship between SLO percentage and allowed downtime is non-linear. Each additional 9 makes the goal 10x harder, not 1.1x harder.

SLA vs SLO

Internally, you target an SLO. Externally, you commit to an SLA. Always set the SLO stricter than the SLA — the SLO is your warning track before legal liability hits.

The SRE Workbook Approach

Google's Site Reliability Engineering book and the SRE Workbook codified the playbook most modern SLO practice descends from.

• SLOs are user-facing. They measure what users experience, not internal CPU or memory metrics.
• One SLO per user journey. Don't define 50 SLOs. Pick the 3-5 critical user paths and SLO each.
• Error budget > 100% uptime obsession. Burning down a budget intentionally (chaos testing, risky launches) is healthy. Refusing to spend any budget means you're under-shipping.
• Burn-rate alerts, not threshold alerts. "Latency > 300ms" pages on every spike. "We're using budget 14x normal" pages only when it actually matters.
• The error budget policy. Pre-agreed actions when the budget is exhausted: deploy freeze, on-call rotation tightens, postmortem-driven reliability sprint. Written down in advance, not negotiated mid-incident.

Tradeoffs

FAQ