Deployment

Prompts are code. Models are dependencies. Deploy them like it — gateways, environments, version pinning, and rollouts that can't surprise you.

▶ Watch this reel

What you'll learn

LLM gateways
Environments & promotion
Version pinning
Feature flags & rollout

Remember this

AI gateways centralize auth, policy, observability, and provider abstraction — cross-cutting concerns at the boundary, config as reviewed code
Prompts/models/configs are artifacts: PR → eval-gated CI → dev/UAT/prod promotion, with versioned registries and environment parity
Pin everything by version/hash — updates become tested, reversible decisions; staged rollouts with feature flags and instant kills keep blast radii small

AI gateways

One front door: auth vault, rate limits/quotas, observability, provider abstraction + failover (APIM, LiteLLM, Portkey).
Config = reviewed code; verify streaming passthrough.

Environments & promotion

Prompts/models/configs are artifacts: PR → CI eval gates → dev/UAT/prod.
Registry with hashes; env parity except artifact versions; UAT on production-shaped data.

Version pinning

Pin model snapshots, prompt hashes, full config. Updates = eval-gated promotions.
Watch provider deprecation schedules; staging may float to catch drift early.

Feature flags & rollout

Per-request flags, canary 5→25→50→100 with metric gates, instant global kill.
Deploys become routine: small, reversible, observed.

Code: Promotion pipeline definition (the whole reel as YAML)

# prompt v42 change → this pipeline runs
pipeline:
  pr:
    gates: [diff_review, linked_eval_expectation]
  ci:
    evals:
      golden_set: { dataset: golden/v7, min_hit_rate: 0.92 }
      faithfulness: { min: 0.90 }
      agent_regression: { suite: agents/v3, allow_fail: 0 }
      security_canaries: { jailbreak_set: redteam/v2, contain: 100% }
  promote:
    dev:   automatic_on_ci_green
    uat:   automatic · shadow traffic 24h · compare vs prod
    prod:  flag_canary [5%, 25%, 50%, 100%]
           each_step_gated: [quality_delta < 2pts, cost_delta < 10%, errors < baseline]
           halt_on_red: true
  rollback: flag_to 0% + artifact v41 hot

# Everything versioned: prompts by hash, models by snapshot,
# configs in repo. 'Latest' appears nowhere.