Module 7c — AI Gateway: Protect & Implement

Goal: Put Cloudflare AI Gateway in front of the AI models your applications and agents call — so every programmatic AI request is authenticated, observable, cost-controlled, moderated for unsafe content, and scanned for sensitive data with your Zero Trust DLP profiles.

🧭 Where AI Gateway fits. Module 7 governs people using AI in a browser. Module 7b governs AI agents connecting to tools (MCP). AI Gateway governs the other direction: your own apps and agents calling AI models over the API. It's a proxy that sits between your code and the AI provider, so you get control and visibility without changing the model or the provider — and crucially, it works at the API layer with no device client or TLS decryption required.

   Your app / agent / coding tool
            │  (change base URL → gateway endpoint)
            ▼
   ┌─────────────────────────────────────────────┐
   │           CLOUDFLARE AI GATEWAY             │
   │  Auth ▸ Guardrails ▸ DLP ▸ Rate/Spend limits│
   │  ▸ Cache ▸ Logs ▸ Fallbacks                 │
   └───────────────────┬─────────────────────────┘
                       ▼
   AI providers: OpenAI · Anthropic · Workers AI · Google · …

Part A — Create & connect a gateway

1. 👉 In the Cloudflare dashboard, go to AI → AI Gateway.
2. 👉 Click Create Gateway, give it a name (e.g. production), and create it.
   • 📺 You'll see the gateway's OpenAI-compatible endpoint and setup guidance.
   • 💡 Shortcut: you can also use the gateway ID default — AI Gateway auto-creates it on the first request.
3. 👉 Point your app at the gateway. In your code, change the AI provider's base URL to your gateway endpoint:

   https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}
   

   For example, an OpenAI client's base URL becomes …/{account_id}/production/openai. No other code changes — the gateway forwards to the provider.
4. 👉 Send a test request from your app.

✅ Checkpoint: In AI → AI Gateway → your gateway, the request appears with metrics (request count, tokens, cost, latency). Traffic is now flowing through the gateway.

💡 No WARP client, no TLS decryption, no network changes — AI Gateway is an API-layer control point. That's what makes it the right tool for backend services, batch jobs, and autonomous agents.

Part B — Protect the gateway (Authenticated Gateway)

By default anyone who knows the endpoint URL could send requests through it. Lock it down so only your authorized callers can use it.

1. 👉 In your gateway, open Settings.
2. 👉 Enable Authenticated Gateway.
3. 👉 Create an AI Gateway authentication token and store it in your secrets manager.
4. 👉 Have your app send it on every request as a header:

   cf-aig-authorization: Bearer <your-aig-token>
   

✅ Checkpoint: requests without the token are rejected; requests with it succeed. Even if the endpoint URL leaks, it can't be abused.

⚠️ Watch out: this token authenticates use of the gateway; your provider API keys are separate. Consider BYOK / Store Keys (Settings) so the provider key lives in Cloudflare and never ships in your app code.

Part C — Guardrails (block unsafe content)

Guardrails moderate the content flowing through the gateway — catching harmful or inappropriate prompts and responses before they reach users or models.

1. 👉 In the gateway → Settings → Guardrails → enable.
2. 👉 Set the evaluation scope: moderate user prompts, model responses, or both.
3. 👉 Choose the hazard categories to monitor, and for each decide Block (stop it) or Flag (allow but log).
4. 👉 Save.

✅ Checkpoint: a prompt/response in a blocked hazard category is stopped (or flagged in logs), visible in the gateway's logs.

💡 Guardrails vs DLP: Guardrails handle safety (toxic, harmful, unsafe content); DLP (next part) handles sensitive data (PII, secrets, source code). Use both.

Part D — DLP for AI Gateway (scan for sensitive data) (Enterprise)

This is the Zero Trust integration: AI Gateway can scan the text of requests and responses against your Zero Trust DLP profiles — the same profiles you built in Module 6 — with no HTTP filtering and no TLS decryption.

1. 👉 In the gateway → Features → DLP → Set up.
2. 👉 Select the DLP profiles to apply — e.g. Credentials and Secrets, PII, or an AI Prompt profile (PII, Source Code, Financial, jailbreak/intent topics; even custom natural-language prompt topics).
3. 👉 Choose what to scan with Check: Request (prompts going to the model), Response (what the model returns), or Both.
4. 👉 Set the action — flag (log) or block — on a match.
5. 👉 Save.

✅ Checkpoint: a request containing a fake secret / test PII is flagged or blocked, and the detection shows in logs with the matched profile.

⚠️ Streaming caveat (important for latency): if you enable Response scanning, AI Gateway buffers the full provider response before returning it, which increases time-to-first-token — noticeable for chat/coding agents that stream. If you need low-latency streaming, set Check = Request only, or route latency-sensitive traffic through a separate gateway.

💡 Perfect for coding agents: tools like Cursor/Claude Code send source code, config, and secrets to model providers. Because AI Gateway sits in the path, DLP catches secrets/regulated data leaving (or returning) without modifying the agent.

Part E — Cost, rate & privacy controls

AI Gateway also gives you operational guardrails:

💡 Privacy tip: for regulated data, use cf-aig-collect-log-payload: false so you keep usage metrics (tokens, cost, model, status) without persisting sensitive prompt/response text.

⚠️ Don't enable caching or rate limiting on a gateway shared by an AI Search / RAG instance — it interferes with embedding/indexing. Use a dedicated gateway for that.

Part F — Observe & verify

1. 👉 In the gateway, review Analytics (requests, tokens, cost, cache hit rate, errors) and Logs (per-request: model, provider, status, cost, duration, user agent, and — unless disabled — payloads).
2. 👉 Filter logs to confirm your Auth, Guardrails, and DLP actions are firing as expected.
3. 👉 Test end-to-end: send (a) a normal request → succeeds; (b) a request missing the auth token → rejected; (c) a prompt with unsafe content → guardrail blocks; (d) a prompt with a test secret → DLP flags/blocks.

✅ Checkpoint: all four behave correctly and are visible in logs.

✅ Module 7c complete!

You now have:

• ✅ An AI Gateway in front of your AI providers (single endpoint, multi-provider)
• ✅ Authenticated Gateway — only authorized callers can use it
• ✅ Guardrails blocking unsafe content in prompts/responses
• ✅ DLP scanning prompts/responses for sensitive data using your Zero Trust profiles
• ✅ Cost, rate, cache, and privacy controls
• ✅ Full logging & analytics for every AI call

The complete AI security picture

Quick troubleshooting

👉 Next: Module 8 — Cloudflare WAN

Connect whole offices and data centers to Cloudflare's network.