Ship It — Your AI App in Production on Azure

Four chapters in, we have an app that answers from your data with citations (chapter 2), calls tools with guardrails (chapter 3), and proves its quality with evals in CI (chapter 4). It also runs exclusively on your laptop, authenticated as you. Time to fix that. This final chapter takes the app to production the way you'd want to find it if you inherited it: keyless, observable, rate-limited, and cheap.

The target architecture

Azure Container Apps (ACA) is my pick for hosting, and it earned it on my own site: it runs any container, scales to zero (a personal-project superpower — idle costs approach nothing), gives you managed TLS and a stable HTTPS endpoint, and — the decisive feature — supports system-assigned managed identity, which means the keyless story we've kept alive since chapter 1 survives contact with production.

User → ACA (FastAPI container, managed identity)
         ├→ Foundry project  (model + evals)   [Azure AI User]
         └→ AI Search        (retrieval)       [Search Index Data Reader]

No arrows carry API keys. Every hop is Entra ID.

Step 1 — Wrap the app in an API

Whatever framework you like; FastAPI is the community default:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
 
app = FastAPI()
 
class AskRequest(BaseModel):
    question: str = Field(max_length=500)   # input cap = cost cap
 
@app.post("/ask")
async def ask_route(req: AskRequest):
    answer = await ask(req.question)        # chapters 2-3, unchanged
    return {"answer": answer}

DefaultAzureCredential is the quiet hero here: this exact code authenticates as you locally and as the container's identity in ACA. No environment-specific auth branches.

A lean Dockerfile — pin versions, run as non-root:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
RUN useradd -m appuser
USER appuser
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Step 2 — Provision, deploy, grant

RG=rg-my-first-ai-app
az acr create -n acrmyfirstaiapp -g $RG --sku Basic
az acr build -r acrmyfirstaiapp -t ai-app:v1 .
 
az containerapp env create -n cae-ai-app -g $RG -l eastus2
az containerapp create -n ai-app -g $RG \
  --environment cae-ai-app \
  --image acrmyfirstaiapp.azurecr.io/ai-app:v1 \
  --registry-identity system --system-assigned \
  --ingress external --target-port 8000 \
  --min-replicas 0 --max-replicas 2 \
  --env-vars PROJECT_ENDPOINT=<your-project-endpoint> SEARCH_ENDPOINT=<your-search-endpoint>

Then the two role assignments that replace every API key we never created:

PRINCIPAL=$(az containerapp show -n ai-app -g $RG --query identity.principalId -o tsv)
 
# Call models + evals on the Foundry resource
az role assignment create --assignee-object-id $PRINCIPAL \
  --assignee-principal-type ServicePrincipal \
  --role "Azure AI User" --scope <foundry-resource-id>
 
# Query (not write) the search index
az role assignment create --assignee-object-id $PRINCIPAL \
  --assignee-principal-type ServicePrincipal \
  --role "Search Index Data Reader" --scope <search-service-id>

Note the asymmetry: the app can read the index but not write it. Least privilege isn't a slogan; it's role scoping like this, everywhere.

For repeatable deploys, a GitHub Actions workflow with OIDC federation (no stored credentials — GitHub proves its identity to Entra per-run) runs az acr build + az containerapp update, gated behind the chapter 4 eval job. Quality gate, then ship.

Step 3 — The production checklist

Things a public AI endpoint needs that a laptop demo doesn't:

Content safety. Azure AI Content Safety filters run on Foundry model deployments by default — review the configured severity thresholds in the portal rather than assuming defaults fit your app. Add Prompt Shields to screen inputs for jailbreaks and indirect injection (chapter 3's threat, now internet-facing).
Rate limiting + caps. Per-IP limits at your API layer, max_output_tokens on every model call, and that 500-character input cap. An AI endpoint without limits is a bill with a URL. Bot protection (Cloudflare Turnstile or similar) if it's fully public.
Observability. Wire your Foundry project to Application Insights and enable tracing; you get per-request traces covering retrieval, tool calls, and token usage. When someone reports a weird answer at 2 a.m., the trace shows which chunks were retrieved and what the model was told — debugging RAG blind is misery.
Budget alarm. az consumption budget create with an email alert at your pain threshold. Do it today; it takes ninety seconds and has saved more weekend projects than any other command in this series.
CORS lockdown to your actual frontend origin, structured logs on every tool invocation (chapter 3's audit trail), and no PII in any of them.

What the whole thing costs

Roughly, at hobby traffic, per month: ACR Basic ~$5 (the one fixed cost), ACA compute ~$0–3 with scale-to-zero, AI Search $0 on Free, model usage $1–5 for light traffic, App Insights $0 in the free grant. Call it $6–15/month — check current pricing pages, but the shape holds: one small fixed cost, everything else scales with use.

Wrapping up the series

Look at what's running: a containerized AI app on Container Apps, grounded in your documents via AI Search, calling tools with human approval on writes, quality-gated by evaluations in CI, traced end-to-end, rate-limited, content-filtered — and not one API key anywhere in the system.

More useful than the app is the loop you now own: deploy → ground → act → evaluate → ship. Every AI feature I build follows it, including the agents running on this very site — the chat assistant answering questions about me, and the blog pipeline that drafts posts as GitHub PRs. Swap the model when a better one lands, re-run the evals, ship with confidence.

Thanks for building along. If you ship something with this series, I'd genuinely love to hear about it — the contact form goes through an AI triage pipeline I built with these exact patterns, so your message will be well handled.