The GitHub Actions Flow — CI Checks, OIDC, and Evaluation Gates

Chapter 2 gave us the building blocks; now we wire them into pipelines. This chapter follows a single change — say, tightening the system prompt — from pull request to production, using the GitHub Actions workflows from Eric Chansen's foundry-agents-lifecycle repo (.github/workflows/ci.yml and cd.yml; credit where it's due).

The shape of the flow

PR opened          →  ci.yml: lint → unit tests → dry-run deploys
merge to main      →  cd.yml: DEV    deploy + evaluate   (automatic)
approval           →          TEST   deploy + evaluate   (1 approver)
approval           →          PROD   deploy + evaluate   (2+ approvers)

Two workflows, cleanly split: CI proves the change is valid, CD proves it's good — environment by environment, with humans at the gates.

CI: three checks before anyone merges

The PR pipeline runs in seconds and touches no Azure resources:

Lint (ruff check src/) — the usual hygiene.

Unit tests (pytest src/tests/unit/) — chapter 2's machinery checks: configs parse, required fields present, prompt files exist, agent names satisfy Foundry's naming rules, tool schemas valid.

Dry-run deploys — the underrated one:

python src/scripts/deploy_agent.py --env dev --dry-run
python src/scripts/deploy_agent.py --env prod --dry-run

This exercises the entire config-to-SDK-parameters path for every environment without calling Azure. The failure mode it catches — a config edit that's valid JSON but breaks parameter building, only for prod — is precisely the one that otherwise surfaces at 5 p.m. during a prod deploy. Red PR check instead. If CI fails, the PR can't merge; prompt changes get the same discipline as code because, as established in chapter 1, they are code.

CD step zero: authentication with no secrets

On merge to main, cd.yml logs into Azure like this:

permissions:
  id-token: write   # allows GitHub to mint an OIDC token
 
steps:
  - uses: azure/login@v2
    with:
      client-id: ${{ secrets.AZURE_CLIENT_ID }}
      tenant-id: ${{ secrets.AZURE_TENANT_ID }}
      subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

Those three values are identifiers, not credentials. The actual authentication is OIDC federation: GitHub presents a short-lived signed token proving "I am workflow X on branch Y of repo Z," and Entra ID — configured to trust exactly that claim — exchanges it for an Azure token. Nothing stored that can leak, nothing to rotate, and the federation subject can be pinned per environment so only main deploying to the prod environment can touch prod resources. It's the same pattern this website deploys with, and I'd call it non-negotiable for new pipelines in 2026.

The deploying identity needs only what the deploy script does: Azure AI User on each environment's Foundry resource. Least privilege, per environment.

Deploy, then interrogate

The DEV stage is two commands you already know from chapter 2:

python src/scripts/deploy_agent.py --env dev      # create_version → new agent version
python src/scripts/run_evaluation.py --env dev    # the interesting part

run_evaluation.py is where this stops being a generic deployment tutorial. It fires a dataset of test questions (src/tests/integration/eval_dataset.jsonl) at the freshly deployed agent and scores the responses with AI-assisted evaluators — the same groundedness/relevance machinery I covered in chapter 4 of my Build Your First AI App on Azure series, now standing guard in a pipeline. Score below threshold → the step fails → promotion stops. A bad prompt can reach dev, but it cannot reach prod, because between it and prod stand two more evaluations and two humans.

The graduated thresholds are a design worth copying:

| Environment | Threshold (of 5) | Philosophy | |---|---|---| | DEV | 3.0 | room to experiment | | TEST | 3.5 | shaping up | | PROD | 4.0 | earn it |

Promotion: GitHub Environments as the gates

The stages map to GitHub Environments (dev, test, prod), each carrying two things: required reviewers (none for dev, one for test, two-plus for prod) and environment-scoped identifiers for the per-environment Azure targets. When the DEV stage passes, the workflow simply pauses — reviewers get a notification, and the TEST stage starts only on approval. No plugins, no cron polling; it's built into Actions.

What makes the approval meaningful is the evidence attached to it. The approver isn't vibing — they're looking at a green dev evaluation with scores. "LGTM" backed by numbers.

The complete journey, replayed

Your prompt-tightening PR: CI validates it in seconds → reviewer sees a clean markdown diff of the prompt plus green checks → merge → DEV gets agent version N+1, evaluated at 3.0+ → teammate approves → TEST deploys and clears 3.5 → two approvers wave it into PROD → prod evaluation confirms 4.0+. Every step logged, every version numbered, rollback one git revert away.

Compare that to "I edited the prompt in the portal on Friday." Same five-minute change — utterly different risk profile.

Wrapping up

The GitHub flow adds three habits to chapter 2's building blocks: dry-runs in CI so config breaks die in the PR, OIDC so pipelines hold zero credentials, and evaluations as gates rather than dashboards — quality as a blocking check, not a retrospective.

Next — the finale: the same lifecycle in Azure DevOps for the ADO shops, what changes (service connections, ADO Environments, stage templates), what doesn't, and a checklist for adapting this whole pattern to your own agents.