Governed AI Pull Request Review

Problem

An AI review can sound confident while missing the actual requirement, reading irrelevant context, or treating a suggestion as authority. A useful engineering workflow needs to control what the model receives, what tools it may use, how long it may run, and what action follows its output.

The goal was to turn AI review into a bounded evidence process rather than an unrestricted repository agent.

Constraints

The workflow operates around proprietary engineering work. Public documentation cannot expose private repository names, source code, work item contents, customer information, or internal tool configuration.

The model also could not own repository decisions. Research needed read only access. File paths and output locations needed validation. Long running or expensive analysis needed explicit limits.

Ownership

My responsibility covered the workflow shape, evidence packaging, guardrails, review stages, and auditable output. I worked within existing engineering and review practices rather than replacing repository owners or human reviewers.

The published architecture is sanitized. It describes the controls and data flow while withholding private inputs and implementation details.

Decisions

I separated research from review. The research stage gathers only the context needed to understand requirements and code. The review stage evaluates correctness and requirement coverage against that prepared evidence.

The workflow accepts requirements, a pull request diff, selected reference changes, and repository context as explicit inputs. That makes missing context visible before review begins.

Guardrails include path validation, read only tool access, timeouts, cost limits, source citations, and a known output location. The final output is a report for a human decision, not an automatic merge action.

Implementation

The evidence package defines the review target and the sources the workflow may inspect. A read only research pass resolves relevant context and records citations. A separate review pass turns that material into concrete findings tied to requirements and changed code.

Control logic rejects invalid paths, bounds execution, and writes results to an explicit location. The resulting report remains inspectable after the model run so reviewers can verify the evidence behind each finding.

Result

The result is a governed review path with a clear authority boundary. AI stages collect and analyze evidence. A human reviewer decides whether any finding should change the code or approval outcome.

The implementation reached a working, documented, and tested demonstration and retained one end to end Azure DevOps review run. The workflow did not reach an adoption decision, so this case study does not claim routine team use or organization wide deployment.

Private scripts, tests, and the retained run support claims about the implemented workflow. The public architecture explains the controls but does not independently prove implementation. No source supports a claim that the workflow finds every defect or improves review speed by a measured amount.

Limitations

The underlying requirements, code changes, repository integrations, test suite, and review report are private. No acceptance rate, defect count, cost reduction, or productivity number is published.

AI review remains dependent on the quality of the supplied requirements and repository context. Guardrails reduce risk, but they do not convert model output into ground truth.

Reflection

The most important component is the authority boundary after the model responds. A polished report is still a recommendation until a person checks its evidence and owns the action.

Separating research from review also improved clarity. One stage gathers facts. The next stage evaluates them against the requested change.