Quality gates in the AI-native SDLC: designing trustworthy evidence

Consider a task to add discount codes. An agent changes the pricing logic, writes tests, and prepares the pull request description. Everything is consistent and green. The agent has assumed that discounts can be combined, although the business rule forbids it. The implementation, tests, and description confirm one another. All three are wrong.

AI increases the speed and scale of this kind of mistake. A large, internally consistent change can appear faster than a person can reconstruct its underlying assumptions. In a qualitative analysis of 1,110 responses from Google engineers, DORA describes this effect as a verification tax: some of the time saved during creation shifts to auditing, reviewing, and preparing the solution for production. This is not a universal measurement of the entire industry. Joint guidance from ANSSI and BSI states that increased development productivity requires QA, AppSec, and DevSecOps capacity to scale with it.

What does AI-native SDLC mean here?

In this article, it refers to a software delivery lifecycle in which AI, particularly coding agents, participates continuously in requirements, code, tests, documentation, and the delivery pipeline. The product itself does not need to contain an AI model.

The focus is on the way software is produced with AI assistance. A product that contains a model, memory, or autonomous actions needs additional evaluation of its behavior, data, and security. That is a separate scope that may exist in parallel.

A gate makes a decision

Not every mechanism that returns a negative result is a gate. Each mechanism has a different function in the quality system:

A linter provides a signal. An isolated environment limits the blast radius. Monitoring reveals system behavior. A gate connects a named decision object, a policy version, required evidence, and an authorized owner to a concrete response: block, promote, roll back, or grant an exception.

Risk first, autonomy second

The team needs a simple change profile. It should cover user and business impact, data sensitivity, access to privileged tools, architectural reach, reversibility, and uncertainty in the requirements. Legal and contractual obligations also matter.

Three classes are usually enough to start:

The profile must be recalculated when an agent expands the scope, adds a dependency, or changes a data flow, permission, or deployment method. NIST SSDF 1.1 describes risk-based and outcome-based practices that organizations adapt to their context. The same logic should drive quality gates.

An agent may propose a class and explain its reasoning. It should not lower the level of control on its own. An elevated or critical change requires confirmation by the named risk owner or an independent specialist.

A gate evaluates a specific object

A gate does not evaluate the general quality of a project. It decides whether a specific object may change state: a work item, an agent session, a source revision, a built artifact, or a deployment to a defined user cohort. Without that link, a team may accept test results from an earlier commit or provenance for a different application image.

In the model I use, the decision contract answers six questions:

1. What exactly is changing state, and what is the intent of the change?
2. Which harms, threats, and uncertainties are included in the risk profile?
3. Which policy version defines the criteria?
4. Which current results apply to this exact object?
5. Which decision and response follow from the evaluation?
6. Who may decide, how long is the decision valid, and what invalidates it?

This connects five perspectives: intent, workflow, validation, evidence assessment, and decision control. It is a model that I propose and use, not an industry standard.

What makes evidence trustworthy?

A green status tells us that a test passed. Its result supports a decision only when several conditions hold:

1. It evaluates the named risk or requirement.
2. It identifies the revision, configuration, environment, and artifact.
3. It comes from a trusted producer and retains its integrity.
4. It remains current, and no invalidating event has occurred.
5. It adds a method or perspective that is independent of the source of the change.

Independence does not always require another person. A property test, compiler, dependency scan, and production observation evaluate different properties. A test can be derived from a business rule or a threat model rather than from the current implementation. A second agent using the same model, instructions, and context may repeat the same mistake.

A practical traceability chain looks like this: requirement or threat -> check -> result and producer -> commit -> artifact -> deployment. A change to any link may require the evidence to be evaluated again.

Invalidation conditions should be encoded in the gate policy. A new commit, a rebuilt artifact, or a change to configuration, environment, or policy may mean that an earlier result no longer describes the decision object. The same is true when the producer’s integrity is compromised or when the model, instructions, or tools used for the assessment change. Trusted CI should record the result producer, time, configuration, and artifact identifier. The gate can then reject evidence that is green but belongs to a different system state.

Example: a gate that stops the change

Return to the discount-code task. The change receives the elevated class because it affects pricing and the contract between checkout and the service that calculates the amount due. The business rule forbids combining promotions. The risk analysis adds another condition: the order in which the discount and tax are applied must not produce a different final price.

The agent prepares the implementation and tests for the main paths. Both artifacts still inherit the assumption that two active codes are allowed. A separate property test is derived from the business rule rather than from the code. It generates combinations of promotions and detects a double price reduction. The result comes from a controlled CI and is linked to the exact commit, so it can be included in the evidence package.

The agent proposes adapting the assertion to the implementation. The protected policy prevents the business-rule test from being changed without review by the domain owner. The gate blocks the merge because the evidence package shows a violation of the pricing risk. After the implementation is corrected, the same commit passes the rule test, contract test, and the other controls required for an elevated change.

The mechanism works because the control has a different source, verification method, trusted producer, and authority to stop the change. Running another agent alone would not provide those properties. It would only create another signal that the team could ignore.

Project observation: migration from .NET 6 to .NET 8

In one project, I used AI to review project documentation and compare the code before migration with the version after moving from .NET 6 to .NET 8. The goal was to reconstruct expected behavior and identify areas where the migration might have changed the system.

Based on the analysis, we prepared tests targeted at the identified risks and defined quality-gate rules. AI agents helped compare the documentation, both code versions, and test results. We detected all known differences covered by the defined comparison criteria. Linking the results to the exact change reduced the risk of regression.

This was not a controlled experiment or evidence of universal AI effectiveness. The result depended on the quality of the documentation, the scope of the comparison, the test selection, and the decision rules. AI accelerated the analysis, while quality came from the complete process.

Where should controls be placed?

The sequence below identifies locations for decisions and evidence collection. It does not mean that every stage needs a separate gate.

Agent boundaries apply throughout the session. A separate identity, short-lived credentials, limited access to data and tools, and action logging reduce the blast radius. OWASP AISVS 1.0 organizes these requirements for AI tools and systems. Task content, repositories, and tool outputs should still be treated as potentially untrusted.

During promotion, the team must confirm that tests and scans apply to the exact artifact. Artifact origin information, known as provenance, helps establish where and how it was built. SLSA 1.2 describes this type of evidence. It does not confirm business correctness, so it remains one component of the package.

Deployment and user exposure are two decisions. A feature flag or progressive delivery allows the team to observe a limited cohort. The gate needs a threshold, observation window, and automatic response. Without a response, monitoring remains only a source of information.

An incident or serious regression should change the control system. The team preserves the evidence, identifies the cause, and names the control that failed or was missing. Findings update requirements, the threat model, tests, the gate policy, or monitoring. Each action has an owner and a due date. Closure requires a result showing that the improved control detects the class of error and reduces the risk of recurrence. Otherwise, the operation and learning stage ends with a report instead of a better process.

Exceptions and accountability

A failed rule should not disappear behind a status such as “conditionally passed.” An authorized exception records the decision object, the failed rule, the remaining risk, the compensating control, the approver, the expiry date, and the remediation owner. At expiry, the system requires a new assessment or blocks promotion.

Responsibility is distributed among several people:

1. The domain owner confirms intent and the acceptable business outcome.
2. The policy owner defines rules and required evidence.
3. An independent specialist evaluates the change within their expertise.
4. An authorized person accepts documented residual risk.
5. The release or service owner decides on promotion and recovery.

A Quality Engineer or Test Developer can design the evidence-and-independence model. That does not make this person the owner of every risk. An agent may collect results and identify inconsistencies, but it should not create the change, certify its own evidence, and approve promotion at the same time.

The engineer’s daily work also changes. Writing more tests does not resolve a shared incorrect assumption. The Quality Engineer or Test Developer determines which risks require evidence, where that evidence must come from, what invalidates it, and who may make the decision. The engineer also designs the response: a block, limited exposure, rollback, or an authorized exception.

This moves part of the work from implementing individual checks to designing a quality system. Technical skills, product knowledge, and the ability to build tests remain necessary. The role also assumes responsibility for ensuring consistency among risk, the execution environment, results, and the business decision.

How should the system be measured and piloted?

Metrics should answer questions about outcomes, effectiveness, and cost:

1. Are severe regressions and recovery time decreasing;
2. Which controls detect problems, and which generate false alarms;
3. How long does a decision take, and what is the verification cost;
4. How often are exceptions used, and are remediation actions closed;
5. Do progressive deployments stop defects before traffic expands?

Token counts, lines of code, pull requests, or AI usage do not measure quality. DORA warns against activity metrics that can become detached from value and delivery outcomes.

The first pilot should be limited to one important flow. The team selects a change, assigns a risk class, and names the decision objects. It then defines the evidence package, roles, and response to a negative result. The gate starts in observation mode. After false alarms have been calibrated, it can begin to block.

After several changes, the team reviews which controls stopped real defects, how much they cost, and whether exceptions were closed. This creates a basis for expanding the system without designing the entire organization at once.

Testing & QA

Ensure the quality, performance, and security of your software with our testing and test automation services.

Testing&QA offering

Conclusion

A quality gate in the AI-native SDLC should answer one question: which error is this control intended to stop?

The answer requires a named decision object, a risk profile, current evidence, its invalidation conditions, a policy version, an authorized owner, and a response to a negative result. The Quality Engineer or Test Developer designs the links between these elements, while the risk owner remains accountable for the decision.

A gate built this way allows a team to use agent speed without conflating higher change volume with higher quality. The team can start with one flow, measure control effectiveness, and extend the system based on the errors it can demonstrably stop.

***

Author’s note: The decision model, risk profile, and map of control locations are a practical synthesis that I propose and use. They are not an industry standard and need to be adapted to the organization’s context.