Your software partner says the team uses AI coding tools and delivers faster. That may be true, and it may help you. It may also mean a lot more code is arriving that someone has to check.
A new study of 718 companies found that teams using AI coding agents wrote far more code but did not finish measurably more work. This guide explains what that means when you pay a vendor for software, and gives you a plain list of what to ask for before you sign or renew.
Why more code written does not mean fewer problems
The study, a Harvard working paper by Fiona Chen and James Stratton, used data from the engineering analytics platform Jellyfish covering 718 firms and 300 million work events up to March 2026. After companies adopted AI coding agents, lines of code rose 30%, commits 20% and pull requests (the packages of changes sent for review) 23%. But the authors found no statistically significant increase in resolved Jira issues or completed projects, and their estimates rule out an output gain larger than 12%.
The reason they point to is review. After adoption, the average time to review a pull request rose 49%, the share of requests sent back for changes nearly doubled, and comments per request rose 35%. More code arrived, and checking it took longer. News coverage headlined this as "zero additional issues resolved". The paper itself says no statistically significant increase, which is a little different, and it is a working paper that has not been peer reviewed, based on one platform's customers rather than outsourced teams.
Two other findings point the same way:
Security. Veracode tested 80 coding tasks across more than 100 AI models and found that the models introduced security vulnerabilities in 45 percent of cases. That is a failure rate on controlled tasks, not a share of all AI-written code, but it is a good reason for checks.
Good teams gain most. Google's 2025 DORA report, a survey of nearly 5,000 technology professionals, describes AI as a mirror and a multiplier: teams with strong habits get more efficient, while weaknesses show up in fragmented teams.
For a buyer, the lesson is simple. You are paying for finished, working features, not lines of code. Whether AI helps you depends on how your vendor reviews and tests what it produces.
Ask the vendor which AI tools the team uses and where
Do not ask "do you use AI?" Almost every team does. Ask for specifics, and ask for them in writing:
Which tools? A suggestion tool that completes a line is different from an agent that works through a whole task on its own.
For what work? Drafting routine code and tests is lower risk than payment logic, login or data handling.
Where does your code go? Which of your code and data is sent to the AI provider, and what do its terms say about storing it or training on it?
What can the agent touch? An agent working in your code repository should have its own account with limited access and a log, as in our guide to AI agent access control.
Is it disclosed? Ask for a short written AI use policy and for notice if the tools change.
Ask who reviews AI-written code and how
The study's strongest message is that review is where the time goes. A vendor that has not planned for it will either slow down or quietly skip it. Ask:
Is every change checked by a person who did not write or prompt it, and are the reviewers named?
How large can one change be? Very large pull requests are hard to review properly.
Does the author have to explain what a change does and why before it is accepted?
If an AI tool helps with review, who makes the final decision? The study notes that firms increasingly use AI to assist review, yet human reviewers still play a central role, and that is the right setup. See our guide to human-in-the-loop AI for how to stop approvals turning into a rubber stamp.
Then ask for three numbers each month: how long changes wait for review, what share are sent back for fixes, and how many defects are found after release. A team that tracks these is managing quality. A team that cannot produce them is guessing.
Tests, reviews and security checks to request in the contract
Put the checks you care about in the statement of work, so they apply whoever or whatever writes the code. Have your lawyer review the final wording.
Ask for | What good looks like |
|---|---|
Automated tests | New code ships with tests, and the full test suite runs on every change with the result visible to you |
Peer review | Every change reviewed by a named person other than the author, before it is merged |
Security scanning | Code and dependency scans plus a check for leaked passwords and keys on every change; serious findings fixed before release |
Acceptance criteria | Each feature has written criteria that you agree before work starts and sign off when it is done |
Ownership and access | You own the deliverables, including code produced with AI tools, and you can see the repository at any time |
Defect warranty | Defects found within an agreed period are fixed at no extra charge |
AI disclosure | The AI tools in use are listed, and you are told before they change |
The contract model also changes who carries the cost of review. Under hourly billing, extra review time can land on your invoice. Under a fixed price, it lands on the vendor, who may be tempted to skip it. Our comparison of staff augmentation, dedicated teams and fixed-price projects covers how each model handles this.
Signs a team is using AI to rush rather than to improve
You do not need to read code to spot these:
Demos look impressive, but nobody can show you the tests.
Progress is reported in commits, lines of code or tickets closed rather than working features you can try.
Changes are approved within minutes, or the same person writes and approves them.
The team cannot explain why a piece of code works the way it does.
The same bugs keep coming back, or fixes break other things.
The estimate got much cheaper and faster, but the scope, the tests and the review time did not change.
One of these in isolation is not a verdict. Several together are a reason to ask for a review of the process.
A short list of questions for your next vendor call
Which AI coding tools does the team use, and for which parts of our project?
What do we send to the AI provider, and what do its terms say about our code?
Who reviews AI-written code, and can the author also approve it?
What are your review times, send-back rate and post-release defect numbers?
Which tests will run on every change, and can we see the results?
What security scans run, and what happens to serious findings?
Who owns the code, and does that include code produced with AI tools?
If AI makes the work faster, how does that show up in the schedule, the price or the quality?
Where to start
You do not have to ban AI tools. Ask for disclosure, review, tests and a way to see the results, and a good partner will welcome the questions. If you would like a second opinion on a vendor proposal or an existing codebase, our software consulting team can go through it with you, and our custom software development team can talk through how it would run a project like yours. Book a call to start the conversation.
