Skip to content

AI code review: what it catches and what it misses

Guide/ 12 min read

What AI code review catches (unhandled cases, names, missing tests) and what it misses (business rules, decisions, security context), and how to split the work with a person.

  • AI
  • QA
In this article (8)
  1. What AI code review catches: the mechanical layer
  2. What is not in the code: business rules, decisions and security context
  3. Two filters: AI first, then a person
  4. Give the reviewer context
  5. When not to approve
  6. Common mistakes
  7. Frequently asked questions
  8. Checklist

AI code review is good at the mechanical layer: bugs you can show with one input, unhandled cases, misleading names and missing tests. It misses what is not in the code: business rules, product decisions and security context. Use it as the first filter and leave the approval to a person who knows those.

This is not a tool comparison or a study with numbers. It is how we split the work in practice, with an example change we ran, the instructions file we hand the reviewer, and the rule that makes a person approve sensitive paths. If you want to set the reviewer up as a subagent, that is in the code reviewer subagent article; this one is about what to expect from it and what not to.

What AI code review catches: the mechanical layer

Mechanical means anything you can check by reading the diff and the code around it, without knowing anything about the business. Google's code review guide (what to look for in a code review) asks reviewers to check functionality, complexity, tests, naming and comments, and to look at every line they were assigned. That last part is what an AI reviewer does without getting tired: it does not skip file 14 of a 20-file change.

Here is a change of realistic size and shape, with illustrative numbers. It computes a cart's shipping cost:

// lib/shipping.js: the change under review
export function shippingCost(cart) {
  const subtotal = cart.items.reduce((sum, item) => sum + item.price * item.qty, 0);
  return subtotal >= 999 ? 0 : 149;
}
  1. shippingCost takes the whole cart.
  2. subtotal adds price times quantity for each item, starting at 0.
  3. If the subtotal reaches 999, shipping is free; otherwise it costs 149.

We ran it on Node 22. A cart with no items throws TypeError: Cannot read properties of undefined (reading 'reduce'); a negative quantity returns 149 without complaint; exactly 999 returns 0. That is what an AI reviewer tends to flag on a change like this, and it can prove each point with an input:

  • Unhandled cases: undefined cart.items, negative or fractional quantities.
  • Magic numbers: 999 and 149 have no name; a named constant says what they are and leaves one place to change them.
  • Missing tests: nothing covers the exact threshold (999) or the empty cart.
  • Naming: subtotal is fine here, but a reviewer flags a name that promises something other than what the code computes.

It is also good at what repeats across a codebase: a missing await, an error swallowed by an empty catch, an environment variable read with no default, a secret pasted into a fixture. None of those depend on who you are or what you sell.

AI code review: what it catches and what it missescatches · in the diffunhandled case: undefined itemsnegative quantity not checked999 and 149 have no nameno test for the 999 thresholdsecret pasted into a fixtureerror swallowed by an empty catchmisses · not in the difffree shipping after discountswhether the feature should existwhat was agreed not to solvewhich field is personal datawhere logs go and who reads thema rule that lives only in a ticketAI reads the diff; the business is not in itexample: shippingCost with 1,100 of items and a 200 coupon

Scroll the figure sideways to see all of it

What is not in the code: business rules, decisions and security context

Now the rule: in this example store, shipping is free from 999 after discounts. We ran the function with 1,100 of items and a 200 coupon: the customer pays 900 and the function returns 0. Free shipping that should not have been. The code is consistent with itself; the bug is against a rule that lives in a ticket, a call, or the head of whoever runs the store. A reviewer who only reads the diff has no way to know, person or model.

The same goes for three kinds of things:

  • Business rules. What you charge, when you refund, what happens to an order that is already paid. The code can be well written and do the opposite of what was agreed.
  • Decisions. Whether the feature should exist, why that library was picked, what you agreed not to solve. Google's guide is explicit that a reviewer can deny a well-designed change if it adds something you do not want in your system (the standard of code review). An AI reviewer does not know what you do not want.
  • Security context. Who calls that endpoint, what is public, which field is personal data, where the logs go and who reads them. The AI sees console.log(order); it does not know that order carries the customer's address and that those logs are read by more people than they should be.

One more problem, about tone: an AI reviewer reports what it checked and what it assumed in the same confident voice. A finding is a claim until someone shows it with an input or a specific line. And it reads code, it does not run it: what only fails at runtime is caught by your tests and your CI.

Kind of problemIn the diff?Who catches it
Unhandled case (null, empty, threshold)YesAI, with the input that breaks it
Missing test, or a test that cannot failYesAI; the person confirms
Misleading name or commentYesAI
Formatting and styleYesThe linter, before anyone
Business rule written in the repoIf you give itAI checks it; the person decides
Business rule not written downNoPerson
Whether the change should existNoPerson
Who sees the data and where it goesNoPerson

Two filters: AI first, then a person

The split that works for us has two filters, in order. Machines first: lint, types and tests in CI. Then the AI review, with the repo's instructions. The author fixes each finding or answers why it does not apply. Then a person reviews, and only that approval unblocks the merge.

The first filter does not take work away from the person; it moves their attention. They no longer hunt for the missing await. They read the findings and the answers, and spend their time on the shipping rule, on whether the change should exist and on where the data goes. They still read the whole diff, because the AI is wrong too and because approving means answering for the change.

two filters before the merge: AI first, then a personchangeshort branchnames the rulethat appliesCIlinttypestestsAI reviewreads the rulesreports, no editsinput per findingpersonrules, decisionsdata, securityreads whole diffmergeonly with aperson'sapprovalthe author fixes each finding or answers why it does not applythe person skips the mechanical pass: reviews what is not in the codean AI comment is not an approvalCODEOWNERS + branch protection: a person on sensitive paths

Scroll the figure sideways to see all of it

An AI comment never counts as an approval. On GitHub you make that explicit with a CODEOWNERS file and a branch protection rule with "Require review from Code Owners" (code owners documentation). The paths where business rules live then cannot merge without a person who knows them:

# .github/CODEOWNERS
# Paths that always need a person who knows the rules.
# Replace @your-username with a user or team that has write access.
/lib/shipping.js         @your-username
/lib/payments/           @your-username
/db/migrations/          @your-username
/middleware.ts           @your-username
/.github/ai-review.md    @your-username
/.github/CODEOWNERS      @your-username
  1. Lines starting with # are comments. GitHub looks for the file in .github/, the root and docs/, in that order, and uses the first one it finds.
  2. Each line is a pattern followed by one or more owners. The leading / anchors the path to the root: /middleware.ts does not cover app/middleware.ts.
  3. /lib/payments/ with a trailing / covers everything inside, at any depth. We checked it with git's own pattern matcher: it covers lib/payments/stripe/x.js and not src/lib/payments/.
  4. /db/migrations/ and /middleware.ts: schema changes and the front door of every request.
  5. The last two lines protect the reviewer's instructions and CODEOWNERS itself: if someone, or an agent, loosens the rules, a person approves that too.

Details from the docs worth knowing: paths are case sensitive, ! and [ ] ranges do not work, the last matching line wins, and an owner without write access is not assigned. A path with no owner can be approved by anyone with write access.

who answers for what in a code reviewtaskCIAIpersonformatting and styletypes and failing testsunhandled case, threshold valuemissing test, or one that cannot failmisleading name or commentbusiness rule written in the repounwritten rule, product decisionpersonal data, logs, who calls itapproving the mergefilled: answers for it · outline: helpsAI never approves; written rules give it something to compare against

Scroll the figure sideways to see all of it

Give the reviewer context

The shipping rule sits in the "misses" column only because nobody gave it to the reviewer. If it is written in the repo, the AI reviewer can check the diff against it. That is why the instructions live in a versioned file, .github/ai-review.md, and not in a message typed differently each time:

# Instructions for the AI review

You review a diff before a person does. Report; do not fix.

## Check
- Bugs you can show with one input: null, empty list, 0, negatives, the threshold value.
- What happens when a call fails or returns nothing.
- Names that say something different from what the code does.
- New branches with no test, and tests that would pass even if the code were wrong.
- Secrets, tokens or personal data in code, logs or fixtures.

## Business rules in this repo
- Free shipping from 999 MXN of subtotal AFTER discounts.
- A paid order is never edited: it is refunded or a new order is created.
- Emails, phone numbers and addresses never go into logs.

## Do not judge
- Formatting and import order: the linter checks those.
- Whether the feature should exist: a person decides that.

## Output
One line per finding: path:line · blocks | question | minor · what fails · input that shows it.
If you are not sure, write "unsure" and what you would need to know.
If you find nothing, write "no findings" and list what you did not check.
  1. The first instruction sets the role: review before a person, and do not edit. A reviewer that fixes mixes two jobs and hands you a diff nobody reviewed.
  2. Check lists the mechanical layer, and each item asks for something verifiable: an input, a branch, a test.
  3. Business rules moves part of the person's column into the AI's. It only works with short rules you can hold code against; anything that needs a conversation does not fit here.
  4. Do not judge cuts noise: the linter already covers formatting, and the product call is not the reviewer's.
  5. Output forces a path, a line and the input that shows the bug. A finding without an input is a suspicion, and it reads like one.
  6. The last two lines turn silence into information: "no findings" plus the list of what was not checked tells the person where to look.

With this file, the reviewer can point out that the function compares the subtotal before the coupon. The person still decides whether that is the rule, because rules change and the file can go stale. For the rest of the context worth giving an agent, read what context to give an AI coding agent; if the change came from a spec, hand that to the reviewer too (how to write a spec for an AI coding agent).

When not to approve

Google's guide says to approve once a change clearly improves code health even if it is not perfect, and never to approve one that makes it worse, short of an emergency. With AI as the first filter, these are the signs to hold the approval:

  • You cannot explain in your own words what the change does and why.
  • A finding marked "blocks" has no fix and no answer.
  • The change touches a protected path and its owner has not reviewed it.
  • A test changed its expected value, was skipped or was deleted with no written reason. If an agent wrote the code, check this first: the agent passed the tests by deleting them.
  • The description does not say which business rule applies, and the change touches money, stock or personal data.
  • The AI answered "no findings" without saying what it did not check.
  • The change mixes a refactor with a behavior change, and you cannot tell which line does what.

It does not matter which agent wrote the change or which model reviewed it: the bar for approval is the same. If you are choosing an agent, compare them on your own code (two agents on the same bug), but do not loosen the review because one gave you better results.

Common mistakes

  • Counting the AI comment as an approval. Without a branch protection rule, someone ends up merging on "the AI found nothing".
  • Giving it only the diff. Without the repo's rules, the AI reviews code with no business behind it, and the shipping rule slips through.
  • Asking it to fix while it reviews. You get back a new change that nobody reviewed.
  • Accepting findings without an input. If it does not say which value breaks the code, check before you change anything.
  • Having the person redo the mechanical pass and skip the rest. The second filter is worth it for the rules, the decisions and the data.
  • Leaving the instructions file without an owner. Whoever edits it can drop the rule that gets in the way.
  • Using AI instead of tests. It reads code; what only fails at runtime is caught by tests and CI.

Frequently asked questions

Does AI code review replace a human reviewer?

No. It covers the mechanical layer well and frees time for what is not in the code. The final approval, and the responsibility for the change, stay with a person who knows the business rules.

What goes in the instructions file?

What to check, phrased as verifiable criteria; short business rules you can hold code against; what not to judge; and an output format with path, line and the input that shows the bug.

What if I work alone and there is no second person?

Be the second filter yourself, but at a different time: leave the change, come back later and review with the person's list, not the AI's. Reading the rules out loud against the diff catches what slipped by while you were writing.

What do I do when the AI and the person disagree?

Ask for the input that shows the finding. If there is one, fix it. If it is a rule or a decision, the person decides; if the rule changed, update the instructions file in the same change.

Checklist

  • Lint, types and tests run before the AI review.
  • The AI reviewer reads a versioned instructions file with short business rules.
  • Every finding comes with a path, a line and the input that shows it.
  • Every "blocks" finding has a fix or an answer before a person is asked to review.
  • The person reviews rules, decisions and data, and reads the whole diff.
  • CODEOWNERS and branch protection require a person on sensitive paths.
  • The instructions file and CODEOWNERS have an owner.
  • You do not approve what you cannot explain.

To keep the diff that reaches review small enough to read, work in short branches and commits: a git workflow for AI coding agents.

Saw this on Instagram? Every Reel guide is at ecomlabs.dev/guides.

Written by

Luis Morales

Luis Morales

Frontend and Shopify developer, EcomLabs

@luismoralesdev on Instagram

Published November 19, 2026 · updated November 20, 2026

← Blog