# Generating Tests with AI — Verification Stays Human

> How good is AI at writing tests? Practical ways to generate and extend tests with LLMs, what AI reliably misses, and the checklist for trusting AI-written tests.

**Published:** 2026-08-06 | **Updated:** 2026-08-06

---


> This is **Part 19** of the "Frontend Testing, Done Right" series. [Browse the full series](/en/series/frontend-testing-done-right/) · [Glossary](/en/posts/frontend-testing-glossary/)

Can you just ask AI (large language models like ChatGPT and Claude) to write your tests? **Yes. Just don't trust them as-is.** AI produces plausible tests in seconds, but tends to quietly skip the boundary, failure, and accessibility cases that matter most. This article is about employing AI properly as a test-writing assistant — from getting good drafts to the checklist that proves the tests actually work.

If you've been following the series, this is where the final chapter begins — the verification skills you've built up turn into your power to handle AI. First time here from a search? The examples are self-contained too.

> Practice code: [frontend-testing-lab](https://github.com/IsaacEryn/frontend-testing-lab/tree/step-19) — this part's `filterByQuery` and `debounce` already landed in earlier parts, so `step-19` is identical to `step-18`. The AI draft and the mutation experiment reproduce from the code blocks below alone.

Three goals for this installment.

- Getting test drafts from AI, fast
- The cases AI misses (boundaries, accessibility)
- The checklist for verifying AI tests

{{< img src="images/contents/ai-generate-verify-en.png" alt="A cycle diagram of AI drafting tests and a human verifying - you give the AI the function code plus requirements, boundary orders and a sample test, it returns a plausible suite in seconds, and the human checkpoint runs a mutation check, looks for implementation pinning and checks boundaries and accessibility, signing only what clears all three and feeding back what was missing" >}}

---

## AI is a great drafter

Hand it the [unit part's `filterByQuery`](/en/posts/frontend-testing-unit-basics/) and say "write tests," and AI returns a plausible suite in seconds. Typically:

```ts
// the AI's draft (typical)
it('filters by name', () => {
  expect(filterByQuery(users, 'Alice')).toHaveLength(1)
})
it('returns all when query is empty', () => {
  expect(filterByQuery(users, '')).toHaveLength(2)
})
```

Not bad. But something's already off — the demo app's mock data has **three** users; the AI assumed two. We never showed it our data, so it made some up. If the number happens to match, lucky you; if it doesn't, a test goes red on perfectly good code. AI is genuinely useful for laying down repetitive combinations fast, but it's an early preview of the theme: **the facts must stay in human hands from the start.**

Draft quality hinges on **what you provide.** Most of us stop at this:

```text
write tests for this function

export function filterByQuery(users, query) { ... }
```

Put yourself on the receiving end and the result makes sense. It doesn't know the requirements, doesn't know how many records the data holds, doesn't know what testing style this project uses. **What it doesn't know, it invents.** That's exactly how three users became two.

Add three things and the draft changes.

```text
Write tests for the function below using Vitest.

[Requirements]
- searches by both name and email
- case-insensitive
- an empty or whitespace-only query returns everything

[Boundaries — required]
empty string / whitespace-only string / zero results

[This project's test style — follow this voice]
it('an empty query returns everything', () => {
  const result = filterByQuery(users, '   ')
  expect(result).toHaveLength(2)
})

[Target code]
export function filterByQuery(users, query) { ... }
```

Each block does a different job. **Requirements** stop the inventing, **boundaries** suppress the habit of only walking the happy path, and the **style sample** returns something you can paste into the team's codebase as-is.

One more thing — **actually show it the data.** Paste the fixture in and it won't have to imagine how many users there are. The prompt gets longer, which feels wasteful, but it's shorter than the time you'd spend fixing one invented number.

---

## What AI tends to miss

The draft above has holes — exactly the cases we caught ourselves in the [unit testing part](/en/posts/frontend-testing-unit-basics/):

- A whitespace-only query (`'   '`) — the `trim()` check
- Searching by **email** (half the requirement!)
- Mixed case (`'ALICE'`)

AI is strong on the happy path and weak on **boundaries, failure, accessibility.** Unless ordered, things like [`role="alert"` checks](/en/posts/frontend-testing-msw/) or [accessible names](/en/posts/frontend-testing-a11y-queries/) almost never appear.

If I had to guess why, the reason is simple. AI learned from the code that's out there, and **that code doesn't tend to handle boundaries either.** Sample code and tutorials mostly show the case that works, and tests carrying accessibility assertions are a minority even among those. Learn the average, produce an average test. So rather than hoping it'll figure things out, **point at what's going to be missing before it starts.**

### So how do you actually find them

"Handle the boundaries" doesn't move your hands. I do three things, in order.

**One, split the requirements into sentences and cross them off one by one.** There were three requirement lines in that prompt — search by name and email, ignore case, empty query returns everything. Lay them out beside the draft's tests, pair them up, and **the email search has no partner** — immediately visible. The AI dropped half a requirement, and you'd never have found it by staring at the code alone. If the requirements aren't written down anywhere, that's the first problem.

**Two, sweep the same boundary list every time.** I keep six fixed: empty value, whitespace-only, zero results, maximum and overflow, duplicates, and case or special characters. Rely on memory and what slips depends on the day; keep a list and it filters mechanically. The `'   '` and `'ALICE'` missing from the draft are items 2 and 6 on exactly this list.

**Three, ask for accessibility separately.** This one barely ever shows up unless you write "include it."

```text
Add accessibility assertions to the tests above.
Find elements by role and accessible name instead of CSS classes or test-ids,
and check that error messages are announced via role="alert".
```

One more — **asking the AI back works surprisingly well.** Say "list the boundary cases these tests missed" and it'll readily name the things it just didn't write. It won't do it up front but knows when asked, which is a little annoying of it. That list isn't gospel either, of course; run it through the two steps above.

---

## The verification checklist

Before signing off on AI-written tests, check three things.

1. **Mutation check**: break the code on purpose (remove `trim()`, etc.). If no test turns red, they're decorations.
2. **Behavior vs implementation**: is it pinning internals (call counts and such)?
3. **Boundaries & accessibility**: empty values, error states, role/name checks — present?

I actually ran number 1 against that AI draft. The fixture is the same two-person one from the unit testing part — the draft's numbers happen to line up, so the defect is the only variable. A three-act experiment.

```bash
# Act 1 — the two AI drafts, healthy code: all pass
  Tests  2 passed (2)

# Act 2 — deliberately remove trim() from filterByQuery: still all pass?!
  Tests  2 passed (2)

# Act 3 — add one boundary test (whitespace-only query), same defect:
  × returns all when query is whitespace only
    AssertionError: expected [] to have a length of 2 but got +0
  Tests  1 failed | 2 passed (3)
```

Act 2 is the point. A real defect went into the code and the AI draft stayed **all green** — proof this suite is a decoration against the trim defect. Then act 3: the moment one boundary test joins, the same defect goes straight to red. The difference between "having tests" and "tests that guard" fits in those three lines of output.

Number 1 should feel familiar — it's the **"break it on purpose"** we've done every single episode. Planting a defect (a mutation) and seeing whether tests catch it has an official name: **mutation testing.** With your own tests you know the intent, so occasional checks suffice — but **AI tests arrive intent-unknown, which makes this verification mandatory.** Coverage only says "tests passed through the code"; mutation says "**tests actually catch defects.**" (As suites grow, tools like Stryker automate this — Vitest included. It plants hundreds of mutations and reports the **surviving mutants** your tests failed to kill, along with the **mutation score**, the share it did catch.)

### Why number 2 is the hard one — behavior vs implementation

Number 1 is mechanical; you just break things. Number 2, "is it pinning the implementation," is the hard one, because the phrase itself is abstract.

`debounce` makes it concrete fast. It's the function we handled with fake timers in the [async part](/en/posts/frontend-testing-async/) — call it repeatedly and only the last one runs.

```ts
// A. checks behavior — "what does it guarantee"
it('runs once, with the last arguments', () => {
  vi.useFakeTimers()
  const fn = vi.fn()
  const d = debounce(fn, 100)
  d('a'); d('b')
  vi.advanceTimersByTime(100)
  expect(fn).toHaveBeenCalledTimes(1)
  expect(fn).toHaveBeenCalledWith('b')
})

// B. pins the implementation — "how was it built"
it('calls clearTimeout twice', () => {
  vi.useFakeTimers()
  const spy = vi.spyOn(globalThis, 'clearTimeout')
  const d = debounce(vi.fn(), 100)
  d('a'); d('b')
  expect(spy).toHaveBeenCalledTimes(2)
})
```

Both pass right now. Green lights alone won't tell them apart. The difference shows up **when you change the code.**

I swapped `debounce`'s internals from cancelling timers to a generation counter. Later calls still win, and the behavior from the outside is identical.

```ts
// same behavior, different construction — no clearTimeout
let generation = 0
return (...args: Parameters<T>) => {
  const mine = ++generation
  setTimeout(() => { if (mine === generation) fn(...args) }, ms)
}
```

Here's the run after the change.

```bash
     × calls clearTimeout twice 1ms
AssertionError: expected "clearTimeout" to be called 2 times, but got 0 times
  Tests  1 failed | 1 passed (2)
```

A survived, B broke. **Nothing changed for the user, and the test went red anyway.** Tests like this block refactoring. Every fix has you editing the test instead of the code, and eventually someone says "we can't change this, there are tests on it."

There's one question that separates them. **"If this assertion breaks, is there a problem a user experiences?"** If not, you've pinned the implementation.

AI falls into this trap especially easily. Tests that peer at internals are probably common in the training data too, and above all **AI doesn't know what the code's contract is**, so it asserts whatever it can see. When you spot a pile of `toHaveBeenCalledTimes`, ask each one — is this a promise, or just how it happens to be built today?

{{< img src="images/contents/mutation-check-en.png" alt="The mutation-check principle - with 20 AI-generated tests all green, you plant a defect like removing trim on purpose. If everything stays green, the tests are decorations that catch nothing; if a red light turns on, they are proven to be real alarms. A note says this is the official name for the series-long break-it-on-purpose habit" >}}

---

> Generation can be delegated; **responsibility cannot.** The person signing off on AI tests is you. It's the same principle as humans reviewing AI alt text in accessibility.

> "AI, write me tests" → 20 in a snap. But if breaking the code leaves everything green? Congratulations — AI has gifted you 20 cousins of `expect(true).toBe(true)`.

---

## One-page summary

- AI is a great **drafter** — repetitive combinations in seconds, strong on happy paths, weak on boundaries, failure, accessibility
- For good drafts, never send code alone — requirements + boundary orders + a sample test + **the actual data**
- Have a method for finding what's missing — **pair the tests against requirement sentences 1:1** · sweep a fixed boundary list · ask for accessibility separately
- Verify what you get in three steps: **mutation check** (break it, expect red) · implementation-pinning · boundaries & accessibility
- One question exposes implementation-pinning — **"if this assertion breaks, is there a problem a user experiences?"** If not, it only blocks refactoring
- Tests that stay green through broken code are decorations — likely cousins of `expect(true).toBe(true)`
- Generation delegates, **responsibility doesn't** — the signature is human

---

## AI drafts, humans confirm

You can now employ AI as an assistant. Next: browser automation meets AI — Playwright × AI.

> **Level up**: you verify AI-written tests instead of transcribing them.

> **Next up**: Playwright × AI — codegen, MCP, self-healing selectors

> Bumped into an unfamiliar term? The [glossary](/en/posts/frontend-testing-glossary/) has them all, one line each.