AI writes correct Laravel now. Idiomatic code is the real test

Laravel declared correct code solved and moved the goalposts

Laravel’s Boost benchmark is 17 real Laravel tasks, each checked by real tests. The last time results were published, the best models cleared 16 of 17. Now the frontier models walk the whole suite at or near 100%.

So “can AI write correct Laravel?” has an answer. Next question. Laravel did the sensible thing with a solved problem: said so, and started measuring something harder.

What they measure instead are the two things that matter to anyone shipping AI-assisted code for a living. What did the correct answer cost? And is it idiomatic, or merely correct? The first is a budget question. The second is the whole game.

Correct code per token is the new cost metric

If every model reaches the right answer, the only thing left to compare on correctness is the price of getting there. On Laravel’s numbers, the spread runs one to two orders of magnitude for near-identical output. Same passing code, ten to a hundred times the cost.

Tests can’t see that difference. An agent that reads half the repository, tries three approaches and throws two away ends up just as green as one that went straight to the answer. If you are choosing a model or an agent setup for a team, pass rate no longer separates the options. Cost per correct change does, and it belongs on the same dashboard.

Cost is the easy metric, though. You can read it off an invoice. The second one is where teams actually get hurt.

Idiomatic, not merely correct

Take a controller that passes every test. Inside it there is a raw query instead of Eloquent, validation written by hand instead of a form request, and a model that skips $fillable. It works. It isn’t Laravel.

It works. It isn’t Laravel.

Each of those shortcuts has a cost the tests don’t show:

• The raw query skips the model entirely, so global scopes, casts and model events don’t run. The day someone adds a scope that hides soft-deleted or tenant-foreign rows, this one query keeps returning them.
• Hand-rolled validation lives in one method. A form request can be reused, holds its own authorisation check and is the first place the next developer looks for the rules.
• Skipping $fillable usually means setting $guarded to an empty array or assigning attributes one by one. Both work. The first leaves every column open to mass assignment the moment someone passes $request->all() to create().

Feature tests check behaviour. They say nothing about shape. A suite can stay green while the codebase quietly turns into three different apps sharing a database.

The failure mode is never “it doesn’t work”

I’ve been running AI-assisted conversions across our theme catalog at ANPS Themes for months. The failure mode is never “it doesn’t work.” It’s “it works, it looks like nothing else in the repo, and the next person now has two patterns to learn instead of one.”

Two patterns is worse than it sounds. The next developer has to work out which one is the rule and which one is the accident, and nothing in the code tells them. An agent reading the repository for context has the same problem: it sees both, has no way of knowing which is preferred, and may copy the wrong one into the next file. Drift compounds.

None of this is new. Developers produced inconsistent code long before AI, as I wrote in AI slop is not new. What changed is the volume. A person introduces a stray pattern now and then. An agent can introduce one in every file it touches, all of them passing review if the reviewer only checks that the tests are green.

Laravel has a benchmark. WordPress has vibes.

Here is the uncomfortable part for my side of the fence. WordPress has no equivalent: no eval suite, no idiomatic scoring for a plugin, no rubric an agent can be held to.

The closest thing is the WordPress Coding Standards ruleset for PHP_CodeSniffer. It is useful. It catches formatting and naming problems, unescaped output, unsanitised input and missing nonce checks. What it can’t tell you is whether a plugin is written the way WordPress code should be written: whether it uses the settings API or hand-rolls an options page, registers blocks through block.json or wires everything up in PHP, hooks in at the right point or fights the load order.

WordPress also makes the question harder than Laravel does. Twenty years of APIs live side by side, so there are often several accepted ways to do the same thing. That makes “idiomatic” a decision each team has to make, not something the framework hands you.

If your stack can’t define idiomatic, you can’t measure it. And the agent writing your code is optimising for green tests and nothing else. Where there are no tests, it is optimising for nothing you can see.

How to define idiomatic for your own codebase

You don’t need Laravel’s resources to do a smaller version of what they did. The framework benchmark answers “is this Laravel?” Your job is to answer “is this our Laravel?” or “is this our WordPress?”

1. Write the conventions down. A short file in the repository: where queries live, how validation works, how models guard attributes, which hooks a plugin uses for what. If the rules only exist in senior heads, neither new developers nor agents can follow them. The same file can feed the guideline files coding agents read.
2. Turn every rule you can into a check. Architecture tests, static analysis and a coding standard run in CI and fail the build. An agent can’t argue with a red pipeline.
3. Score the rest in review. Some conventions can’t be automated. For those, the reviewer asks two questions: does this follow the nearest existing example, and does it introduce a new pattern? A new pattern can be the right call, but then it is a decision that gets written down, not an accident.
4. Build a small eval set from your own history. Take merged changes you’re happy with, strip them back to the task, and give that task to the agent. Run the tests, the checks and the review questions against the result. That is the Boost benchmark at the scale of one codebase.

For the Laravel controller above, the raw query is the easiest problem to turn into a rule. A Pest architecture test fails the build if a controller reaches for the DB facade, and a second rule keeps everything in the requests namespace built on FormRequest:

<?php

arch('controllers do not query the database directly')
    ->expect('App\Http\Controllers')
    ->not->toUse('Illuminate\Support\Facades\DB');

arch('form requests extend FormRequest')
    ->expect('App\Http\Requests')
    ->toExtend('Illuminate\Foundation\Http\FormRequest');

Neither rule spots validation written inline in a controller or a model that opens every column. Those stay with review. Each layer catches something different, and none of them catches everything:

The point of stacking them is to leave the reviewer with only the questions a machine can’t answer. That is the same rule behind how we work generally, described in AI-assisted, human-reviewed: the tool drafts, a person who knows the codebase decides whether it belongs there.

The next question is the same for every stack

Laravel asked it in public: now that the code is correct, is it idiomatic, and what did it cost? Every team using agents has to answer the same question about its own repository, because no framework benchmark knows your conventions.

For Laravel teams, the tools to encode those conventions already exist. For WordPress, someone has to write the rubric first, and as people who sell themes and build plugins, we feel that gap more than most. Until it exists, the definition of idiomatic is whatever your team writes down and enforces. An agent can only be as consistent as the rules it is held to.

If you want WordPress code reviewed against conventions like these, and against more than “it loads”, that is how we run WordPress custom development.

Frequently asked questions