Six systems passed their tests. None was safe to deploy as shipped.

The frozen specification defined two roles. Submitters create invoices, approvers decide them, and anything over EUR 10,000 needs two distinct approvers. Nobody may approve their own invoice. It did not say how a person receives either role.

Four separate runs invented a public registration endpoint that accepted the role in the request body. A caller could send role=approver twice and acquire two user IDs. The service then treated those IDs as two trusted people.

The approval code was good. Every run blocked self-approval. Every run enforced distinct IDs above the threshold, often with a database uniqueness constraint. The workflow control passed exactly where we had placed it. Identity provisioning sat outside the specification, so the control could be defeated before the workflow began. This happened in two Claude Code runs and two Copilot CLI runs, across different languages, session designs, and dependency choices. Their missing identity boundary did not differ.

Five systems had clear object-level authorization failures. Once a caller had any valid account, changing an invoice ID was enough to reach records belonging to someone else. The exposed data varied by run: invoice metadata, supplier contact details, PDFs, audit histories, and actor identities. Most affected systems also let the caller erase another invoice's contact email.

All six systems correctly logged reads of that email. The spec explicitly required it. Most did not first check whether the reader was entitled to see the address. An ordinary account could read somebody else's personal data, and the audit table would record it. The logging requirement passed. The access policy had never been written.

That distinction matters in regulated software. A log tells an auditor that an event happened. Authorization decides whether it should have happened.

The three Node lockfiles returned zero known vulnerabilities in a current dependency audit. One Copilot run did have directly reachable denial-of-service issues in its pinned multipart parser; the other two Python systems avoided third-party packages but used standard-library reference servers with no limits for slow or oversized bodies.

The existing Semgrep run reported informational CSRF notes and six raw-SQL errors. The SQL findings were false positives: fixed clause fragments with properly bound values. Gitleaks reported no secrets and missed both active fallback strings.

None of this makes those tools useless. They confirmed clean SQL binding, current Node dependency trees, and sound password primitives across the six systems. They did not model who may create a privileged identity, who owns invoice 17, or who may erase the email inside it.

The full review scores each run on a comparative threat scale from 0 to 100, where higher is worse. It is not CVSS, and the small differences in the middle are not statistically meaningful. The useful result is that the best draw was still not deployable as shipped.

The complete ranking, scoring rubric, and per-run detail are in the full security report, not in an agent leaderboard. We do not compare the two products against each other; the finding is about what a coding agent is, not which one to buy.

The next version of the invoice specification should close the obvious gaps. Identity must come from a trusted administrator or identity provider. Every invoice, PDF, personal-data, and audit operation needs an explicit role and ownership policy. Erasure needs separate authority. Sessions need expiry, secrets need to fail closed, and request bodies need limits.

Writing those decisions down would have prevented most of the repeated failures. It would not have caught everything. Claude run 1's sub-cent crash is the useful counterexample. The requirement already said the amount must be positive. The route checked for a positive number. A later conversion rounded the value to zero, and the database rejected it in a place the web framework did not catch.

The decision existed. The hostile boundary test did not. For generated software, a security decision needs a corresponding implementation and evidence from the exact build that will ship. Regeneration produces a new build, so the evidence has to be produced again.

Taiga does not promise identical generated code. Its documented workflows connect project context, initiatives, implementation and review. The current capability reference states the prerequisites and limits; the agreed scope and the evidence for the exact run matter more than a general claim of repeatability.[7]

Architecture, data flow, threat model, DPIA input, and risk register exist before a Taiga build. Their presence alone proves nothing. They have to decide how privileged identities are provisioned, which records each role may reach, who may erase personal data, and where resource limits apply — and those decisions only count if executable negative tests hold them to the exact delivery, not to the process that produced it.

That is the standard anyone should hold us to. A description of how we work is not evidence; the record attached to a specific build is. The safe version of this also separates the builder from the security judgment: the reviewer needs a different job, explicit policy, and permission to fail the delivery, with the pass or failure attached to that exact run.

The first article got the variance result right. Reviews attach to one draw and do not transfer automatically to the next. The scanners missed the fallback secret and raised false SQL alarms. But it underestimated the security problem, because we found the secret we had planted and stopped too early.

The fuller review is still limited. Six small services from two agent products are not a benchmark. The threat scores are comparative judgments. We did not test production infrastructure that does not exist in the repository. Someone else may find more. That is why the code, method, report, and correction stay public, and our own governance claims should be tested the same way.

The first article asked what the agent would build the second time. After this review, I would ask for the security record attached to that second build. Then I would try to break it.