The AI Alpha Brief

The AI Alpha Brief

Independent AI research, benchmarks, and practical insights for capital markets professionals.

15.3% of the "fails" were wrong

Oct 18, 2026

•

3 min read

15.3% of the "fails" were wrong

Before you trust an AI score, ask what produced it.

Joseph Yuschak
Joseph Yuschak
A regulator described the work. It didn't say who checks the AI.

Oct 11, 2026

•

2 min read

A regulator described the work. It didn't say who checks the AI.

The Dutch markets regulator named reconciliation and settlement breaks. Its answer to "is the AI right?" was "when properly validated." By whom?

Joseph Yuschak
Joseph Yuschak
Testing comes first, says the Bank of England. We scanned 2,000 of our own test cases.

Oct 4, 2026

•

2 min read

Testing comes first, says the Bank of England. We scanned 2,000 of our own test cases.

The Governor named testing as the starting point for AI governance. Nobody named the tester. Here’s what checking our own tests turned up.

Joseph Yuschak
Joseph Yuschak
Same model, same question: 99% or 5%

Sep 27, 2026

•

2 min read

Same model, same question: 99% or 5%

Whether a reasoning model actually reasons can depend on which server answers, and nothing in the response tells you.

Joseph Yuschak
Joseph Yuschak
A 94.5% score with one blind spot inside it

Sep 20, 2026

•

2 min read

A 94.5% score with one blind spot inside it

One category confusion accounts for a model's entire deficit, and on a third of those cases it answers differently each time you ask.

Joseph Yuschak
Joseph Yuschak
The accuracy number is lying to you (by up to 82%)

Sep 13, 2026

•

2 min read

The accuracy number is lying to you (by up to 82%)

Four 2026 papers independently found that a single benchmark score hides most of what a model can, and can't, actually do.

Joseph Yuschak
Joseph Yuschak
Load more