Social Media Ideas for Experimental Reviews

I Reviewed the 5 Best Study Tools for Exams, Recall, and Focus

I evaluated several data points across G2 reviews, Grid Reports, satisfaction scores, and product research to identify the five best study tools that consistently stood out for dif...

A Study in Priors

There is a replication crisis in the medical literature. Nearly half of all positive trials, when repeated, do not produce a similar positive result.¹,² While many reasons have bee...

How well does AI peer review work?

Claude and I planted 100 known errors into 10 open-access psychology papers and then ran them through frontier models and two commercial AI review tools. In brief: The best single...

I Measured Every RAG “Best Practice” on 746 Pages of Product Manuals. Only Four Survived.

Two Best Practices made things actively worse. One was a bug in my own instrumentation, and fixing it produced a better result than the…Continue reading on Data Science Collective...

“Spin” in systematic reviews and meta-analyses of plantar fasciitis

Evaluation of “spin” in systematic reviews and meta-analyses of physical therapy for plantar fasciitis: A Systematic Review Prajwal Guruprasad et al Background Physical...

I handed a UX review over to AI. Here’s what happened.

AI is a powerful partner for UX reviews, not a replacement for the expert, and the accuracy comes from the expertise you feed it, not theР’В model.I got a task that isnРІР‚в„ўt com...

LLM Evaluation Frameworks Compared: How to Actually Measure What Your Model Does

In this article, you will learn how to evaluate LLM applications using the three dominant open-source frameworks — RAGAS, DeepEval, and Promptfoo — and why...

What is evaluation research? (methods and examples)

Few things derail a project faster than a team relying completely on guesses and instinct. It doesn’t matter whether you’ve got a team doing market research for a new product or yo...

Adaptive Experimentation with Meta’s Ax: A Practical Coding Guide

In this tutorial, we explore adaptive experimentation using Meta’s Ax with the modern Client API. We work through a complete workflow where we tune a RandomForest model on a synthe...

Poll results on sending revised mss back out for review

Recently, I polled readers on their views as editors (or would-be editors) on sending revised mss back out for review. Here are the results! tl;dr: Interesting range of views out t...

Meta FAIR Introduces AI Research Preference Models (RPMs): Ranking ML Experiments Before Spending GPU Hours

AI research agents can propose far more experiments than they can afford to run. Meta FAIR, Oxford and UCL introduce AI Research Preference Models — frozen LLM judges that rank 15...

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deplo...

“Placebo tests deserve a model, not just a glance.”

Miha Gazvoda shares this post with the above title and the subtitle, “Using Bayesian multilevel models to correct bias and calibrate uncertainty.” He’s using the chickens model fro...

I built a tool to prove my multi-agent harness was worth it. It told me it wasn't.

I spend most of my time on agentic systems, and I had absorbed the same idea everyone else has: a planner improves things, and a panel of drafters with a judge improves them furthe...

Understanding Performance Reviews: A How-To Guide to Meaning

Discover the meaning of performance review in our comprehensive how-to guide. Learn the key components, best practices, and tips for conducting effective evaluations that drive emp...

Why quantitative understanding of effect sizes matters, even if all you care about is the presence of the effect

In reaction to my article with Andy King proposing post-publication review, Dan “Fast and Frugal” Goldstein writes: Your process limits information search, computation, and time so...

Ranking Test Suites by What They Caught [Testμ 2026]

Partha Sarathi Samal of Paramount on replaying 18 months of incidents against 4,200 tests, the proven, duplicate and unproven verdicts, and their limits.

(1) “Do you think the culture of research has genuinely changed since the replication crisis became widely discussed, or...

Luke Ford writes: [Regarding] the replication crisis, researcher degrees of freedom, and the gap between what statistical methods claim to establish and what they can actually supp...

Vizra Evals and Pest's Evals Plugin: When You Want Which

If you are testing AI agents in Laravel, there are now two packages with "evals" in the description, and the obvious question is whether you need both. Short answer: probably not,...

AI Code Generation 2026: What 3,000+ G2 Reviews Reveal

Key Findings According to G2's analysis of 3,000+ verified AI Code Generation reviews, 992% of reviewers rated their tool positively, with an average category rating of 4.6 o...

Persistent Highlights Enable Efficient Review

Persistent Highlights Enable Efficient Reviewby Everlaw

Combining AI Experiments to create ChangeAtlas

Review: Calculation Training by Robert Ris

With his FritzTrainer “Calculation Training”, Robert Ris tackled one of the most important topics in chess back in 2018: how to calculate correctly – and clearly struck a chord. Se...

The Research Habits Behind Every Good Purchase Decision

Anyone who has spent real money on a leather bag knows the feeling of doubt that creeps in before the box even arrives. Product photography flatters almost anything, and marketing...