Benchmarks Alter Behavior

What Dungeons and Dragons Tells Us About AI

A text from my son

This week my son — a millennial who literally spent his teens at LARP camps and still dives into Dungeon & Dragon campaigns — texted me a link to a story about AI models playing D & D. His takeaway? AI isn’t so smart after all. The reality is more nuanced and much more interesting.

Here’s the essence of what he sent.

A team of researcher…