Can AI Make Better Business Decisions? An Inside Look at Frontier Models in Action

Imagine a busy garage where every decision counts—whether it’s about accepting a big deal, reading hidden documents, or resisting pressure from shady tactics. Now, picture AI models running this entire operation, each with their own personality and decision style. Which would you trust to keep your business honest and profitable? Welcome to the world of live AI management experiments, where the true test isn’t just how well these models chat, but how reliably they make tough decisions.

The Live Experiment: Testing AI in the Trenches

At the forefront of AI management testing, a real small software company is undergoing a rigorous trial, monitored live at firmulate.com/live. Every weekday, the company faces the same set of crises, customers, and temptations—only the decision-making AI model varies. This isn’t a simulation; it’s a controlled but open experiment where each model’s choices are recorded and auditable.

The Contenders and Scores

The AI models are ranked based on their performance during the trial:

• gpt-5.6-sol: Score 95 – Found the hidden document reference that led to closing a €55,000 deal at full price, demonstrating comprehensive understanding and honesty.
• Kimi K3: Score 93 – The newcomer closed the same deal with impeccable discipline, refusing manipulative tactics and reading the critical file.
• Sonnet 5: Score 88 – Also closed the deal but with some process slips, like missing some key document references.
• Fable 5: Score 77 – Managed to close the deal, but discipline slipped further, with some decisions left unexecuted or delegated to less secure departments.

The baseline score, representing no AI intervention, was 26, emphasizing how much AI can influence management outcomes.

Decision-Making Under Pressure

All four models demonstrated remarkable vigilance: they detected every crisis, refused every manipulation attempt, and maintained integrity. For example, when fake CEO messages escalated using staged threats and a reporter’s trick—posing a simple yes/no question behind the scenes—every AI model refused to cooperate, citing suspicion or impersonation concerns.

The Hidden Weaknesses and the Power of Document Reading

The key to closing the deal at full price wasn’t just about diagnosing the crisis—it was about reading the company’s own files. The decisive weakness of all models was discovering a buried document reference two layers deep in internal files. Those that read these references successfully closed the deal at +€4,583 MRR, turning a potential loss into full profit.

Personality and Discipline in AI

Among the models, OPUS 4.8 stood out as the most thorough, analyzing over 80 learned rules and performing deep assessments. Yet, paradoxically, it left the deal on the table and showed discipline slipping, often directing critical decisions into a locked department rather than escalating properly—highlighting that even the most comprehensive AI can falter in execution consistency.

Implications for Business and Automation

This experiment underscores a fundamental point: the ability of AI to read, understand, and act honestly is crucial for real-world deployment. In automotive or garage settings, AI systems might manage customer queries, maintenance scheduling, or supply chain decisions. But their true value lies in their integrity under pressure—can they stay honest when temptations or manipulative tactics arise?

What Does This Mean for You?

For business leaders and decision-makers, the takeaway is clear: AI models differ significantly in their management personalities and decision quality. It’s not enough for an AI to generate convincing chat or simple responses. The real measure is whether it can finish what it starts, read critical hidden information, and resist unethical shortcuts.

Try Our Interactive Quiz

Curious to see which AI model might make decisions like a disciplined manager or a terse analyst? Visit firmulate.com/quiz.html and test your guesses against the actual decisions made by these models during the live experiment. It’s a fascinating way to understand AI management personalities, with real decisions from a real business—no fiction, just facts.

In real business scenarios, AI’s true potential is its honesty and decisiveness under pressure. This live experiment reveals not just what AI can do, but what it *will* do when it matters most. Trustworthiness and thoroughness matter more than clever chat—test your AI workforce before hiring it.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI