Sierra’s tau-bench: State-of-the-art agents pass under 60% of real-world retail and telecom tasks

Bilanz der Quelle: Sierra’s tau-bench is a benchmark for the typical agent-user interaction in customer service tasks. Even the best state-of-the-art agents achieve only around 50% success rate, while top models reach 60% – for tasks that are routine for humans. The benchmark shows that benchmarks without multi-step interaction and real firm guidelines significantly overestimate the agent’s performance.
🔗 Im Original weiterlesen → · 🗃 Wayback Machine (Archiv)