Open-source framework from ServiceNow for evaluating conversational voice agents end to end.
EVA (Evaluating Voice Agents) scores multi-turn spoken conversations across two dimensions: EVA-A measures whether the agent completed the task correctly, and EVA-X measures whether the interaction was natural for spoken dialogue. A bot-to-bot architecture runs fully automated evaluations with no human listeners. The framework ships with 213 scenarios across 3 domains, results for 12 systems, and a perturbation suite for background noise, accents and degraded connections. EVA is open source from ServiceNow, with its dataset on HuggingFace and a leaderboard.
Testing, Evals & Observability
Open-source voice agent benchmark & leaderboard, noise/accent perturbation testing suite
HQ COUNTRY
United States
DEPLOYMENT
Self-hosted
PRICING MODEL
Open source
COMPANY STAGE
Public
Is this your company? To claim this listing or update its details, contact us at sunil@uniocommunity.com
