AI Simulations FAQ
How Anthropos AI Simulations work, how participants are scored, how custom simulations are built, and what makes them different from every other assessment format.
What exactly happens inside an Anthropos AI Simulation?
When a participant enters a simulation, they take on a specific professional role — for example, a Senior Account Manager at a software company. They’re given scenario context: a key account is at risk of churning and they need to retain the client before end of quarter.
They complete a series of tasks: a voice call with the VP of Procurement (an AI actor), a follow-up chat message to their internal team, and a renewal proposal document. Each interaction is observed in real time, and performance is scored against a predefined rubric when the simulation ends. Results are available immediately.
How do the AI actors work?
AI actors are powered by large language models (using OpenAI and Anthropic models) and configured with specific personas, goals, and behavioral tendencies. An actor playing a skeptical CFO might push back on pricing, demand specific data, and interrupt when the candidate goes off-track.
Actors respond dynamically to what the participant says and does — two participants can have significantly different conversations with the same actor based on their approach. Voice actors use ElevenLabs text-to-speech with natural prosody, making calls feel realistic rather than robotic.
What types of tasks can simulations include?
Simulations support four task types:
- Voice calls: Real-time audio conversations with AI actors using natural voice synthesis
- Chat: Text message exchanges — email drafting, Slack-style communication, written negotiation
- Code tasks: Live coding in an integrated multi-file editor, with conversation context available to AI actors
- Document tasks: Creating or working with PDFs, presentations, or spreadsheets
Most simulations combine two or three task types to mirror how work actually happens across different channels and formats.
How long does an AI Simulation take?
Most simulations run 30–45 minutes. Micro-simulations for quick skill checks or certification topics run 10–15 minutes. Duration depends on scenario complexity and the number of tasks included.
How is performance scored in a simulation?
Every simulation has a Verification Plan — a set of defined skills and rubric criteria built around the scenario. Interactions are analyzed against this plan. Each rubric criterion receives a score from 0–100, mapping to competency levels:
- Level 1: Needs supervision (>60)
- Level 2: Works independently on routine tasks (>65)
- Level 3: Delivers consistent results and trains others (>75)
- Level 4: Owns complex projects and mentors team (>85)
- Level 5: Drives innovation and sets company standards (>95)
The result is a per-skill competency level, not a single aggregate pass/fail score.
Who defines what “good” looks like in each simulation?
The evaluation criteria are defined by the simulation designer — your team, the Anthropos team, or both — when the simulation is created. You specify which skills are tested, what behaviors indicate each competency level, and role-specific context. The scoring reflects your standards, not generic benchmarks. Criteria can be reviewed, tested, and adjusted before full deployment.
Is the scoring consistent across different participants?
Yes. Because scoring is based on explicit rubric criteria evaluated programmatically, the same behavior receives the same score regardless of who the participant is or when they took the simulation. This is a fundamental advantage over human-evaluated interviews, where scores vary with interviewer fatigue, candidate sequencing, and demographic similarity bias.
How long does it take to build a custom simulation?
With Anthropos Studio, an experienced user can draft a complete simulation in a few hours. A first draft from a job description takes minutes. Complex simulations typically take 1–2 days for design, pilot testing, and criteria calibration. Anthropos offers guided content creation for expert support.
Can we use our own documents and internal knowledge in simulations?
Yes. Anthropos Studio lets you upload company documents — technical specs, policy documents, product materials, org charts — that AI actors and the scenario context can reference. This makes a simulation about your specific product, architecture, or compliance policies possible without generic placeholders.
What’s in the pre-built simulation library?
The library contains 300+ simulations across categories including:
- Sales & Account Management: Enterprise renewals, procurement negotiations, objection handling
- Engineering: Code reviews, architecture decisions, incident post-mortems, AI pipeline reviews
- Product Management: Roadmap prioritization, stakeholder alignment, launch readiness
- Finance: Budget reviews, audit pressure, financial modeling
- Leadership: Team conflict, performance conversations, organizational change
- Compliance & Risk: Regulatory escalations, data breach response, vendor risk
How does Anthropos prevent cheating?
Anthropos monitors for: AI tool usage (ChatGPT, Claude, GitHub Copilot, etc.), copy-paste from external sources, typing pattern anomalies, and unusual session timing. Detected signals are flagged in the dashboard with severity indicators. Session monitoring logs are available for review if a result is questioned.
Are simulation sessions recorded?
Yes. All session outputs are recorded and available to authorized reviewers: full transcripts of voice calls and chat exchanges, documents and code produced during tasks, a timeline of session events, and the rubric evaluation breakdown. Access is controlled by your organization’s permission settings. Recordings are stored in compliance with GDPR and CCPA.