Log inBook a Demo
AI Simulations
Skill Paths
AI Labs

AI Labs Library

Fix, build and secure real code with Claude Code at your side, in a private cloud workspace. Pick a lab to see what it asks of you and what it measures.

Practice a skill

Hands-on challenges — filter to what you want to work on.
68 labs
BuildSenior
35 min

Build a pipeline that hands state between agents

Build the sequential handoff chain behind every multi-step agent: run_pipeline(agents, state) runs an ordered list of agents, threading one evolving shared state through each so agent N sees what agents 0..N-1 wrote. It short-circuits when an agent sets state['done'], and isolates a failing agent (records the error, keeps going) instead of crashing the chain. The shipped version passes the same original dict to every agent (writes never propagate, agents clobber each other), ignores the done flag, and dies on the first raise. Grading injects deterministic mock agents that tag and record the state they saw — no real LLM, no gateway — so the checks are about correct control flow, not model quality.
python
BuildMid
35 min

Cap an agent's steps and retry flaky tools

Make a tool-using agent loop ROBUST: enforce a hard step budget (stop and return a structured partial result instead of looping forever), retry a tool that raises up to a limit, and when a tool keeps failing, feed the error back to the model as an observation instead of crashing. The shipped agent ignores the budget, crashes on the first tool error, and never retries. Grading injects a deterministic scripted model and counting tools — no real LLM, no network.
python
BuildSenior
50 min

Build a tool-using agent with Claude Code

Build a small autonomous agent that uses Claude Code with tools — file I/O, web fetch, shell — to complete a multi-step task end-to-end.
python
BuildSenior
40 min

Fix a transfer that loses or invents money

A double-entry money-transfer module violates its core invariants: transfer() uses float (rounding drift), permits overdrafts and negative balances, loses money when an account is unknown, and isn't atomic on rejection. Rebuild it so every transfer conserves money (debit == credit), rejects overdrafts without mutating state, and uses exact Decimal arithmetic.
python
BuildJunior
30 min

Write base64 encode and decode from scratch

Write RFC 4648 base64 encode and decode from scratch, without Python's base64 or binascii modules. The current version gives non-standard output for some inputs, and some bytes change on the way back.
python
DebugJunior
20 min

Fix a search that returns the wrong index

find_first() and find_last() should return the first and last index of a value in a sorted list with duplicates, or -1 if it's missing. Callers get wrong indices, calls that hang, and an index where -1 was expected. Fix both and keep them binary searches.
python
BuildMid
35 min

Patch a filter that forgets items you added

A BloomFilter(size_bits, num_hashes) is producing false negatives — items you added report as absent because add() and contains() compute different bit positions. Fix the hashing so there are zero false negatives and the false-positive rate stays low.
python
BuildSenior
45 min

Build a token-bucket limiter that returns 429

Build a token-bucket rate limiter to a written spec. It allows a burst up to capacity, answers 429 after that, refills over time, and holds up when many requests arrive at once.
python
BuildSenior
45 min

Cut off a failing dependency and probe recovery

Implement a circuit breaker that wraps a flaky dependency: trip OPEN after N consecutive failures, fast-fail without calling the dependency during the cooldown, then HALF_OPEN a single trial call that closes the circuit on success or re-opens it on failure. Time is driven by an injectable clock so behavior is exactly testable.
python
BuildSenior
45 min

Fix a pool that leaks database connections

Implement a bounded connection pool that creates connections lazily, reuses released ones (factory called at most size times), raises PoolExhausted when all connections are checked out, and supports a context manager that auto-releases. The shipped version is buggy: it opens a fresh connection on every acquire, never reuses, never caps, and loses connections on release.
python
BuildSenior
45 min

Stop a hash ring from reshuffling every key

Make HashRing in ring.py a real consistent hash ring. Right now a node joining or leaving moves almost every key, and two processes don't always agree on which node owns a key.
python
SecureJunior
20 min

Stop a CSV export from injecting spreadsheet formulas

A reporting feature exports user-provided data to CSV by writing each cell verbatim, so a cell beginning with = + - @ (or TAB/CR) is executed as a formula when the file is opened in Excel or Google Sheets — a CSV/formula injection. Fix export_csv to neutralize dangerous leading characters (prefix with a single quote) without corrupting benign data, keeping the output valid CSV.
python
DebugJunior
20 min

Reconcile a bank ledger against a payments export

Finance has two CSVs that should match but don't -- a bank ledger and a payments export, with different date formats, messy money values, duplicate transaction IDs, and a few genuinely missing entries. Write a reconciler that normalizes both sides and emits an exact discrepancy report.
python
BuildMid
30 min

Fix a CSV importer that mistypes columns

A CSV import pipeline guesses each column's type from only the FIRST data row, so it mislabels mixed int/float columns as integer, ignores whether a column has empty cells, treats leading-zero codes like "01" as numbers, and crashes on an all-empty column. Rewrite infer_schema to decide each column's type and nullability from ALL rows, following an exact type-precedence contract.
python
BuildMid
30 min

Fix debounce and throttle that mistime event bursts

Implement debounce and throttle. Debounce runs once after a burst of calls goes quiet. Throttle runs straight away, then at most once per interval. Time comes from an injected clock, so there are no real timers.
python
BuildSenior
90 min

Software Engineer: Delete a user’s data everywhere, as GDPR requires

With your coding agent, ship GDPR data export and account deletion in a gym-booking product whose member data is spread across many stores, under rules set by product, finance, the data protection officer and the SRE. Scored on test members searched for in every table, file, log and restored backup, on your tests, on the proof you leave on the live data, and on how you worked with the agent.
pythonsqlite
BuildSenior
45 min

Order a graph so dependencies come before dependents

Implement resolve_order(graph): a topological sort that orders nodes so every dependency comes before its dependent (including transitive deps and disconnected nodes), and raises CycleError on a cyclic graph. The shipped version loops forever / drops nodes — fix it.
python
BuildSenior
45 min

Fix a lock that lets two workers in

Implement a lease-based distributed lock: acquire grants a key if it's free OR its lease has expired, returning a unique fencing token and recording owner + expiry; release only succeeds for the matching token (a stale owner can't free someone else's lock); is_held respects TTL expiry. Time is read through an injected clock so behavior is exactly testable.
python
SecureSenior
45 min

Software Engineer: Ship a security-critical access change

Use your coding agent to ship an access-control change in an internal document service, where a chat thread partly overruled the original requirements. Scored on hidden cases, your tests, your written decisions and how you worked with the agent.
python
DebugJunior
30 min

Software Engineer: Find out why seats were sold twice

A theatre ticketing service sold 23 seats twice on a sold-out night. With your coding agent, reproduce it from the event log, fix the cause without leaving a paying customer empty-handed, settle the case two colleagues disagree on, and prove the fix.
python
BuildSenior
110 min

Full-Stack Engineer: Build a live drone fleet map with expiring commands and replay

With your coding agent, build the operations map for about 100 crop-survey drones: live positions, commands that wait for an offline drone and expire on time, and a replay of any past moment.
reactvite
BuildSenior
35 min

Score a model's answers into a pass rate

Fix run_eval(dataset, model, scorer), a small eval harness that runs a model over a labelled dataset and reports a pass rate. Its numbers don't match the scorer, and one model error crashes the whole run.
python
BuildSenior
45 min

Evaluate arithmetic with correct precedence and parentheses

Make evaluate(expr) handle basic arithmetic. Today it returns 20 for 2 + 3 * 4, crashes on parentheses and returns a number for input like 2 +. Write the parser yourself: eval and similar shortcuts are not allowed.
python
BuildMid
30 min

Make a percentage rollout stable per user

Implement evaluate(flag, user): honor a master enabled switch, evaluate ordered targeting rules first-match-wins, and fall back to a STABLE percentage rollout (a hash bucket keyed on flag key + user id, so the same user always gets the same answer). The shipped version uses random(), gets rule precedence backwards, is off-by-one on the boundary, and ignores the master switch — fix it.
python
RefactorSenior
30 min

Make a prototype feedback board production-ready

A polished but prototype customer-feedback board (Kanban, drag-and-drop, persisted in localStorage) ships in the sandbox. Productionize it with Claude — add a test suite, migrate the UI to React + Vite, and make it mobile-friendly — previewing live through the Ports tunnel. Bonus: a 4-D AI-collaboration assessment of how you drove the migration.
javascriptreact
DebugMid
25 min

Fix a test that fails randomly in CI

A leaderboard test suite fails at random in CI and sometimes goes green when the job is re-run. Find what makes it non-deterministic and fix the code, without skipping a test, loosening an assertion or adding a retry.
python
BuildJunior
30 min

Fix signup validators that accept the wrong inputs

A signup form's backend validators pass the happy path but quietly accept (and reject) the wrong inputs in production: surrounding whitespace, None, unicode, boundary lengths, plus-addressed emails, international phones and leap-year dates. Harden every validator so it is edge-case-correct.
python
BuildSenior
45 min

Catch duplicate customer records that aren't exact matches

A customer-records deduper ships as exact-match only: it misses 'Jon Smith' vs 'jon smith', reverses the similarity threshold, and double-counts a record into two groups. Implement dedup(records, threshold): normalize (case, whitespace, punctuation, accents), compare with a documented similarity (normalized edit-distance ratio), single-linkage cluster the near-duplicates, and return groups where every record lands in exactly one group.
python
BuildMid
40 min

Fix a glob matcher that flags wrong files

Fix match(pattern, name), the glob helper a build tool's CI uses to pick files. It flags files it should ignore and misses ones it should flag. Write the matching yourself, without fnmatch or glob.
python
OptimizeMid
65 min

Cloud Architect: Cut a doubled cloud bill without losing data

Use your coding agent to cut a doubled cloud bill on a simulated cloud driven from Terraform-style code. You apply staging yourself after reading each plan, with constraints from the DBA, compliance and the CFO. Scored on what the cloud recorded, your policy rule and its tests, whether your numbers are true, and how you worked with the agent.
python
BuildMid
30 min

Honor an Idempotency-Key so retries don't double-charge

A payment/order handler re-runs the charge on every retry — a duplicate request with the same Idempotency-Key charges the customer twice. Make handle() idempotent: execute the side effect once per key and replay the stored response on retries, while different keys still execute independently.
python
BuildMid
30 min

Stop a redelivered message from being handled twice

Your queue delivers at least once, and redelivered messages are charging customers twice. Make consume() handle each message id once, while a delivery that failed can still be retried.
python
BuildMid
30 min

Fix a JSON Pointer that mangles escaped tokens

Implement RFC 6901 JSON Pointer resolution. The shipped resolve() mishandles the reference-token escapes (~1 -> / and ~0 -> ~, and the order matters), treats array indices as string keys so it never indexes into lists, and gets the empty pointer wrong. A missing key or out-of-range index must raise a defined ResolveError, not leak KeyError/IndexError. Make it match the RFC exactly.
python
SecureMid
35 min

Fix a login that never verifies JWT signatures

A login service trusts JWTs without actually verifying their signature — the classic auth bypass. Fix verify_jwt so it checks the HS256 signature in constant time, rejects alg:"none", and enforces expiry. Then attackers can no longer forge tokens.
python
BuildMid
35 min

Fix a cache that evicts recently used keys

An LRUCache(capacity) keeps throwing out the keys services are reading and holding on to cold ones. Fix it so eviction is true least-recently-used.
python
DebugJunior
25 min

Fix invoice totals that drift by a cent

A billing module computes invoice line totals, percentage tax, and a discount using float arithmetic -- so it's off by a cent on tiny prices, half-cent rounding, and long invoices. Fix it with decimal.Decimal (or integer cents) so every amount is penny-correct and round-half-up.
python
DebugSenior
35 min

Reproduce and fix a wallet that overspends under load

An Express wallet API works fine one transfer at a time, but under real traffic too many transfers succeed and money appears from nowhere. Reproduce it under load and fix it.
nodejavascript
BuildJunior
30 min

Software Engineer: Add a discount without changing any other price

Use your coding agent to add a residents' discount to a 2016 ferry fare engine and its booking page without changing any other price by a cent, with requirements added by two colleagues. Scored on unseen bookings, on your tests against broken engines, on a sign-off checked against your engine, and on how you worked with the agent.
javascriptnode
OptimizeMid
30 min

Kill the per-post queries in a /feed endpoint

A /feed endpoint loops over 500 posts and fires a separate query per post for the author and the comment count — the classic N+1. Batch it down to a constant number of queries while keeping the response byte-identical.
python
BuildMid
65 min

Software Engineer: Build the on-call handover tool

Work with your coding agent to build a CLI that turns a night of production logs into the on-call handover your team has asked for. Scored on unseen nights, your tests, what you leave in the repo for the next session, and how you worked with the agent.
python
BuildMid
30 min

Fix pagination that skips records with tied timestamps

A /records list endpoint pages through an ordered dataset, but it uses a fragile OFFSET-style cursor that skips and duplicates rows at page boundaries whenever two records share a timestamp. Rebuild page() as proper keyset (cursor) pagination so a full walk yields every record exactly once, in order.
python
BuildSenior
35 min

Fan out to agents and merge what succeeded

Fix fan_out(items, worker, reduce_fn), which runs a sub-agent on each item and reduces the results in input order. Some sub-agents will raise, and one failure must not take down the batch.
python
SecureMid
35 min

Replace unsalted password hashes with salted PBKDF2

A user store keeps passwords as bare SHA-1 hashes and checks them with a plain equality test. Move it to salted PBKDF2 with a constant-time check, without changing the two functions the app calls.
python
SecureMid
25 min

Keep user paths from escaping the base directory

A file-serving helper joins a user-supplied path onto a base directory by naive string concatenation, so ../../etc/passwd, a leading /etc/passwd absolute override, and nested a/../../b all escape the sandbox. Its only 'defense' is a literal '../' substring check, and a sibling directory like /base-evil is mistaken for being inside /base. Rewrite safe_join to resolve the path purely (normpath) and reject anything that lands outside the base, raising PathTraversalError.
python
BuildSenior
90 min

Cloud Architect: Make the standby region take over, safely

Use your coding agent to make a payments app's standby region take over a regional outage safely, within the constraints the CTO, the database lead, finance and compliance have set. Scored on failure drills you have not seen, on your own drills against broken controllers and infrastructure, on your runbook run step by step, and on how you worked with the agent.
python
DebugMid
30 min

Find and fix an orders service throwing 500s

Checkout on the orders service started failing with 500s right after a config change. Work from the logs and metrics to the cause, patch the service and get the error rate back to zero.
python
SecureSenior
40 min

Stop retrieved text from hijacking an LLM prompt

Retrieved documents and tool output go straight into an LLM prompt, and some of them carry injection attempts. Write sanitize_context(text) so the injected instructions lose their effect and normal lines come through unchanged.
python
BuildMid
35 min

Build a prompt renderer that resists template injection

Build a SAFE prompt templating engine: render(template, vars) substitutes {{name}} placeholders and honors {{#if key}}...{{/if}} and {{#each list}}...{{/each}} — while treating variable values as inert DATA, never template. A value containing {{...}} or an injection like 'SYSTEM: ignore previous instructions' must appear literally and never be re-interpreted. The shipped version re-renders substituted values (template injection), mishandles conditionals, and crashes on missing keys. Graded deterministically against a held-out fixture set — no model call.
python
BuildMid
35 min

Parse and rebuild URL query strings by hand

Build the URL query-string codec every web framework needs: parse(qs) turns "q=hello%20world&tag=a&tag=b" into a dict and stringify(obj) turns it back — by hand, WITHOUT urllib.parse.parse_qs/urlencode. Handle percent + "+"/%20 decoding, repeated keys collapsing to a list, and empty values; make stringify a stable, encoded inverse. The shipped version doesn't decode, drops repeated keys, crashes on empty values, and doesn't encode — make it correct.
python
BuildJunior
30 min

DevOps Engineer: Stop planned maintenance from paging on-call

A logistics company's alerting service paged the on-call rotation 49 times on the night of an announced database maintenance. Use your coding agent to add maintenance silences. The SRE lead has set a rule and left you one decision, and QA a sign-off replay for you to run. Scored on replays of nights you have not seen, on your tests, on the decision you record, on the replays you run yourself and on how you work with the agent.
pythonsqlite
BuildMid
30 min

Chunk a document into overlapping retrieval passages

Implement the ingestion step of every RAG pipeline: a chunk(text, max_chars, overlap) that splits a document into overlapping, boundary-aware chunks. The shipped version is broken — it cuts mid-word at a fixed offset, ignores overlap so a fact on a seam is lost, and drops the tail (off-by-one). No model, no embeddings API: the text is plain data, and the grader locates each chunk back in the source to verify size, coverage, overlap and boundary invariants itself.
python
BuildSenior
40 min

Build a ReAct loop that reasons then acts

Build the core of an agent: the ReAct (Reason+Act) loop. run_react(task, model, tools) prompts a model, parses its 'Action: name[arg]' line, calls the matching tool, feeds the result back as an 'Observation:', and loops until the model emits a 'Final Answer:'. The shipped version has brittle parsing, never feeds observations back, and botches final-answer extraction so it never terminates. Grading injects a deterministic scripted model (no real LLM, no gateway) and checks the control flow + parsing, not model quality.
python
DebugMid
20 min

Fix a React counter that increments unpredictably

A React counter doesn't count properly. Auto-increment stops climbing after the first tick and +1 doesn't reliably add one. Find out why and fix it in place.
reactjavascript
SecureJunior
35 min

DevOps Engineer: Make the release pipeline a real gate

With your coding agent, turn a start-up's release pipeline into a real gate: find every way a broken or wrong build, or the deploy token, can still get out, close each one, prove it with pipeline tests, and decide how a flaky test is treated. Runs on a local CI runner. Scored on commits and events you have not seen, on your pipeline tests, on the CI's records as you leave them, and on how you worked with the agent.
pythonbash
BuildMid
30 min

Add backoff to a client that hammers upstream

An LLM/API client retries on every failure with no backoff and a wrong attempt count — it hammers a flaky upstream and retries errors that will never succeed. Fix retry() to do exponential backoff, give up after max_attempts, and only retry RetryableError (let everything else propagate immediately).
python
BuildJunior
25 min

Fix a converter that writes 4 as IIII

A Roman-numeral converter ships with broken subtractive notation: it writes 4 as IIII, 9 as VIIII, fumbles IX/XL/XC/CM, and round-trips fail. Fix int_to_roman() and roman_to_int() so both directions are correct and inverse across 1..3999, and so invalid input is rejected.
python
SecureMid
30 min

Redact live credentials from exported support logs

A support-bundle exporter pastes raw logs into tickets — and those logs carry live credentials: AWS keys, bearer/JWT tokens, password=/api_key= values, long hex/base64 secrets, and card numbers. The shipped redact() misses most formats, mangles ordinary prose, and — worst — blanks the key while leaking the value after 'password='. Rewrite redact(text) so every secret is fully gone (not even a substring survives) while all non-secret text (words, code, emails) is preserved. Graded deterministically against a held-out golden log — no model call.
python
BuildMid
35 min

Fix a rate limiter that leaks across windows

Fix a rate limiter that leaks across window boundaries. The shipped version is a fixed-window counter — it admits up to 2×limit around the seam and never evicts old requests. Implement a true rolling window (evict entries older than now-window, count the rest) so the limit holds in EVERY window. Time is driven by an injectable clock so behavior is exactly testable.
python
OptimizeMid
30 min

Speed up a slow /report endpoint

The /report endpoint returns the right per-session page-view totals, but it's the hot path in every profile and gets much slower as the log grows. Make its work linear in the number of events, with the same output.
python
SecureMid
25 min

Fix a login where user input becomes SQL

A login + user-search module builds SQL by concatenating user input straight into the query text, so 'alice' -- logs in with no password and ' OR '1'='1 dumps every user. Rewrite it to use parameterized queries against a real sqlite database without changing the legitimate behaviour.
pythonsqlite
SecureSenior
35 min

Stop a URL fetcher from reaching internal addresses

An outbound 'fetch this URL for me' feature has a broken SSRF guard: it only string-blocks 'localhost' and '127.0.0.1', so the cloud metadata IP (169.254.169.254), private ranges, non-http schemes, and integer/hex/octal-encoded IPs all sail through. Rewrite is_safe_url to classify the host structurally with the ipaddress module — no DNS, no sockets — and block every SSRF class.
python
BuildMid
30 min

Extract valid JSON from messy LLM replies

Write extract_json(text) so it recovers the intended JSON object from messy LLM replies: prose around it, code fences, trailing commas, single quotes. A reply with no object should raise ParseError.
python
BuildMid
30 min

Validate and coerce an LLM's tool call arguments

Write validate(args, schema), the check between an LLM's tool call and your code. Coerce the strings models send, like "5" and "true", to the declared types, and raise ValidationError naming the field when a value can't fit.
python
BuildMid
35 min

Split a dataset reproducibly with balanced classes

Implement split(X, y, test_size, seed, stratify=True): a stratified, seeded train/test split that is reproducible, leaks no rows (train ∩ test = ∅, train ∪ test = all), sizes the test set to ~test_size, and preserves each class's proportion in both halves. The shipped version is unseeded, leaks/drops rows, ignores stratification, and hardcodes the size — fix it.
python
BuildMid
35 min

Fix an undo history that replays stale commands

Implement an UndoRedo(max_history) command-history manager — the engine behind editor undo/redo. do(command) applies a command and pushes it on the undo stack; undo() reverts the last command and moves it to the redo stack; redo() re-applies it. THE KEY RULE: a new do() after an undo() must CLEAR the redo stack (you can't redo a diverged branch). History is bounded by max_history (oldest dropped), and undo/redo at the boundaries are no-ops, not crashes. The shipped version is buggy: it never clears the redo stack on a new command (redo resurrects a stale branch and corrupts state), crashes when you undo past the start, keeps unbounded history, and redoes with no prior undo.
python
BuildMid
35 min

Build a URL matcher with params and wildcards

Build the route matcher at the heart of a web framework: match(method, path, routes) resolving static segments, named params (/users/:id) and a rest-capturing wildcard (/files/*), with correct precedence (static > param > wildcard), method matching, and trailing-slash handling. The shipped version ignores the method, resolves by table order, and over-captures — make it correct.
python
BuildMid
30 min

Rank documents by cosine similarity for RAG retrieval

Implement the retrieval core of every RAG system: a search(query_vec, corpus, k) that returns the top-k corpus items by cosine similarity, highest first, ties broken by id. The shipped version is broken — it ranks on raw dot product (no normalization), so longer vectors win regardless of direction. No model, no embeddings API: vectors are plain data, and grading computes the cosine oracle itself.
python
BuildMid
30 min

Stop a virtual list clipping its last row

A fixed-row virtualized (infinite) list renders only the rows in view, but the shipped windowing math is wrong: it clips the last visible row (off-by-one on the exclusive end), ignores the overscan buffer, computes the wrong spacer offset, and never clamps — so it hands the consumer a negative start or an end past the array and blows up with an IndexError. Rebuild visible_window() so the rendered range is exactly right at the top, at the bottom, across a partial row at the viewport edge, and with overscan clamped at both ends.
python
AI Simulations →Skill Paths →