Interviews Are Testing the Wrong Thing
As AI becomes central to software engineering, the gap between how engineers work and how they’re interviewed has grown significantly. Most interview formats still rely on LeetCode-style problems, isolated coding exercises, and limited or no access to AI tools
But in reality, engineers today rely heavily on AI-assisted coding, work within large, complex codebases, use terminal-based workflows and tooling, and solve problems collaboratively with systems, not in isolation.
Shrivu and the team realized that existing interview formats weren’t just outdated, but were actively measuring the wrong skills.
Poor Candidate and Interviewer Experience
Even before redesigning the interview itself, there were clear operational problems.
Candidates were asked to set up tools like Cursor locally, often on the spot. Shared or ad hoc accounts created confusion and friction. Tooling didn’t reflect how engineers actually preferred to work. Interviewers struggled to extract meaningful signal from artificial exercises.
The result was frustration on both sides. Candidates were having a poor experience and interviewers had low confidence in evaluation quality.
The Interview Box
The Interview Box reimagines the entire interview environment. Built on Modal (the same infrastructure powering Nora and internal AI workflows), it creates a fully sandboxed, browser-based coding environment.
It includes:
- a preloaded question repo
- Claude Code integration
- a terminal-based UI (similar to VS Code workflows)
- no local setup require
Each interview generates a unique environment with its own URL, allowing candidates to jump in instantly and start working.
From Problems to Real Work
One of the biggest shifts is the nature of the problem itself. Instead of small, isolated coding questions, Interview Boxes supports large, realistic codebases (10,000+ lines), real-world engineering tasks, and workflows that mirror how engineers operate day-to-day.
Candidates aren’t solving puzzles. They’re effectively working at Abnormal for an hour. And importantly, they can use AI tools naturally within that workflow.
Because the environment is fully instrumented, Interview Boxes captures far richer signals:
- how candidates use Claude Code
- how they navigate and understand large codebases
- how they structure problems and iterate
- what code they write and how they arrive at it
This shifts evaluation from, “Did they get the right answer?” to, “How do they actually work as an engineer?"
The Candidate Experience Shift
One of the most immediate impacts has been on candidate sentiment. Instead of frustration with artificial constraints, candidates are responding positively. They recognize the workflow as realistic, feel they can demonstrate their actual skills, and are relieved not to be evaluated on LeetCode-style problems.
As Shrivu noted, many candidates explicitly called out how refreshing the experience was compared to industry norms.
Interview Boxes represents a broader shift in hiring philosophy. In an AI-native world, tools are part of the skillset, workflows matter as much as outputs, and realism beats abstraction. By aligning interviews with real engineering work, signal quality improves, candidate experience improves, and hiring decisions become more grounded.
What’s Next
The current version is already producing strong results, but the team plans to:
- evolve and refine interview questions over time
- expand coverage across different engineering domains
- improve analytics on candidate behavior and workflows
- continue aligning interviews with how engineering at Abnormal actually operates
Ultimately, Interview Boxes isn’t just a new tool. It’s a model for evaluating engineers in an AI-first world.
