A few weeks ago we connected with Claus Moberg, Roblox’s VP of Engineering, to get his thoughts on the state of hiring software engineers. One theme that came up is the impact of false positives. He noted that live first-round technical interviews showed.

Starting every hiring process with a live technical interview might seem like a luxury in a world where software engineering time is severely constrained and the financial outcomes of product development can have a seismic business impact. However, due to poor candidate experience and poor signal compared to live interviews, code tests often end up taking more engineering time to vet candidates and decrease hiring yield from an already limited talent pipeline.

One reason for the poor signal produced by code tests is the requirement that candidates’ solutions be both complete and correct in order to “pass.” These are critical drivers of signal but, requiring perfection filters out too many candidates, and as Moberg pointed out, often filters the wrong ones in. On one hand, this is unfair to candidates and on the other, costly to recruiting efforts.