Want To Recruit Better Developers? Give Them Broken Code

Want To Recruit Better Developers? Give Them Broken Code

By:  August 13, 2013


Hiring the right programmers is key for development, but it can be a nightmare to find time to administer coding tests and interview candidates. So rather than relying on a scripted phone round that potential hires can easily game, the mobile consulting firm Mutual Mobile is implementing more pragmatic screening in which candidates have to fix a broken app and run it in order to get to the next round of interviews. That’s it.

Hiring the right programmers and designers is crucial for any software business. Without a cohesive team, projects may never come together. But hiring a qualified developer can be such a long process that it can sometimes literally be endless: Just when one position is finally filled another may be opening up.

Big companies fix this problem by staffing up giant HR departments that take care of initial candidate screening and organize the company’s developers to administer coding tests and other specialized evaluations. But startups and smaller companies don’t have the same resources, even if they have a few HR people. With 325 total employees and about 60 iOS engineers, Mutual Mobile is large enough to have an HR “talent team,” but when they first opened their HR department, they found that hiring iOS devs was draining all of its resources.

“We started originally with the traditional phone screening,” says Ron Lisle, Mutual Mobile’s director of iOS. “And we found that folks were getting pretty good at fooling us on the standard list of questions. We’ve had some folks that have applied to engineering jobs and I’ve gotta believe they’ve never programmed at all.”

The iOS team found that the only way to get a clear sense of someone’s skills was to fly them to Mutual Mobile’s Austin offices for in-person interviews and code tests. But relying on on-site meetings to sort through everyone who got past the phone screen was costly and inefficient, especially because the candidates coming in weren’t always prepared.

“About a year ago we were still doing the phone screen, but we got some folks where it was just really hard to tell,” Lisle says. “It was like, ‘okay this person might be really good, but I’m just uncomfortable about this or that. I can’t tell if they really know coding.’ And so we thought, well, instead of just writing the person off, if we had some sort of test we could send to them then we could actually see what they would do with coding up something. We could verify that they can actually compile and build code.”

Asking developers to go through a pre-interview code test isn’t new: Startups like Codility even offer automated testing as a product. But Mutual Mobile took their tests a step further. To get extra insight into whether a developer was right for the company (Lisle claims the company wants developers to “literally be crazy about mobile”), the company asked a few of its engineers working on the programming test to start adding run-time issues and other bugs to the code they would send to potential new hires. To decrease the impact on the company’s business, Mutual Mobile turned to what it calls “bench resources,” or employees that have finished one client’s project and are waiting to begin something new, to write the tests.

“We thought, okay, so let’s send them some code that they’ll have to build, and let’s throw a few bugs into it while we’re at it and we’ll kind of get a sense for are they able to find and fix bugs,” Lisle says.

To see what the experience is like, Co.Labs asked veteran iOS and Android developer Chris White (who works with Co.Labs editor Chris Dannen) to download the test and give it a shot.

“It’s interesting to engineer problems into the test, things that are broken” he says. “You don’t want them to be too onerous, but you don’t want them to be too obvious either. So I thought they did a reasonable job. If you’ve been using the iOS development tool exchange and XCode for a while then you should be able to fix these in a reasonable amount of time and already have provisioning profiles to run it on a device.”

Mutual Mobile usually hires senior engineers, but the iOS team intentionally built three levels of bugs into the code so they can use it to evaluate a range of experience levels. Lisle emphasizes that the goal is not to hire people based on the test, but simply to use it as a productive screening tool so that candidates who come to Austin for interviews and problem-solving tests already pass the company’s basic requirements.

“We’re almost always looking for really senior people and we hope this test will be such that somebody who gets everything would be at that level,” Lisle says. “However, when we’re hiring the middle experience level folks, they would get most of the questions but not all of them.”

In thinking about the test a few weeks after taking it, White notes that coding tests which throw a candidate into the deep end are becoming more common. For example, White recalls that recently while applying for a job he was asked to do a pair programming project using a company’s own code during an on-site interview. But sending a test like Mutual Mobile’s out to first-round candidates, before they are even offered an interview, is still unusual.

“It’s a pretty interesting concept because oftentimes you’re new on a project, you don’t know the code at all, and you’re given assignments so you’ve just got to basically jump in,” White says.

Mutual Mobile is beginning to implement the test in hiring and Lisle hopes that the result will be a much more streamlined process. He took the test himself and said that it was “a bit of a challenge,” but should be fun for engineers who love what they do. White says he enjoyed the test.

“I hope they were honest with me in telling me how well I really did,” Lisle says, laughing. “The intent was it would not be this big burden, like do this work to prove yourself, but more here’s something to play with and show us what you can do.”