It’s tough for a job-seeker out there. Stories of interviewers demanding ever-increasing scores on coding tests or even asking candidates to complete hours-long demonstration projects are becoming routine as companies offer potential staff to the altar of efficiency.
Interviewing.io founder and CEO Aline Lerner, whose platform pairs job applicants with industry veterans for mock interviews, said their data shows the trend is much more than anecdotal. For example, Interviewing.io users training for jobs at top-tier companies like FAANG firms needed to score in the 83rd percentile on coding tests in December 2023, up from the 65th percentile in early 2022.
Lerner sat down with IT Brew to explain why technical interviews are out of control, warn firms about cheating, and relay advice for applicants.
This interview has been edited for length and clarity.
What does the intensification of these interviews look like for the applicant?
It’s not necessarily harder questions, but people are expecting you to solve them faster, and they’re expecting you to have pretty much an optimal solution. It’s kind of sad, because the purpose of these interviews is to see if you can think.
Now, because the bar is going up, and I think because so many people are grinding on [code interview platform] LeetCode and just memorizing a bunch of questions. It’s like the boiling frog thing, right? They keep seeing people solve problems faster and faster.
If we read some feedback on our platform more recently, interviewers are like, “Well you did well, but you didn’t get to a perfectly working solution. So this would be a fail in real life.” And that’s shocking.
When you have more and more applicants for fewer positions, you can afford to be increasingly picky. And I think that that is the same pressure [drives employers and applicants].
With the rise of ChatGPT and stuff like that, is there a risk that these technical interviews are losing validity?
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
We did an experiment. We told the interviewers that we were studying different question types, but we didn’t tell them anything about ChatGPT. And then we told interviewees that they had to try to use ChatGPT to cheat...Each interviewer got assigned to a type of question [either verbatim off LeetCode, tweaked from LeetCode, and completely custom].
Crazy thing! None of the interviewers could tell.
The more interesting finding is that for LeetCode questions and LeetCode questions with a twist, interviewees passed interviews at something like 70%. Out in the wild, a good pass-through rate at a top-tier company is like 25%. On our platform, [the] average pass-through rate is 53%. So, it was still significantly higher...On those custom questions, the pass-through rate was like 23%. So, ChatGPT is much worse at those.
What advice do you have for job applicants who are worried about the workload of applying for jobs, whether that’s prepping for a test or spending 20 hours building this project for an employer?
One practical piece of advice might be to just sit this out. I think that hiring will be back and feel more like 2021, early 2022, in as little as the next three to six months.
If you don’t have the luxury of waiting, I would say that it’s important to practice, much more important than it used to be, because the bar is higher. That doesn’t have to be on our platform, but just practice with another human. It’s much more effective than grinding on LeetCode. Get used to talking out loud. Get used to explaining what you’re doing, working on your speed. Get to the point where you can answer questions fairly quickly.