Foreword

It’s well known that most programmers wannabes can’t code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening.

The whiteboard interview has been the standard for a while, followed by puzzles [now abandoned], then FizzBuzz.

The latest fad is HackerRank. It’s introducing automated programming tests to be done by the candidate before he’s allowed to talk to anyone in the company.

A lot of very good companies are using HackerRank as a pre-screening tool. If we can’t avoid it, we gotta embrace it.

What to find in a HackerRank test?

There are 3 types of questions to be encountered in a test:

• Multiple Choice Questions: “What is the time complexity to find an element in a red and black tree?” -A- -B- -C- -D-

• Coding Exercise: “<Long description of a problem to be solved>, <input data format>, <output data format>.” Start coding a solution.

• SudoRank Exercise: “Your ssh credentials are tester:[email protected] <long description of what’s wrong with that server>.” SSH to the server and start fixing.

Any amount of any question can be put together, in any order, to make a complete test. A company should give some indications on what to expect in its test.

HackerRank provides a library of hundreds of questions and exercises ready to use. It’s also possible for a company to write their own (and recommended).

Defeating Multiple Choice Question

The majority of the multiple choice questions can be solved by an appropriate Google search. Usually on the title, sometimes on a few select words from the text.

Select Text => Right Click => Quick SearchGoogle has spoken! => all in favour of setuid()

Defeating Coding Exercises

Searching for a 10 lines long paragraph in Google is not an acceptable option. Not to mention that the HackerRank website disables copy/paste in the description area.

The workaround is to search for the title of the exercise. A title uniquely identifies a question on HackerRank. It will be mentioned in related solutions and blog posts. Perfect for being indexed by Google.

Select Text => Right Click => Quick SearchTouché!

The first result is the question, the second result is the solution. Well, that was easy.

Bonus: That google solution is actually wrong… yet it gives all the points.

// [boilerplate omitted]
int main() {
    int N;
    cin >> N;
    int tmp, result = 0;
    for (int i = 0; i < N; i++) {         cin >> tmp;
        result ^= tmp;
    }
    cout << result;
    return 0;
}

This solution is only correct if duplicated numbers are in pairs. All the HackerRank unit tests happen to fit this criteria by pure coincidence [cf. Addendum].

Originally, we put this simple question at the beginning of a test for warm-up. We received that answer from a candidate in our first batch of applicants. It was quite puzzling, what are the odds that someone would come up with an algorithm that convoluted if given only the text from the question? A quick investigation quickly revealed the source.

---

Addendum:

The “Lonely Integer” question is worded slightly differently in the public HackerRank site and the private HackerRank library but the input, output and unit tests are the same. Hence why the solution is off but works. HackerRank is obviously copying questions from the community into the professional library. That’s another copy-cat busted!

---

Recruiter Insights: Cheating brought to the next level

We have a lot of candidates coming from recruiters. How are they comparing to candidates from other sources?

Let’s see the statistics on a hard question [i.e. dynamic programming trading algorithm].

Distribution over all attempts, by all companies (log scale). 1234 zero vs 303 full score.

Most candidates get 0 points: ran out of time, unable to answer, wrong algorithm, or incomplete/partial solutions (i.e. good start but not enough to pass any unit test yet).

We wanted to show the same distribution over our pool of candidates but HackerRank doesn’t provide that graph anymore. It used to.

Anyway, we remember approximate numbers. The distribution for our candidates is about 50/50% on each extreme. That’s significantly better than the 75/17% from the general population. We can correlate that number with the time spent on the question and a visual code review.

The result is that candidates coming from recruiters perform better, especially on hard exercises. In fact it is unbelievable how much better they perform! (Special kudos to the guys who are able to solve a problem -with a perfectly optimized solution- in less time than it would take to actually read it =D).

The conclusion is plain and simple: Our recruiters give away the test to the candidates.

Do recruiters give away your questions? Of course!

Lesson learnt:

• For candidates: Remember to ask the recruiter for support before the test.

• For recruiters: Remember to coach the candidate for the test and instruct him to write down changes (if any).

• For companies: Beware high-score candidates coming from recruiters! In particular, don’t calibrate scoring based on extremes scores from a few cheaters.

Challenge: How long does it take you to solve a trading challenge? [dynamic programming, medium difficulty].

Custom HackerRank Tests

Companies can write custom exercises and they should. It’s hard and it requires particular skills but it is definitely worthwhile.

It is the only effective solution against Google, if done carefully. (It’s actually surprisingly  difficult to make exercises that are both simple AND not easily found with Google on 1000 tutorials and coding forums).

Sadly, it won’t help against recruiters. (Excluding the first batch of candidates who will be sacrificed as scouts).

Conclusion

Did we just ruin HackerRank pre-screening? Of course not! There is a never-ending supply of bozos unable to tell the difference between Internet and Internet Explorer.

We could write a book teaching the answers to 90% of programming interviews problems, yet 99% of job seekers would never read it. Hell, it’s been written for a while and it had no impact whatsoever.

Only the handful of devs following blogs/news or searching for “What is HackerRank?” will be able to come better prepared.

If anything, this article makes HackerRank better and more relevant. Now a test is about looking for help on Google and fixing subtly broken snippets of unindented code written in the wrong language.

HackerRank is finally screening for capabilities relevant to the job!

A last word of advice. You’re gonna need training if you are trying to pass the interview bar at good tech companies and you didn’t have the chance to attend MIT. This book is the gold standard when it comes to data structures and algorithms questions. Highly recommended. It contains hundreds of questions that you will be asked over and over again, in every company. At a minimum, read the free preview on Amazon.