Online job boards don’t work — how big data can fix the problem

In September, there were 3.5 million unfilled job openings, according to the Bureau of Labor Statistics. Yet only roughly three percent of those positions ended up being filled during that month. While that may surprise some, it’s sadly business as usual for those of us in the recruiting space. Some have argued that there is just not enough qualified talent out there to fill these positions, but in the course of my career I’ve come across hundreds of smart, capable candidates unable to even get their feet in an interviewer’s office.

It would be convenient to fault the candidate for not conducting a proper, modern job search. But not me – I blame it on technology.

Resume spam

Human resource experts will tell you that for any one job posting, they receive hundreds, if not thousands, of resumes. As a result, they’re able to spend just six seconds evaluating each resume, typically scanning only the candidate’s education and their last job. Such a brief perusal means candidates who are a good fit for a company’s culture or who can bring a different and much-needed fresh outlook can easily fall through the cracks. There is simply not enough time in the day to wade through all the resumes that flood the inbox.

The resume deluge started when businesses began to rely on online job boards to find candidates. With a few clicks of the mouse, job seekers were suddenly able to upload their resumes and cover letters and then apply to dozens of jobs at a time – sometimes even more than once. This “spray and pray” strategy has completely clogged up the system. Just last month, Facebook announced a new jobs app that, at launch, boasted it already offered users access to 1.7 million job opportunities. You can imagine how that will only compound the problem even further.

The adoption of digital databases to conduct the initial winnowing has done little to stem the tide. Job seekers know now to litter their resumes with the right key words to game the automated systems and grab recruiting eyeballs. It’s no surprise then that, on average, 66 percent of applicants for a given job meet the minimum qualifications. The current system is set up to deliver the wrong people to prospective employers. This bottleneck has meant a huge financial drain on businesses. Millions of man-hours are wasted sifting through inappropriate or unwanted resumes, costing a company on average $5,504 and up to six months per hire. Conversely, job seekers now spend a median of 19 months looking for the next position.

A technological solution

So how can we make labor markets move more efficiently and effectively, for the benefit of all? The solution is a technological one – big data. (Full disclosure: My company specializes in software that uses big data to connect job seekers with opportunities. We’re joined by companies like and TalentBin, both of which are trying to make job searching more simple and intuitive with the help of big data.)

Big data can help recruiters find the right candidates to interview by cutting through the noise created by the chaos of the current job search process. Big data tools such as modern distributed file systems and map/reduce/clustering techniques make large data sets accessible and more easily analyzed. Five years ago this simply wasn’t economically possible. Back then, it was cost prohibitive to purchase enough computer servers to make these calculations, and further, vendors were constrained by the physical size limits of data centers.

Now, vendors can process billions of transactions in the cloud at a fraction of the cost of local servers. Thus, employment-related data, regardless of size, can be leveraged to find subtle patterns reflecting a current candidate’s qualifications.

Another added benefit is the reduction of human bias. All human recruiters, regardless of background, bring a bias to the resume evaluation – it’s human nature. Big data algorithms, though, are blind to names on resumes that may surface a job applicant’s race, ethnicity, religion or gender. Machine-learning algorithms, utilizing large implicit- and explicit-feedback datasets, can be trained to simulate decisions made by professional recruiters and thus reduce or eliminate evaluator bias.

As a result, big data allows for a multi-faceted statistical approach to the filtering process at the first level, and thus helps identify better candidates from a deeper pool.

To be clear, this isn’t to say big data can replace a job interview. The interview is about culture fit, body language, eye contact, voice intonation, and the discussion of a general fit between the job seeker and the position. Technology can’t solve that on its own. Technology can, however, make sure that candidates you bring in for an interview are the best qualified, right from the first screening.

The job search is not rocket science. But the application of data science and big data can streamline the process so interviews are filled with the best fit candidates more quickly, efficiently, and at a lower cost. Big data can revolutionize the labor space.

Steve Goodman is CEO of  

Photo courtesy of Everett Collection/