These days, the media focus is on how AI is destroying jobs like customer support, and rightly so (AI is indeed decimating the need for a lot of customer support).
However, there is another aspect of AI that is killing the entire hiring process.
And that is the job application process.
Every job requirement is getting hundreds (if not thousands) of applications, due to which hiring managers and recruiters are getting completely swamped.
To make it worse, resumes these days seem to be hyper-optimized with AI as if Shakespeare himself wrote them.
In this post, I want to cover the hiring process (rather than the inherent change in the “nature of work” that AI has caused) – and the creation of the world’s first AI hiring assistant, SortResume.ai
So What Is The Problem At Hand?
The major problem is that due to AI — and possibly the recessionary job market — job postings are getting flooded with a tsunami of resumes.
There used to be a time when getting 10 applications would be a cause for celebration for hiring managers and recruiters. Now, every job application on Indeed and Linkedin is getting hundreds — if not thousands — of resumes.
This is causing a ton of headache for hiring managers and recruiters.
Problem 1: Sort Through Thousands Of Resumes.
With the spread of AI and its accessibility to a broader audience, job applications are easier to submit, resulting in an overwhelming number of resumes for each job posting.
To make matters worse, the platforms like Linkedin and Indeed have made it dead simple to apply for a job. Candidates can pretty much go “click-click-click” and apply for the job.
Dang — there are even “auto apply” bots that will automatically apply for jobs on Indeed. How messed up is that (for hiring managers!)
This deluge means that hiring managers and recruiters must sift through hundreds, sometimes thousands, of applications for a single position.
Tell me about it — every night, I had to spend hours sifting through a thousand applications to find the good ones. It completely crushed the hiring funnel — at a time when I badly needed to hire people.
The sheer volume of resumes makes the process time-consuming and less efficient, as it’s harder to give each resume the attention it might deserve.
Problem 2: Good Applications Getting Lost In The Pile
As the number of applications per job increases dramatically, the likelihood of high-quality applications being overlooked also rises.
I spoke to an amazing engineer a few weeks ago who is on the job market and he sadly told me:
I put a lot of energy to apply for a specific job I am perfect for — but then the resume gets buried by these 800 robo-applications.
This occurs because, in an effort to manage their workload, recruiters might use keyword searches to filter applications, which can sometimes dismiss potentially excellent candidates based on rigid criteria that may not capture the nuances of a candidate’s experience or potential.
Alternatively, manual reviewers burdened by volume may overlook finer details that differentiate a great application from a good one — like the example of my friend, the good engineer. This leads to potentially valuable candidates not being shortlisted for interviews, affecting the quality of the hiring process.
Problem 3: Difficulty In Assessing Genuine Candidate Skills
Due to resumes being hyper-optimized with ChatGPT, it becomes exceedingly difficult for hiring managers to discern genuine skills and experience from embellished credentials. This leads to challenges in accurately identifying the best candidates for the job, potentially resulting in poor hiring decisions.
The old technique of “keyword search” on resumes won’t cut it anymore, especially when you have a 100 or 1000 resumes in front of you.
Lets’ Go — AI To The Rescue – Introducing SortResume.ai
Well you saw this coming, if there is a problem these days, it’s AI to the rescue. And believe it or not, the AI solutions are doing an incredibly good job helping with this problem at hand.
So rather than giving you an abstract view, here is my experience using AI to solve this problem.
As mentioned earlier, I was sick and tired of staying up at night filtering through hundreds of resumes to find the matching candidates. After doing this for couple of nights, it raised the questions:
Hey, can’t GPT-4 do what you are doing manually?
Heck yeah — absolutely yes!
And that led to the creation of an automated AI hiring assistant — SortResume.ai – one that can quickly score resumes against the job description and other “unspoken” criteria.
So instead of manually filtering candidates, which can cause bias and missing out on key skills, the AI is now able to give me a “Candidate Score” along with a Leaderboard.
So basically, the entire filtering process is now about uploading the resumes to the AI system and getting back a score (like a FICO score) — saving oodles of time and stress.
Yeah — that’s it — the process is now as simple as :
- Upload resumes.
- Analyze leader board.
- Select candidates for interview.
This new process has made the hiring process MUCH faster — irrespective of the number of candidates. I could now get 1000 candidates and it would not matter or cause more manual time sinks.
In other words: AI is scalable — while manual filtering is NOT.
How Does The AI Hiring Assistant Work?
OK — since everyone is technical these days, I’m sure you would want to know the exact breakdown on how the score is calculated and how the AI works — so without further ado, here goes.
Step 1: Breakdown The Job Into Criteria
The first step in the process is to use the GPT-4 API to break down the job description into N criteria. So by simply copy-pasting the text of the JD into a good prompt into GPT-4, the reasoning capabilities allows the JD to be broken down into say 10 weighted criteria.
Do note: The weights are important — because not all criteria are equally important to the final score.
Ninja tip: You should also have “unspoken” criteria that are not clearly described in the public job description. This could be your secret sauce related to your company’s culture, values, mission or hiring practices.
So at the end of this step, you will have a list of weighted criteria as shown below.
Please note: This criteria is automatically calculated by the AI — you don’t have to do this manually. AND the best part: the AI will explain itself and tell you how it is working.
Step 2: Score Each Resume For Each Individual Criteria
The next part is to score each resume using GPT-4 for each individual criteria. So for example, in my example above, there is a criteria “STEM Degree” with a weight of 8.
So if John Doe applies for the job, his entire resume and the job description is used to calculate his score on a scale of 1–10 using GPT-4.
PS: If you want to replicate this process, you will need to do some prompt engineering to get it right.
In the example above (from a candidate we recently hired), he scored a 10 for this criteria, giving him a weight score of 80 (10 x 8)
Ninja Tip: Notice how the AI is clearly explaining its scoring mechanism, thus giving explainability and trust for the hiring manager (or compliance departments).
Each of the weighted scores from the individual criteria are then summed up to give a final total score. This would be like the FICO score for this candidate for this job.
Technical note: This means about 20 GPT-4 calls per resume.
Step 3: Human-In-The-Loop : Analyze The Leaderboard
The final step is for the human-in-the-loop (aka: the hiring manager or recruiter) to analyze the leaderboard and see which candidates to bring in for the interview.
In this step, the hiring manager can choose to slice-and-dice the individual criteria analysis, and can even change weights if needed (for example: If you want to weight a certain criteria more or less than what the AI assessed)
At this point, based on the leaderboard, the total score and the candidate metadata, the hiring manager can now make a clear decision about whether to bring in the candidate for an interview.
The beauty of this process is that the hiring manager gets a full-fledged X-ray report on the suitability of these candidates FOR this given job. This becomes a key insight in the later stages of the hiring process. It’s like a mortgage officer being equipped with the full credit report.
For example, here is a sample X-ray report for a given candidate.
Please note: the AI is NOT making the decision here. The AI is simply giving the hiring manager the ammunition they need to make a decision. So in essence, the AI is a scalable hiring tool — rather than the decision maker.
What’s Next?
While this is a great start for a scalable “AI hiring assistant”, there are clearly some enhancements that need to be done in the future.
Future 1: Blind Evaluation
The AI system currently evaluates the raw resume as received in the application. To reduce bias, it would make sense to first anonymize and remove any bias factors. For example, removing the name would be a perfect first step, so that the AI (or anyone!) is not biased based on the name.
There have been numerous studies that human recruiters (and AI) are severely biased based on the name and gender — so anonymizing the resume would greatly help reduce that bias.
Future 2: Platforms
I would expect platforms like Linkedin and Indeed to integrate such scoring algorithms straight into their platforms. To be honest, these platforms have gone overboard in how easy they have made it for candidates to apply to a job — so basically adding fuel to the fire.
This almost reminds me of the common app for college applications, where students now go “click-click” and apply for a college — making it a much more difficult task for admissions officers to filter in (or out) good candidates.
If not the platforms, I would expect the ATS systems like Greenhouse or Bullhorn to do this — although these systems are way outside the affordability of most startups and small recruiting firms.
Future 3: Automated AI Interviewer
I would expect that the next step in the process would be to have an automated AI interviewer that would further enhance the X-ray report on the candidate. Currently, a huge amount of time is spent phone screening candidates. (If you don’t believe me, just ask your recruiter why they charge 20% of first year salary!)
With an automated AI interviewer, the candidate could be screened just like how a real human interviewer would — thus giving a clear transcript and “Interview Score” — without a human having to go through the arduous process of scheduling a phone interview.
Frequently Asked Questions
What LLM are you using and Why?
We use the GPT-4 API. GPT-4 has been shown to have the highest reasoning ability and so it is best suited for an application like this. In particular, scoring the criteria on a scale of 1–10 AND justifying the score requires a very high level of reasoning capability.
For cost reasons, we currently use GPT-3.5 on our Free plan. The current processes do burn a TON of API calls, so it makes sense to use a lesser model in order to increase accessibility of the service. Another option we’re exploring is Claude Haiku, as it would be a good tradeoff to save costs, while not sacrificing much in reasoning capability.
How does this play into bias?
So the good news is that this process is a LOT (and I mean a LOT) less biased than humans. However, “bias” is like security, you will never get to 100% — all you can hope for is to reduce it as much as possible.
I raised this issue in a recent Facebook forum and got some good ideas to mitigate bias, which could be part of future enhancements.
How much time is this really saving?
This process is saving HOURS of time in initial tests. More than time, it reduces stress — because sifting through a thousand resumes at night is quite stressful and makes the entire process very tedious, for no additional gain.
Besides the time and stress, if a hiring manager or recruiter is going to spend 10 seconds on each resume, it is very difficult to be objective, and lots of good skills can be missed. A highly intelligent AI like GPT-4 can assess each criteria deeply and make the process much more efficient than any human ever could.
From personal experience, I recently uploaded 28 resumes for an Open job. Analyzing these resumes would have taken me over an hour. With this process, I simply uploaded the resumes and within 5 mins, I had the leaderboard ready for analysis.
What is the cost of this process?
This process is indeed quite expensive, amounting to about 20 GPT-4 API calls per resume. In initial tests, I am seeing the processing cost at about 85 cents per resume. As AI becomes cheaper, this cost will invariably drop tremendously.
Also, I’m currently using GPT-4 because it is the best at reasoning. By using GPT-3.5 or Claude Haiku the costs would drop to 2–3 cents per resume.
Side note: This “cost” does not include the cost of building the software if you are considering building this in-house. My advice would be to “Buy It” — rather than “Build It” (See Why)
Is this AI process scalable?
Absolutely — and that is its №1 advantage. It is impossible for humans to filter through 1000 resumes, but an AI can get this done within seconds.
If you think I am kidding, check out this actual job posting of mine from Linkedin from 2 months ago — it got 1,361 applications which would be impossible to filter manually. (PS: I had to revise the job posting and use this new AI process to actually be able to hire effectively)
A few years back, such a system would not be needed. If you are getting 10 resumes, then you don’t need AI to filter and score them. But with job applications getting 100s and 1000s of resumes, a tool like this is needed to be able to do it scalably and efficiently.
Can I trust the AI to do a good job?
Yes — especially when combined with “explainability”. You would have noticed that my prompts instruct the AI to justify its scores with a clear explanation. This explanation is critical to build trust and survive an audit.
For example: For some of my job postings, I was getting resumes from far-off places that were clearly not within commuting distance of our office. By just adding a single instruction “Must be within commuting distance of Needham, MA”, I was able to get a clear score between 1 and 10 for this “commuting criteria”.
Side note: I was quite stunned at how good GPT-4 was estimating the commuting distance between the candidate’s location and our office. While it is not actually calculating the distance, it’s uncanny ability to estimate the distance and provide a score was astonishing.
How do I explain myself to the boss? (or to compliance)
Firstly — the AI is NOT making a decision here. The AI is being used as a TOOL to make you more efficient. It’s like an X-ray report or a FICO score to help you be more efficient.
That said, the AI has clear explainability built in. So every score is accompanied by the justification and explanation of why the AI gave it a certain score on a scale of 1–10.
My recommendation: Give your boss the X-ray report along with the list of candidates that should move to the next step in the process.
How does an AI hiring assistant play with current and future laws?
The laws around AI in the hiring process are changing rapidly, so it’s important to stay current with them. In particular, the EU Act and NYC’s law are worth noting.
In both cases, the laws prohibit using AI as automated decision makers. Due to this, it’s important to use AI tools as just that — AI TOOLS — that make the human more efficient to make the decision.
Again: The HITL (human-in-the-loop) aspect is important — because it’s clear that neither HUMAN — nor AI — can do this by themselves.
Does the AI need to be “fine tuned” or “customized” for my industry?
No “fine tuning” in the classical sense is required. The beauty of Gen AI is that “English is the New Coding Language” — so you can customize the AI using simple English instructions.
For example, I mentioned how I was able to give preference based on commuting just by including a single English instruction.
Take a look at No 3 above. Using this simple sentence, the AI can give preference to Harvard over say your local college (no disrespect intended!)
The benefit of this is: Non-technical hiring managers and recruiters can customize the requirements with ZERO coding. For real, no data scientists and AI/ML people are needed in this process.
Can this work for other “candidate selection” processes?
Aha — possibly. I can see this exact process being used in any form of candidate selection like grants or proposals. In fact, I won’t be surprised if the same process and algorithm could do this — possibly something to test in the future.
The only requirement would be : The number of applications would be something that the human is not able (or willing!) to manually process. For example, if you get a 1000 grant proposals, then using this process could help.
It’s very likely that colleges will use some variation of this process to go through the deluge of admission applications.
Will GPT-4 train on the candidate resumes?
No. This system uses the GPT-4 API from OpenAI, which OpenAI has guaranteed not to train on.
That said, if you are using the public ChatGPT to do this process, indeed your resume data could be used for training. So please check the ChatGPT terms if you plan on replicating this process using the public ChatGPT interface.
Wait — why do you need 20 GPT-4 calls per resume?
Because by evaluating each criteria separately in a unique API call, it gives the highest accuracy with the best explainability.
Previous attempts at doing a single call or short-cutting using RAG had sub-optimial quality results.
Dude, can’t you just simplify this with RAG ?
Actually no — in fact, our first version was based on RAG built with CustomGPT.ai — we loaded 65 resumes into a RAG and then asked questions like “Give me the top 10 candidates based on criteria X” .. this option greatly reduced the number of API calls needed, but was extremely inaccurate. RAG and GPT-4 are not very good when it comes to math and ordering (like “show me top 10”)
The better approach is this non-RAG approach where each criteria is evaluated separately in a single API call.
However, that said, I do see a future where the single API call to provide a score between 1–10 becomes a RAG agent — and that is when companies need the evaluation to happen per their own specific policies.
For example: if a company needs to evaluate a candidate based on criteria X and the company has very specific rules around this evaluation for X, I see the GPT-4 call becoming a CustomGPT.ai RAG API call with the agent loaded with the company handbook or policy documents.
Can you explain the algorithm in mathematical terms ?
Ok — I get it — the math geeks need the Greek letter formula .. so here it is (with a little help from ChatGPT to format it)
I’m direct sourcing from other companies and have a list of Linkedin Profiles. Does your system work with Linkedin Profiles?
Yes — you can simply “Save as PDF” in Linkedin and use that as the resume (which it is!) — the system can score the resume for that job accordingly.
And Finally …
SortResume.ai is now live and available for general use — you can get started for free.
