How to shortlist candidates fairly.

Shortlisting means choosing which candidates move on to the next interview. It’s where most hiring decisions really get made. By the final interview you’re choosing between a handful of people, but the bigger choice was which handful to invite. Yet shortlisting often has the least structure of any step. A recruiter skims a pile, goes with a feeling, and nobody can say afterwards why one person moved forward and another didn’t. A fair shortlist uses the same criteria for everyone, looks at evidence rather than impressions, and leaves a record someone else could check.

1. Agree the criteria before you look at anyone

Criteria set after you’ve seen candidates tend to describe the candidate you already liked. Decide them first, with the hiring manager, and write them down.

Good shortlisting criteria are:

• Tied to the job. Each one should connect to something the person will do in the first few months.
• Visible. You can point to something in the interview or application that shows it.
• Few. Three to five skills. With more than that, reviewers start deciding by instinct which ones matter.
• Clear about level. Say what level you expect. A beginner level for a trainee job is not the same as an advanced level for a team lead.
• Weighted, if some matter more. If handling difficult customers matters twice as much as product knowledge, say so up front.

Keep must-haves, like shift availability or the right to work, separate from skills. They’re yes-or-no facts. A person should read them, rather than letting them quietly lower a score.

Never include anything related to protected characteristics, such as age, gender, religion or disability, which anti-discrimination laws cover. Also be careful with things that can stand in for them, such as gaps in employment, graduation year or where someone lives, unless the job genuinely requires it.

2. Review evidence, not impressions

Confidence, smooth speaking and a familiar accent are easy to mistake for skill. The fix is to ask, for each criterion, “what did they actually say or do that shows this?”

A simple habit: before you rate a skill, find the sentence that supports your rating. If you can’t find one, you’re rating an impression.

The evidence column is also what you’ll need if anyone later asks why a decision was made.

3. Get your reviewers to rate the same way

Two reviewers reading the same interview will often disagree. That’s normal. What matters is whether you notice and correct it. Getting reviewers to agree on what each rating means is called calibration.

Before you review a large group of candidates:

1. Pick five to eight completed interviews that cover a range, from clearly strong to clearly weak.
2. Have every reviewer rate them on their own, against the criteria.
3. Compare the ratings side by side. Discuss the biggest differences: what did each person see, and which reading is closer to the criteria?
4. Write down what you agreed, for example “an answer that names the steps but not the outcome is intermediate, not advanced.”

Do this again briefly every few weeks, or whenever a new reviewer joins. Reviewers’ ratings shift slowly over time, and it’s hard for them to notice it themselves.

4. Take a second look at people who just missed out

Any cut-off, whether it’s a score, a gut feeling or “the top 20”, will leave out some good people. Plan a regular second look.

• Check some people below the cut-off. Each week, read a handful of candidates who just missed out, not only those who scored far lower.
• Look for patterns. If the people who just missed out share something unrelated to the job, like a particular college, region or language, stop and check the criteria.
• Watch for short answers. A nervous candidate, or a weak connection that dropped words, can produce a short answer that hides a strong person. Check the recording or transcript before setting them aside.
• Move people back when you’re wrong. A shortlist can change. It isn’t a final verdict.

If you use a pass mark (a minimum score), the second look shows you whether it’s set in the right place. The pass mark guide covers how to set and adjust one.

5. Write down every decision

A short record protects candidates and your team. For each person, you should be able to say:

• Which criteria they were assessed against, and at what level.
• The rating for each, and the evidence behind it.
• Who made the shortlisting decision, and when.
• If a decision went against the ratings, why.

This doesn’t have to be long. One line of evidence per skill and a name against the decision is enough for most jobs. It also improves the next interview, because final interviewers can start from what you already know.

Common mistakes

• Shortlisting on résumé keywords. It rewards people who know how to write résumés, not people who can do the job.
• Letting the first few candidates set the standard. Whoever you review first quietly becomes the standard for everyone else. Agree on ratings using a mixed set of interviews instead.
• Treating the shortlist as final. Candidates just below the cut-off deserve a second look before you close the job.
• Keeping no record. If you can’t explain a decision a month later, you can’t check it was fair.

Where AI scoring fits

AI scoring can make shortlisting more consistent, as long as a person still decides. In LessRounds, each skill is scored 0 to 100 against the level you set. Every score comes with the reasoning and a quote from the transcript, which covers much of steps two and five. Candidates below the pass mark are set aside without being contacted. Anyone on your team can move them back at any time, which makes the second look quick. Scoring reads the words of the answer, not faces or voices, and never marks down language or grammar. It won’t make a process perfectly fair, so keep comparing your reviewers’ ratings and keep checking below the cut-off.