Practical guides
The use of data-driven tools is rising across the recruitment sector*. This presents important opportunities for the sector, including improved efficiency of time-intensive processes. However, there are also a number of significant risks, with high-profile case studies highlighting the risk of discriminatory and unfair results if these tools are not designed and used with care.
The CDEI’s review into bias in algorithmic decision-making highlighted these issues and recommended that guidance be developed to help recruiters make informed decisions on their use of data-driven tools. This guidance aims to give recruiters who are looking to buy data-driven tools the mechanisms to effectively evaluate and responsibly deploy them, ensuring that appropriate steps have been taken to mitigate risks and maximise opportunities.
This guidance has been developed jointly by the Recruitment and Employment Confederation (REC) and the Centre for Data Ethics and Innovation (CDEI). The CDEI is an expert government body enabling the trustworthy use of data and AI. More information about the CDEI can be found on the CDEI web page or by contacting cdei@cdei.gov.uk.
Section A is an introduction to the data-driven tools used in recruitment processes, covering existing types of tools available in the recruitment funnel - across sourcing, sifting and interviewing - and the kinds of risk they may bring.
Section B provides guidance on incorporating and using data-driven tools, including:
We use the term “data-driven tools'' to refer to technologies and techniques that help to generate useful insights from data. This could range from the use of software for big data analysis, to process automation systems, and on to machine learning or other forms of artificial intelligence (AI).
This is predominantly a buyer’s guide: it assumes that recruiters are deploying tools bought from external technology suppliers. However, much of the guidance will still be applicable if the tool is built in-house.
The focus is AI and data-driven tools: this guidance should be used in addition to existing practice for compliance in recruitment.
* (CDEI Review into Bias in Algorithmic Decision Making Chapter 3: Recruitment)
This table summarises the guidance provided in the rest of this document and is particularly applicable for tools procured for the sifting and interviewing stages of the recruitment process. For more details and explanations, please see the full guidance below.
|
Before purchasing |
During purchasing |
Before use |
During and after use |
|
Articulate requirements |
Seek information from the vendor Verify legal compliance |
Consider running a pilot Put transparency processes in place |
Monitor and assess the tool’s performance |
|
Set out your objective for the tool in a procurement document |
Seek information from the vendor about how the tool works and articulate any concerns |
Ensure the tool works correctly, using a pilot to test this where appropriate |
Monitor against original business case. |
|
Develop internal baselines for protected groups and agree acceptable limits for disparity between groups
|
Seek information from the vendor about the underlying assumptions and testing for bias and accessibility. Ask for copies of auditing documentation and check how often audits take place. Consider completing an Equalities Impact Assessment |
Assess the equalities outcomes of the tool in a pilot test
|
Continue to test the equalities outcomes of the tool in practice Assess any disparity between different groups |
|
Begin completing a Data Protection Impact Assessment (DPIA)
|
Check for compliance with relevant legislation: UK GDPR and Equality Act 2010 |
Give candidates appropriate notice for data collection Provide a means for human review |
Ensure ongoing legality (e.g. maintaining DPIA) Reflect on any issues and discuss with the supplier |
|
Engage with vacancy holders or clients to understand where automation will fit into existing processes |
Consider engaging with protected groups and representatives to understand potential risks and respond to them |
Provide information to candidates, making them aware of the use of a data-driven tool and its purpose |
Seek continual feedback from candidates and vacancy holders |
In this section, we introduce existing types of tools available in the recruitment funnel and the kinds of risk they may bring. Guidance on how to approach and mitigate these risks can be found below.
| Sourcing ➔ | Screening ➔ | Interview ➔ | Selection |
| Job description review software | Qualifying screening tools | Voice and face recognition in video interviewing | Background check software |
| Targeted advertising | CV matching | Offer predicting software | |
| Recruiting chatbots | Psychometric tests and games | ||
| Headhunting software | Ranking algorithm |
Sourcing tools are used across the recruitment sector to attract a wide pool of high-quality candidates and begin to engage individuals with employment opportunities. For example, the audience for a job advert will have a substantial impact both on the quality of the pool of candidates, and the individual rights of those who may be interested in applying.
On social media for example, targeted advertising can reach suitable or new candidates using specific criteria.
These technologies present opportunities to make the recruitment process more effective, reach a wider pool of candidates, and provide a more bespoke service to potential applicants. In some cases, these technologies can be used to improve the diversity of an applicant pool and some recruitment professionals have highlighted the underutilisation of these tools as a barrier to diverse recruitment.
However, there are also a number of risks. The sourcing stage dictates which candidates are aware of opportunities and, without proper usage, these tools may present unforeseen barriers - in particular to protected groups - and may have discriminatory or otherwise unfair consequences.
|
Example tools in the sourcing stage |
|
Recommendation systems on platforms and job boards which prioritise job vacancies for candidates based on their criteria. |
|
Chatbots or conversational AI to engage with candidates: these are often used to guide applicants through the stages of the process and can improve efficiency. |
|
Multi-database candidate sourcing: enabling recruiters to search for the most relevant candidates either through a simple keyword search or more complex identification of active and passive candidates. |
Social media companies collect information on the users of their platform, such as name, education, interests, as well as patterns of location and usage. Platforms can then use this data to train machine learning models, which are able to make inferences about users and group similar types of accounts together.
Based on the information provided and the inferences made, a recruiter can use online platforms to target advertisements at users with relevant characteristics. They can use demographic qualities to target users, or use “lookalike” features where an ad is targeted at profiles similar to one identified by the client.
These features can be used to positive effect: for example, a company with low levels of diversity could consider using targeting to ensure job ads are seen by a wider or more diverse audience than their typical applicant pool. However, there are also examples of these targeting tools replicating historical biases and limiting some groups’ access to opportunities.
The sifting and interviewing stages are the key decision-making moments in a recruitment process and data-driven tools to support those choices are available. In most cases these tools are not yet sophisticated enough to make decisions on recruitment without any human intervention at all. It is likely to be more appropriate to see these tools as providing useful assistance in making recruitment decisions.
Human bias is a known issue in the sifting and interviewing processes. Automated tools have the potential to reduce this human bias by providing a standardised approach to evaluating applications. However, badly designed or implemented tools - for example those trained on biased historical datasets or which do not adequately take into account the needs of protected groups like disabled people - risk perpetuating existing biases and creating new barriers.
The introduction of automated systems can lead to concerns around transparency, fairness, and accuracy as candidates are not clear on how they have been assessed. Ensuring compliance with data protection and equalities law is key to maintaining trust and legal compliance when deploying data-driven tools.
| The sifting stage: | The interviewing stage: |
| CV screening and evaluation software | Video screening software including facial, voice, and emotion expression recognition (these tools are very high risk) |
| Structured question assessments based on experience | Audio transcription |
| Game-based assessments including psychometric testing | Screening and evaluation of transcription material |
| Task-based assessments |
In 2018, it was reported that Amazon had been using an algorithm that favoured male candidates in its internal recruitment process. Their recruitment algorithm was trained on 10 years of historical employment data, and the majority of the candidates had been male. From this pattern, the algorithm “learnt” to favour men at the expense of women.
As reported by Reuters, the algorithm downgraded applicants who attended women-only universities, as well as those with CVs that included the word “women’s”, as in “women’s sports team”. Upon discovering this unintentional bias against women, Amazon stopped using the algorithm. This incident illustrates the broader issue of algorithmic bias: when historical data is used to inform future decisions, patterns of exclusion can be repeated or even exacerbated.
https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G
This section suggests generalised steps to follow when incorporating any data-driven technology into your recruitment process; it is in three parts:
1. Evaluating the value and effectiveness of the tool
2. Key considerations for data-driven recruitment tools
2.1 Accuracy and legitimate underlying science
2.2 Bias and discrimination (including accessibility)
The level of assurance you apply to the procurement and usage of a data-driven tool must be proportional to the risk involved. The risk level exists on a spectrum; it is guided by the law, but based on your professional judgement, and this guidance should be implemented to a degree that is proportional to that risk.
You need to make sure that you know who in your organisation is responsible for taking each of the steps laid out below, as well as who has overall responsibility for the process. This is likely to include a mixture of procurement teams, legal or compliance teams, data protection officers, and others. It might be that you identify someone internally to upskill and give them overall responsibility for ensuring compliance with the guidance.
Where there are any questions or steps you do not feel qualified to answer or undertake, and you do not have the expertise in-house to support you, you should seek external expertise to ensure your procurement decision-making process is robust.
| Before purchasing the tool: |
| Identify who will have overall responsibility for the purchase and use of the tool. |
| Identify who will be responsible for each of the steps in the process outlined in the guidance |
Set a clear vision of what you’re hoping to achieve in using the tool, and where it will sit within the recruitment process overall. This should include deciding whether you want the tool to make fully automated recruitment decisions, or whether you intend to use the tool to support recruitment decision-making. If it is the former, specific data protection legislation (Art 22 UK GDPR) is triggered and additional provisions need to be made (see 2.4).
Some tools may also help standardise assessments and address any bias in existing recruitment processes if used strategically - for example, to standardise the bar for entry at the first round of testing. They may also widen access to talent pipelines, help source candidates with particular skill sets and reduce operational costs. Think carefully about what the intended benefit of using the tool is and therefore how it should be incorporated into the recruitment funnel.
These are important exercises to undertake before starting to procure a tool to ensure that the product will meet your specific business needs and provide value for money.
|
Before purchasing the tool: |
|
Write down your proposed use case clearly before starting procurement, this should include: If and how the automated element will interact with manual steps and integrate with the wider process. What the business need and objective is for procuring the tool (such as addressing biases in current recruitment practices; widening access to talent pipelines; sourcing candidates with a particular skill set; reducing operational costs). |
| Use this information to assess the tool when deciding whether it meets your specifications during procurement. |
If a tool or it’s interface does not function as intended, the benefits of using the tool will be limited and it may hinder applicants unfairly, or put them off. Vendors should be able to provide you with information about the tool’s functionality and take you through this in a demo.
| Before purchasing the tool: |
|
Seek information from the vendor on the functionality and processes behind the tool.Ask for a demo of the tool, particularly if the tool will have a substantial impact on decisions made in the recruitment process or candidates will interact directly with the tool. Examples of questions you could ask are:
|
| Ensure you have appropriate technical support built into the contract so there is the option for troubleshooting support from the vendor, should any technical problems arise. |
| Explore the option of running a pilot of the tool to understand how it operates in the context of your organisation. |
Data-driven tools provide opportunities to make hiring processes more transparent and equitable. However they are not without risks and legal requirements; these must be carefully considered and addressed alongside existing methods to mitigate against discrimination, promote transparency and meet legal requirements.
It is important to understand what decision-making process the tool is automating (e.g. CV sifts, assessments, or interviews) and understand how the tool makes decisions in order to judge whether your organisation is comfortable with the underlying assumptions made by the technology. For instance, if the tool is using game-based assessments to evaluate candidates, then ensure that the underlying science of those games is sound.
The method of candidate evaluation should not be based on arbitrary factors or aspects of human behaviour. Above all, assessments must be linked to the skills needed for the job.
|
Check for a clear scientific evidence base before purchasing the tool: |
| Seek information from the vendor about the underlying science behind the given decision-making process. |
|
What is being evaluated, and is there a clear evidence base? You could ask the vendor to cite the scientific studies that the tool is based upon. Be wary of any methods of assessment that are not based on high-quality scientific methods, in particular tools that claim to infer human emotions from video interviews, as evidence suggests these have substantial inaccuracies and are potentially discriminatory. Ask for copies of auditing documentation in order to understand whether the outcomes produced by the tool (for example, who is recommended for recruitment) have been evaluated in order to make sure that those outcomes are accurate. |
In data-driven tools, bias may emerge in two separate but related ways which can compound to create cumulative bias.
a) Learnt bias: where data-driven tools “learn” how to make hiring decisions based on existing employment data. If there are patterns of bias in that data, then the data-driven tool is likely to repeat those patterns, which can lead to unfairness.
b) Inaccuracy: tools that analyse face, speech, or voice may be less accurate for particular demographic groups and introduce discrimination through simply working less well for those groups. Research has shown that facial recognition technology can be less accurate for people with darker skin, and particularly for women with darker skin. Tools that analyse aspects of voice or tone or that transcribe audio to text may be less accurate for certain accents and manners of speech.
Good vendors will test their technology, for example: auditing and validating the underlying algorithms to protect against potential bias. Recruiters should demand this testing when looking to procure such tools. However, be aware that this testing may have been conducted in a different country, such as the US. It’s important to be aware of the differences in demographics and legal standards, such as the four fifths rule.
The Equality Act 2010 prohibits discrimination on the basis of nine protected characteristics (age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex and sexual orientation) which applies to the use of data-driven tools. More information is available on the Equality and Human Rights Commission’s website.
Direct discrimination occurs when someone is treated worse than another person because they have - or are perceived to have - a protected characteristic, or because of an association they have with a person with a protected characteristic. For example, if a data-driven sifting tool ranked women lower than men based on their recorded gender.
| Test for bias before purchasing the tool: |
|
Seek information from the vendor: vendors will likely have tested tools for bias in their own relevant testing contexts. Use this as a basis for assessing whether you are comfortable with the tool. Be aware that UK populations may be different, and that UK legal standards may not have been used. In particular, many vendors use the US-relevant “four-fifths'' rule (see above) to determine how much adverse impact is permissible. Decide if your team is comfortable with the vendor’s results, keeping in mind that the context may be different. |
| Seek information on the diversity of the vendor’s engineering and product development team including the composition of their team. Check if the team has been trained on unconscious bias. |
|
Ask for copies of auditing documentation and check how often audits take place, in order to understand whether the outcomes produced by the tool (for example, who is recommended for recruitment) have been evaluated in order to make sure that those outcomes are non-discriminatory. For an additional level of assurance, you may wish to commission independent evaluation of the tool. |
| During purchasing the tool: |
| If the analysis (method set out in the box below) suggests that the tool produces bias and you want to continue with procurement, you should provide an objective justification for this decision (a proportionate means for achieving a legitimate aim). This standard is unlikely to be met unless you can evidence that the assessment tests for required attributes and there are no alternative measures for testing candidates on these requirements. |
| Before, during and after using the tool: |
|
Consider running tests for bias yourself, in pilots of the tool and throughout its use: to assess any potential bias arising from the tools used in relation to the actual hiring process, consider running tests of the system using existing applications on file from candidates with a diverse range of backgrounds. Note that: Collecting and analysing demographic data for the purpose of testing fairness is permitted within UK data protection law where explicit consent has been provided (see 2.3). Statistical analysis may not reveal all forms of discrimination. Please see the box below for more information. |
| An Equality Impact Assessment may be a useful tool for considering the important issues relating to discrimination and is strongly recommended for tools that are likely to have a significant impact on decision making, particularly where a large number of applicants are involved. |
This rule is used to evidence the presence of “adverse impact” in the United States. Adverse impact is present where a selection rate for a protected group is less than ⅘ (80%) of that of the group with the highest selection rate. This is not a test that is applied in law in the United Kingdom where a smaller but persistent degree of adverse impact may also need to be justified.
While this may be helpful context, this standard doesn’t hold the same legal weight in England and Wales. A tool might be lawful in the US but lead to discrimination in the UK.
Section 159 of the Equality Act 2010 allows an employer to treat an applicant with a protected characteristic more favourably in a hiring decision than someone without that characteristic, as long as they are equally qualified for the role.
This mechanism can be used to improve the diversity of an organisation and remedy underrepresentation in senior roles.
Statistical testing for bias
Statistical testing is a recommended part of ensuring data-driven tools are non-discriminatory. This testing is designed to highlight disparity between groups and therefore concerns about the tool, it is not designed to guide decision making about individual applicants.
The testing works by setting a baseline for your organisation and then testing the tool against this baseline to assure you that the tool is operating as it should, or to alert you of any issues. This testing should be conducted at the pilot stage and monitored throughout the lifecycle of the tool (note that this will require sustainable internal resource).
1. Articulate your organisation’s baseline: before introducing a data-driven tool, you should produce a baseline of progression for applicants from groups with different characteristics. [If there is enough data to draw meaningful conclusions, use intersectional categories, such as “Asian women” or “women above age 50.”] To start this process, you will need to identify the range of characteristics that are relevant in your context (you will likely be drawing from the nine protected characteristics). The information you could consider in developing your baseline includes:
a. The progression of applicants in previous recruitment rounds, where you are not using the tool.
b. If data on previous recruitment rounds is unavailable, consider ways to identify a baseline such as running data collection on a new recruitment round.
c. In addition to your baseline from previous recruitment rounds, you may wish to consider a number of factors which set a higher bar for combating bias in recruitment; this is particularly important if your organisation is not demographically diverse. Factors include:
● The demography of the population you are recruiting from (e.g. the London workforce).
● The industry baseline.
Note: you will need to consider which of these data sets are accessible to you and adjust your baseline accordingly.
2. Measure the impact of the tool in new recruitment rounds: conduct the same analysis (sift, interview, offer) to understand the trends for the same protected characteristics whilst using the tool.
3. Evaluate: compare how different demographic groups progress when the recruitment tool is used in comparison to the position previously. If the analysis shows worrying trends, for example fewer women are now progressing, then this should alert you to a problem with the new tool which requires assessment to ensure that no discrimination is taking place. Alternatively, if the analysis shows parity of outcomes for different demographic groups, this is evidence that the tool is functioning appropriately.
| Check the accessibility of the tool during purchasing: |
|
Seek information from the vendor: Recruiters and employers should seek to understand and anticipate the effect of these tools on candidates with disabilities and older people. Vendors may not have quantitative information or statistics about the effect of tools on people with disabilities given the wide variety of disabilities (see box on engagement with representative groups below). |
|
Before using the tool: |
|
Conduct analysis and directly engage with groups that represent the interests of people with disabilities and older people. This could include discussion with trade union equalities officers. It may be difficult to collect enough data on accessibility for effective statistical analysis so discussing the tool with groups that represent the interests of people with disabilities or older people may give greater insight on potential risks. |
| During and after using the tool: |
| Reasonable adjustments and alternatives: be prepared to offer candidates reasonable adjustments or an alternative evaluation process. If offering an alternative process, make sure that the process is as robust as the data-driven approach, and that there is no bias against candidates who use the alternative process. Explain the reasonable adjustments clearly and transparently. |
| Test impact: once a tool is in use, monitor its impact in practice. Create a channel for feedback from individuals about their experiences from an accessibility perspective in both the data-driven and alternative processes. |
| Reflect on results: note that introducing tools which require people to request adjustments can introduce unfairness simply because candidates with disabilities may not want to declare those disabilities for fear of stigma or disadvantage, or may not understand enough about the system in order to understand whether it places them at a disadvantage such that they need adjustments. Consider whether this will apply to your workplace context (or engage with your client to understand). |
Data-driven tools in recruitment are very likely to require personal data and therefore engage data protection law. In addition, as noted above in the discrimination section, in order to monitor the hiring system’s effect on protected groups, it will be necessary for the employer (or the vendor) to collect and analyse protected characteristics data on candidates, which is lawful where appropriate safeguards are in place.
| Check for compliance with data protection law before using the tool: |
| Agree who has the role of data controller (vendor or recruiter or both) before a procurement contract is signed - controllers will be responsible for complying and demonstrating compliance with UK GDPR. |
| With the vendor, clarify how candidates’ data will be used and that you have a lawful basis for collecting their data. |
|
Ensure that the vendor’s practices comply with UK GDPR requirements, paying particular attention to the guidance that applies to special category data. This includes: Understanding how your candidates’ data will be used in the future. For example, if it is used to train the vendor’s model, consent will be needed from candidates before they interact with the tool. You could consider whether there may be appropriate use of data minimisation (where only personal data needed for your purpose is collected), de-identification techniques (where identifying data is removed), and privacy enhancing technologies. Data collection should be adequate, relevant, and limited to what is necessary. See ICO anonymisation Code of Practice. Ensuring robust processes are in place for any data storage and mechanisms are in place to delete data. |
|
Complete a Data Protection Impact Assessment (DPIA) - you may want to complete this with the vendor. Beginning the DPIA as early as possible is advisable to assess and mitigate the data protection risks at an early stage and keep them updated throughout. A DPIA must be done before any type of processing that is “likely to result in a high risk”. More information on conducting DPIAs can be found on the ICO website here. It is good practice to publish the DPIA so it is accessible to applicants and workers. You may also consider completing an Equality Impact Assessment (see 2.2.) alongside or combined with a DPIA. |
|
Where protected characteristic data is being collected for equalities and fairness monitoring, ensure that: Candidates are given appropriate notice, are provided with a clear explanation of how data will be used and are also given an option to opt out; and This data is not accidentally “leaking” into the decision making process; it is being kept ring fenced and separate. |
Personal data is information that relates to an identified or identifiable individual - This could be as simple as a name or a number, or could include other identifiers such as an IP address or cookie identifiers.
Special category data is personal data that needs more protection because it is sensitive - Special category data includes personal data which reveals racial or ethnic origin; political opinions; religious or philosophical beliefs as well as data concerning health, a person’s sex life and their sexual orientation, amongst other things. [Article 9 of UK GDPR].
|
Build transparency into the process before using the tool: |
|
Understand whether the tool is making decisions based solely on automated processing (whether Article 22 applies): have a good understanding of how the data-driven system works internally (within your organisation), on what basis decisions are made and the level of human oversight. If this provision does apply, ensure you have a lawful basis and provide routes for human intervention, the individual to express their point of view, and human review where requested. Take appropriate measures to ensure that GDPR duties are met, including conducting a Data Protection Impact Assessment (DPIA). It is good practice to check that the vendor has also undertaken a DPIA of the tool. More information on fulfilling Article 22 duties can be found on the ICO website here. |
|
During and after using the tool: |
|
Regardless of whether the process is solely automated, good practice includes many of the following steps, such as: Explain that an automated tool is being used and create a clear appeals process, including a point of contact for human intervention and review. Prepare to explain the factors that went into data-driven decisions, and to provide documentation from the vendor about the process. Consider providing a non-automated alternative to the tool with human review. Where feasible, run a dip sample of applications examined by the two systems to ensure equivalence. Communicate the broad criteria for success and basis of decisions clearly and meaningfully to candidates. Ensure that vacancy holders are prepared to communicate decisions if need be. |
There are legal requirements around transparency set out in data protection law about the use of AI in decision making (as well as wider use of candidate data) which must be complied with. In addition, both candidates and recruiters may be nervous of the “dehumanisation” of the process: clear communication and transparency may help to mitigate some of these concerns (explored further in Building Trust below).
When using a data-driven tool in hiring, it can be difficult for vacancy holders to understand why decisions are made and to communicate this process with candidates. Similar to traditional hiring processes, data-driven systems may use factors that are unknown to vacancy holders or candidates. Communicating effectively and proactively about how data-driven tools are used can improve trust in these systems and the organisations that use them.
Even if in practice a data-driven tool is more standardised and tested than traditional methods, technological tools can elicit public trust concerns about fairness, transparency, and privacy, as well as how to navigate online systems.
| Before using the tool: |
| Discuss these issues with your clients: if you are a recruitment agency, raise the above risks and mitigations with your clients. |
| Consult with groups that represent marginalised individuals in deciding whether to adopt tools. For example, run challenge panels with diversity and inclusion networks, trade union representatives, or interested staff. |
| If you are an REC member, ensure the tool complies with the REC Code of Practice. |
| At the point of application: |
| Communicate relevant aspects of how the system works in plain language to candidates, explaining how they can prepare for success. |
| Provide contact information for candidates should they face technical issues: where the tool is making a decision about the applicant, ensure candidates have access to contact details in case the automated system does not work as intended. |
| Provide an appeals mechanism with a human reviewing the decisions made. |
| Following the decision-making process: |
| Explain how decisions about candidates have been made to them. |
|
Create channels to solicit feedback from candidates and vacancy about their experience. You may also consider having systems for redress that allow candidates to challenge decisions (see transparency and due process above). |
| During and after use of the tool: |
|
You could consider publishing documentation around the use of the tool to proactively communicate its purpose, risk mitigations in place and to build trust. This might include:
|
Share this article