New NYC Law Requires Employers To Prove Their Hiring Algorithms Are Bias-Free
New York has become the first city to pass legislation that regulates AI-driven hiring practices.
Based on the New York City Local Law 144, also known as the anti-bias law, the new legislation requires all NYC businesses to use AI tools in their hiring processes to prove their selections are free from sexism and racism.
Local Law 144
New York employers using AI algorithms in their hiring processes must submit used algorithms for an independent audit while publicizing the results.
The companies also have to share an ‘average score’ of candidates of different races, ethnicities, and genders in the form of a score, classification, or recommendation.
Company reports also must list the algorithm’s impact ratios, defined as the average algorithm-given score of all people in a specific category (e.g., Black candidates) divided by the average score of people in the highest-scoring class.
If a company does not comply with Local Law 144’s requirements, it will face financial penalties of up to $1,500 per day per tool, increasing with each violation.
The law protects any person performing or applying for a job in the city, regardless if they are based in New York or elsewhere.
The Concerns with AI Hiring
With nearly one in four organizations using AI for their hiring processes and several reports finding them discriminatory, it has raised concerns among many.
“We’re not yet at a place where algorithms can or should be trusted to make these decisions on their own without mirroring and perpetuating biases that already exist in the world of work,” Khyati Sundaram, CEO of Applied, a recruitment tech vendor, told TechCrunch.
In 2018, Reuters reported that Amazon removed a recruiting engine after finding it discriminated against women candidates. Just a year later, academic studies showed AI-enabled anti-Black bias in recruiting.
Hiring algorithms have additionally been found to assign job candidates a range of scores based on criteria such as wearing glasses or a headscarf and have penalized those with a Black-sounding name and physical disabilities.
What forms of algorithms are being used?
Hiring algorithms include text analyzers that sort resumes and cover letters based on keyboards.
However, there are now chatbots that conduct online interviews that automatically screen out applicants with specific traits and interviewing software designed to predict a candidate’s problem-solving skills, aptitudes, and “cultural fit” from their speech patterns and facial expressions.
A study by Pew Research Center last year found that two-thirds of Americans wouldn’t apply to employers who use AI in hiring, with 20% of those being Black people – the highest among all racial groups.
Although some respondents agreed with AI in recruitment to make it more fair and balanced, Monica Anderson, Associate Director of Research at Pew Research, told Capital B that some were skeptical.
“People who wouldn’t want to apply for a job using AI talked about how biases are already programmed into AI… and that AI cannot be the thing that solves things like racism or sexism because it just reflects what’s already a part of our communities.”
Is Local Law 144 Enough For Change?
The New York branch of the American Civil Liberties Union (NYCLU) claims that the law falls far short of providing protections for candidates and workers.
“AI is evolving more rapidly than laws can be passed to regulate its use,” Daniel Schwarz, Senior Policy and Technology Strategist at NYCLU, wrote in a policy memo.
“Laws that are eventually passed are likely to be hugely complicated for this reason. This will leave companies at risk of misinterpreting or overlooking various legal intricacies and, in turn, see marginalized candidates continue to be overlooked for roles.”
He noted that Local Law 144 could, as written, be understood to only cover a subset of hiring algorithms – for example, excluding tools that transcribe text from video and audio interviews.
Read: Is Buzzfeed Using AI To Develop Black, Asian, Latine Content In “Authentic Voice”?
Others have noted a worrying legal loophole: the rules are so narrow that they only apply to companies that rely on algorithms without human oversight.
However, many recruiters accept machine recommendations, so algorithmic hiring decisions become the default even when there is technically human oversight.
Rather than hoping regulators catch up and curb the worst excesses of recruitment AI, Schwarz said it’s down to employers to be vigilant and exercise caution when using AI in hiring processes.
However, critics have said Local Law 144 requires that audits be conducted by independent entities that haven’t been involved in using, developing, or distributing the algorithm they’re testing and don’t have a relationship with the company submitting the algorithm for testing.
What’s Next?
Other states, such as Washington DC, are considering a rule that will hold employers accountable for preventing bias in automated decision-making algorithms.
Two bills in California that aim to regulate AI in hiring were introduced within the last few years. In December last year, a bill was introduced in New Jersey to control the use of AI in hiring decisions to minimize discrimination.
