Be careful about ID verification as a solution to applicant fraud

For a few years now I've been hearing VidCruiter's clients express worry about various forms of applicant fraud, and observed vendors selling various conceptions of the problem that they in turn try to mitigate. But this year there is a real explosion of both supply and demand for identity verification solutions, and I've seen a lot of vendors who come from financial and law enforcement contexts entering the recruiting market, as well as existing HR tech vendors adding in-house capabilities, and many startups.

We all know about the risk of algorithmic bias in hiring, that models inherit from their training data (maybe the hiring patterns were biased, or the data is incomplete, etc) and their model design (maybe they make unfair inferences from the school you attended, or disadvantage accented speech, etc). One of the underlying worries is that these models often end up being shared across organizations by way of the hiring tools they use. Vendor X might provide tools to thousands of companies, and if their tool stacks the deck against you, it's harder to get into all of those companies. The AI ethics literature has named this the risk of an "algorithmic monoculture" and the harm as "outcome homogenization". The mechanism is the shared model.

ID verification systems can inadvertently introduce a similar pattern, but instead of a shared model it is shared state that can systematically disadvantage an applicant. If these tools are wrong about you, and maintain and re-apply a record, that mistake follows you. Some vendors describe identity networks that persist across customers and learn from customers' fraud outcomes. It sounds like a good idea, and it's the same underlying tech used to combat things like online advertising click fraud, by trying to verify that the long tail of behaviour appears "authentic". Signals such as how long you've had a phone number, if you used a VPN, if you were able to pass a video "liveness" check to verify you are a real person, even the browser extensions you have installed, can all bear on the contextual factors that influence these scores.

The problem is that if the system is flawed, if it's bad scanning faces for some groups of people, or understanding certain documents you provide photos of, or over-weights lossy signals like VPN usage or flags legitimate assistive technology as suspicious, that error lives with the verification state, which in turn can be shared across thousands of companies. If the verification is done early in the screening process, instead of at offer time, this can become a systemic barrier to employment. It also looks a lot like a hidden consumer reporting agency, and the sort of thing the FCRA regulates.

This difference between shared state and shared models is important. A shared model repeats a mistake about people like you. Shared state repeats a mistake about you. A model can be audited, even with synthetic data, but an audit can't find wrong records for specific persons, for that you need access and a way to contest it. And finally, shared state is invisible from the client side, who typically get limited information out of the system, and even more limited information about how it works. Fairness in hiring requires that rejections are independent, but shared state removes that independence.

Sound familiar? This is a lot like a consumer credit report. And this kind of worry is the reason why regulations like the Fair Credit Reporting Act (FCRA) were developed to protect consumers from the persistence of errors that systematically would deny them access to credit.

I'm working on a larger piece about this, but the reason I put this post together is that there are some things employers need to consider as they increasingly rush to implement identity verification as part of their effort to combat applicant volume, misrepresentation, synthetic video/audio, and proxy applicants.