Algorithms Used by Government to Determine Financial Assistance Exclude Qualified People, Human Rights Watch Investigation Finds

Tuesday - June 13 2023, 10:53 UTC - 1 year ago

An algorithm funded by the World Bank to determine which families should get financial assistance in Jordan likely excludes people who should qualify, according to an investigation published by Human Rights Watch. The algorithmic system is called Takaful and it rank families from least poor to poorest using a top-secret calculus. Takaful has cost over $1 billion, and is being used in 8 other countries in the Middle East and Africa. Human Rights Watch identified several fundamental problems with the algorithm that resulted in bias and inaccuracies, as well as reinforces existing gender-based discrimination. The onus is on the government of Jordan to provide transparency into the algorithm’s decisions, according to the report's authors.

An algorithm funded by the World Bank to determine which families should get financial assistance in Jordan likely excludes people who should qualify, according to an investigation published this morning by Human Rights Watch. The algorithmic system, called Takaful, ranks families applying for aid from least poor to poorest using a secret calculus that assigns weights to 57 socioeconomic indicators. Applicants say that the calculus is not reflective of reality, however, and oversimplifies people’s economic situation, sometimes inaccurately or unfairly. Takaful has cost over $1 billion, and the World Bank is funding similar projects in eight other countries in the Middle East and Africa.

Human Rights Watch identified several fundamental problems with the algorithmic system that resulted in bias and inaccuracies. Applicants are asked how much water and electricity they consume, for example, as two of the indicators that feed into the ranking system. The report’s authors conclude that these are not necessarily reliable indicators of poverty. Some families interviewed believed the fact that they owned a car affected their ranking, even if the car was old and necessary for transportation to work.

The report reads, "This veneer of statistical objectivity masks a more complicated reality: the economic pressures that people endure and the ways they struggle to get by are frequently invisible to the algorithm." "The questions asked don’t reflect the reality we exist in," says Abdelhamad, a father of two who makes 250 dinars ($353) a month and struggles to make ends meet, as quoted in the report.

Takaful also reinforces existing gender-based discrimination by relying on sexist legal codes. The cash assistance is provided to Jordanian citizens only, and one indicator the algorithm takes into account is the size of a household. Although Jordanian men who marry a noncitizen can pass on citizenship to their spouse, Jordanian women who do so cannot. For such women, this results in a lower reportable household size, making them less likely to receive assistance.

The report is based on 70 interviews conducted by Human Rights Watch over the last two years, not a quantitative assessment, because the World Bank and the government of Jordan have not publicly disclosed the list of 57 indicators, a breakdown of how the indicators are weighted, or comprehensive data about the algorithm’s decisions. The World Bank has not yet replied to our request for comment.

Amos Toh, an AI and human rights researcher for Human Rights Watch and an author of the report, says the findings point to the necessity of greater transparency into government programs that use algorithmic decision-making. Many of the families interviewed expressed distrust and confusion about the ranking methodology. "The onus is on the government of Jordan to provide that transparency," Toh says.

"It seems like this is yet another example of a bad design that actually ends up restricting access to funds for people who need them the most," she says.

The World Bank funded the program, which is managed by Jordan’s National Aid Fund, a social protection agency of the government.