‘Dangerous future’ in NDIS tech experimentation

There is a lack of ethics and oversight in the government’s experimentation with new technologies through the National Disability Insurance Scheme (NDIS) that could lead to significant human rights violations, former NDIS Technology Authority chief Marie Johnson says.

In a submission to the Joint Standing Committee Inquiry into the Independent Assessments under the NDIS, Ms Johnson, who is now the chief executive of the Centre for Digital Business, pointed to “fundamental defects” in the National Disability Insurance Scheme (NDIS) technology systems and processes, and raised concerns about the potential to incorporate new technologies into this system.

Ms Johnson has direct experience of the NDIS on both sides. Ms Johnson’s husband has a disability, and daughter and grandsons access the NDIS.

“There would be perhaps few other NDIS applicants or families who would have the insight at the beginning of their journey to make such detailed documentary recordings from the outset,” Ms Johnson said in the submission.

The committee is examining the new Independent Assessments being used to decide what people’s needs are and how much funding individuals would receive through the NDIS, in the place of using doctors and medical reports.

There are concerns that this is a cost-cutting measure that may not give the full idea of an individual’s needs.

Ms Johnson labelled the new assessment process as “utterly flawed, unethical and dangerous on every level”, and that this is the direct result of a lack of an ethics framework at any level of the NDIS.

Ms Johnson also looked at experimental digital activities being conducted across government, including facial recognition through the digital identity scheme and the application of blockchain for NDIS payments.

“The [committee] needs to be alerted to the linkages between future blockchain and facial recognition applications as a means to control and monitor NDIS participants, and the risk that algorithms pose for people with disability in accessing services,” she said in the submission.

“The absence of an ethics and co-design framework exposes NDIS participants to potential human rights violations from these experimental whole-of-government digital activities.”

The Digital Transformation Agency has been leading a trial of using blockchain technology with the NDIS, along with the Commonwealth Bank.

The trial combined blockchain technology with the New Payments Platform to create smart money, which was programmed to know who it could be spent by, what it could be spent on and when it could be spent.

“This enables NDIS participants, service providers and the government to have real-time visibility of service eligibility, with appropriate access controls for privacy,” the DTA said.

NDIS participants have been invited to trial the initiative through a Smart Money prototype app.

But Ms Johnson said she is concerned about the on-the-ground use of the technology, associated civil liberties risk, and what happens when something goes wrong.

“Given the horrendously complex NDIS environment, defective processes and vulnerable people, there needs to be considerable caution in the application of blockchain technology. Blockchain in itself – as with other technology innovations – does not address fundamental design and human rights issues. Ethics is paramount,” she said.

“The involvement of the Commonwealth Bank itself raises further ethics issues, given the value of participant data, the size of the market, and the yet to be realised market honeypot of data, funds and services.”

If such a technology was implemented more widely with the NDIS, transparency would be greatly reduced for participants already subject to the new Independent Assessments, she said.

“So, in a not-too-distant future scenario, a participant would be served a robo-plan, arising from an Independent Assessment, with the robo-plan services transacted using blockchain programmable ‘smart money’,” Ms Johnson said.

“Far from participant choice and control, there would be no transparency to the Independent Assessment, no transparency to the robo-plan algorithms or rules, and no transparency as to the blockchain algorithms.

“Whilst there is no transparency or effective appeal rights for the participant, the system would achieve real-time and pre-emptive life-long monitoring and control of NDIS participants by the government.

“Whether by intention or inadvertence, this is a dangerous future emerging without governance or ethics.”

A spokesperson for the National Disability Insurance Agency said there are “no plans” to introduce blockchain for NDIS payments, and that the trial was conducted by the government in 2018 independent of the agency.

The government is also planning to introduce facial verification technology into its digital identity offering, myGovID, with UK tech firm iProov recently awarded a near-$11 million, three-year contract to provide a “liveliness” solution.

While this is not directly related to the NDIS yet, Ms Johnson said its use has “significant human rights implications for people with disability”.

“The committee needs to be alerted to the linkages between future blockchain and facial recognition applications as a means to control and monitor NDIS participants, and the risk that algorithms pose for people with disability in accessing services,” she said.

Invitations were sent to NDIS participants inviting them to take part in the trials of the new Independent Assessments, including language such as “exclusive invitation”.

This saw participants being “lured” into using the new scheme through similar tactics to identity and financial schemes, Ms Johnson said.

“It is utterly unethical and verging on maladministration that this type of communication is used at all, let alone in communications targeted at people with disability, and which would include vulnerable people with psychosocial disability,” she said.

“This is a direct consequence of the lack of an ethics framework and lack of ethics oversight more broadly.”

Technology should not be used at the expense of people with disabilities, she said.

“Australian civil society must not tolerate the actions of government that forcibly and arbitrarily subject people with disability to lifelong examination, study and monitoring. History is a reminder of where these actions can lead,” Ms Johnson said.

“That this control of people with disability will be effected through technologies such as biometrics, algorithms and blockchain is anathema to a harmonious and inclusive civil society and the human rights of all people.”

This story has been edited to include a response from the National Disability Insurance Agency:

“We reject the premise that NDIS participants are being “lured” into taking part in the ongoing Independent Assessment pilot program. The pilot aims to better understand how independent assessments will work across a range of participants, and it is optional as to whether someone wishes to take part.

“The program is one of the ways we are working with participants and their families to better understand how independent assessments will work in practice. We encourage all who take part to provide us with detailed feedback of their experience.

“The NDIA also rejects any notion that it does not engage in co-design. The NDIA undertakes significant co-design, engaging a diverse set of participants for input on accessibility, usability, and desirability.  This includes both future participant-facing technology platforms, and service design on how the NDIA interacts with participants.”

Do you know more? Contact James Riley via Email.