‘Citizen-centric’ demolished by NDIS algorithms

After more than twenty years of ‘citizen centric’ strategies by governments around the world, this week saw the demolition in Australia of any pretence of citizen-centric, participant-centric, person-centric – or any other version strategy based on ‘human’ centricity.

The theatre of citizen-centric has now run its course. And in the year 2021, citizen-centric has been replaced by the ungoverned rise of machine-centric algorithms.

Last week under questioning at the Joint Standing Committee on the NDIS Inquiry into Independent Assessments, the NDIA revealed for the first time the architecture of the algorithms to be used in robo-planning.

Also in evidence, commentators including myself have raised serious concerns on the application of algorithms, and the additive impact of algorithms, on people with a disability.

Lacking any transparency or ethics, the broad-brush description of the algorithm architecture involves the construction by the agency of 400 personas based on disability, age and other factors.

Would indigeneity, language, culturally and linguistically diverse (CALD) people and homelessness be factors? We don’t know. The determinants of the algorithm personas have not been subject to any independent external scrutiny.

The agency indicated that a technical paper would be released, and it is understood that the sector and academic and legal community will forensically examine it.

But given the timeframes promulgated by the government, any feedback is likely to be too late. The assumptions and determinants will have already been baked into the algorithm – or algorithms, as there will be more than one.

Algorithms that include debt recovery. And algorithms that include the extensive and yet to be legislated changes that the community only found out about because the draft legislation had been leaked.

And a cautionary word about ‘personas’.

Personas are typically the collaborative outcome of a co-design process, used to tell a story from the person’s perspective of their experience, with all the uncomfortable, confronting, and messy details of life. More controversially, they are often used in HCI design although with issues of stereotyping a major concern.

But the apparent use of personas in the construction of the NDIS robo-planning algorithms, is a perverted application of this method.

With the lack of transparency and ethics – and the absence of co-design – the algorithms become a construction of internal modelling where bias is inescapable.

Not only will bias be baked in, but the operation of these powerful algorithms is not subject to independent scrutiny or appeal.

And all this has arisen because the agency has created a seething mess of administrative and conceptual complexity that it cannot manage.

Sliding doors of complexity so unfathomable that the scheme itself has become manifestly unjust.

So their answer is to put over 530,000 Australians with disability into 400 persona boxes and wipe out any pretence of the notion “participant centric”.

We could see this coming over recent years when the word “co-design” disappeared from key corporate documents. I was criticised for pointing this out.

In reality, it matters not whether the number is 400 or 4,000 personas, as this approach treats Australians with disability as an amorphous data mass and not as individuals with rights, and with unique lives and needs.

And it is a harbinger of things to come for those on welfare, aged care and veterans.

Certainly data analytics is critical. And better and transparent data analytics is urgently needed.

But the political arguments around scheme sustainability not only ignore the benefits side of the equation, but completely ignore complexity as a most significant factor impacting sustainability.

The original 2011 Productivity Commission Report referred to complexity as “confusopoly”.

And an overwhelming number of the more than 260 submissions to the Inquiry on Independent Assessments describe the injustice and human rights impact of complexity on all communities, especially First Nations people, CALD communities, and people with psychosocial disability.

The Victorian Council of Social Service (VCOSS) Submission (number 153) for example describes the barriers of complexity, literacy and “magic words”.

The complexity sees documents and highly personal information, lost. This is the experience of a great many people evidenced in submissions to this Senate Inquiry and other inquiries including the Tune Review.

A recent poll on the NDIS Grassroots Facebook group (with 55,000 members), indicated as many as 80 per cent of people experience lost documents. This would be an enormous cost driver across the whole system.

Imagine if 80 per cent of people undergoing an Independent Assessment, had their documents lost. There is no guarantee that this will not happen.

So an end-to-end co-designed time-and-motion study would be immensely revealing. And this should be done before any structural changes to the scheme are made.

Automation and the application of algorithms, which will be inherently biased, do not overcome complexity nor the injustice, disadvantage and cost this causes.

To the contrary, there is international evidence that algorithms deployed by the state cause harm and compound injustice.

So, all up, it is not clear what ‘problem’ the personas are intended to address.

Except the effect of this is to digitally institutionalise people with disability into boxes, and to subject them to lifelong examination, study and control.

In response to the proposed changes that have been unanimously slammed by every sector of the community, Minister Reynolds challenged: “if somebody could come up with a better option…”

As if that challenge had not been well anticipated, in the more than 260 submissions to the Senate Inquiry into Independent Assessments.

People with disability, experts, health professionals, NGOs, service providers, state and territory governments, advocates: thousands of hours of documented evidence and insightful thoughtful solutions and recommendations.

Like many people, my family found the process of writing a submission detailing our harrowing experience, very painful.

But we, and hundreds of thousands of Australians have a vested interest in the sustainability of the NDIS.

And in stirring testimony at the Senate hearing, Mr Dougie Herd Chair of the ACT Disability Reference Group and one of the original campaigners for the NDIS, challenged back “…this is super personal.”

It is personal and yet we have been reduced to personas, components in algorithms.

The upending of the NDIS social contract has precipitated an insidious episode of servicing by algorithm, in which no Australian has had a say.

Until the government publishes the design of the algorithms and intended applications, their use must stop immediately.

Marie Johnson was the Chief Technology Architect of the Health and Human Services Access Card program; formerly Microsoft World Wide Executive Director Public Services and eGovernment; and former Head of the NDIS Technology Authority. Marie is an inaugural member of the ANU Cyber Institute Advisory Board.

Photo credit: NoDisadvantage Community on Facebook

Do you know more? Contact James Riley via Email.