Generative artificial intelligence tools like ChatGPT, an AI-powered language tool, and Midjourney, an AI-powered image generator, can potentially assist people with various disabilities. These tools could summarize content, compose messages or describe images. Yet the degree of this potential is an open question, since, in addition to regularly and , these tools can .
This year, seven researchers at the ĢĒŠÄŌ““ conducted a three-month autoethnographic study ā drawing on their own experiences as people with and without disabilities ā to test AI toolsā utility for accessibility. Though researchers found cases in which the tools were helpful, they also found significant problems with AI tools in most use cases, whether they were generating images, writing Slack messages, summarizing writing or trying to improve the accessibility of documents.
The team presented Oct. 22 at the conference in New York.
āWhen technology changes rapidly, thereās always a risk that disabled people get left behind,ā said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. āI’m a really strong believer in the value of first-person accounts to help us understand things. Because our group had a large number of folks who could experience AI as disabled people and see what worked and what didn’t, we thought we had a unique opportunity to tell a story and learn about this.ā
The group presented its research in seven vignettes, often amalgamating experiences into single accounts to preserve anonymity. For instance, in the first account, āMia,ā who has intermittent brain fog, deployed ChatPDF.com, which summarizes PDFs, to help with work. While the tool was occasionally accurate, it often gave ācompletely incorrect answers.ā In one case, the tool was both inaccurate and ableist, changing a paperās argument to sound like researchers should talk to caregivers instead of to chronically ill people. āMiaā was able to catch this, since the researcher knew the paper well, but Mankoff said such subtle errors are some of the āmost insidiousā problems with using AI, since they can easily go unnoticed.
Yet in the same vignette, āMiaā used chatbots to create and format references for a paper they were working on while experiencing brain fog. The AI models still made mistakes, but the technology proved useful in this case.
Mankoff, whoās spoken publicly about having Lyme disease, contributed to this account. āUsing AI for this task still required work, but it lessened the cognitive load. By switching from a āgenerationā task to a āverificationā task, I was able to avoid some of the accessibility issues I was facing,ā Mankoff said.
The results of the other tests researchers selected were equally mixed:
- One author, who is autistic, found AI helped to write Slack messages at work without spending too much time troubling over the wording. Peers found the messages ārobotic,ā yet the tool still made the author feel more confident in these interactions.
- Three authors tried using AI tools to increase the accessibility of content such as tables for a research paper or a slideshow for a class. The AI programs were able to state accessibility rules but couldnāt apply them consistently when creating content.
- Image-generating AI tools helped an author with (an inability to visualize) interpret imagery from books. Yet when they used the AI tool to create an illustration of āpeople with a variety of disabilities looking happy but not at a party,ā the program could conjure only fraught images of people at a party that included ableist incongruities, such as a disembodied hand resting on a disembodied prosthetic leg.
āI was surprised at just how dramatically the results and outcomes varied, depending on the task,ā said lead author , a UW doctoral student in the Allen School. āIn some cases, such as creating a picture of people with disabilities looking happy, even with specific prompting ā can you make it this way? ā the results didnāt achieve what the authors wanted.ā
The researchers note that more work is needed to develop solutions to problems the study revealed. One particularly complex problem involves developing new ways for people with disabilities to validate the products of AI tools, because in many cases when AI is used for accessibility, either the source document or the AI-generated result is inaccessible. This happened in the ableist summary ChatPDF gave āMiaā and when āJay,ā who is legally blind, used an AI tool to generate code for a data visualization. He could not verify the result himself, but a colleague said it ādidnāt make any sense at all.āĀ The frequency of AI-caused errors, Mankoff said, āmakes research into accessible validation especially important.ā
Mankoff also plans to research ways to document the kinds of ableism and inaccessibility present in AI-generated content, as well as investigate problems in other areas, such as AI-written code.
āWhenever software engineering practices change, there is a risk that apps and websites become less accessible if good defaults are not in place,ā Glazko said. āFor example, if AI-generated code were accessible by default, this could help developers to learn about and improve the accessibility of their apps and websites.ā
Co-authors on this paper are , who completed this research as a UW postdoctoral scholar in the Allen School and is now at Rice University; , and , all UW doctoral students in the Allen School; and , who completed this work as a UW doctoral student in the Information School and is now at the Massachusetts Institute of Technology. This research was funded by Meta, (CREATE), Google, an NIDILRR ARRT grant and the National Science Foundation.
For more information, contact Glazko at glazko@cs.washington.edu and Mankoff at jmankoff@cs.washington.edu.