guest ::
| Home > Publications database > Hallucination, justification, and the role of generative AI in science |
| Home > Publications database > Hallucination, justification, and the role of generative AI in science |
| Talk (non-conference) (Invited) | FZJ-2024-05900 |
2024
Abstract: Generative AI models are now being used to create synthetic climate data to improve the accuracy of climate models, and to construct virtual molecules which can then be synthesized for medical applications. But generative AI models are also notorious for their disposition to “hallucinate.” A recent Nature editorial defines hallucination as a process in which a generative model “makes up incorrect answers” (Jones, 2024). This raises an obvious puzzle. If generative models are prone to fabricating incorrect answers, how can they be used responsibly? In this talk I provide an analysis of the phenomenon of hallucination, and give special attention to diffusion models trained on scientific data (rather than transformers trained on natural language.) The goal of the paper is to work out how generative AI can be made compatible with reliabilist epistemology. I draw a distinction between parameter-space and feature-space deviations from the training data, and argue that hallucination is a subset of the latter. This allows us to recognize a class of cases in which the threat of hallucination simply does not arise. Among the remaining cases, I draw an additional distinction between deviations that are discoverable by algorithmic means, and those that are not. I then argue that if a deviation is discoverable by algorithmic means, reliability is not threatened, and that if the deviation is not so discoverable, then the generative model that produced it will be relevantly similar to other discovery procedures, and can therefore be accommodated within the reliabilist framework.
|
The record appears in these collections: |