Managing information under uncertainty - from human judgment to AI-mediated cognition

Talk given for the Information Quality Talk Series

This talk examines how people form and revise judgments under uncertainty. People rarely have direct access to the state of the world they are trying to infer and must instead rely on incomplete and noisy evidence. How difficult this is depends on the structure of the information environment: some environments provide stable and informative cues, while others are less tractable, socially generated, or mediated through symbolic information. Across these settings, people must not only estimate what is likely to be true, but also how much confidence to place in that estimate.

Drawing on work in metacognition, information seeking, and social decision-making, I begin with perceptual decision-making, where people accumulate evidence and use confidence to regulate subsequent behavior. I then examine what changes when judgments concern other people or depend on mediated information. These settings introduce uncertainty about intentions, preferences, sources, framing, and missing information. As the inference problem becomes more difficult, accuracy declines and confidence becomes less diagnostic of accuracy. People also seek and share information according to practical, emotional, and social goals, rather than accuracy alone.

I will conclude by examining how AI alters the relationship between people and their information environments. AI systems now take on more of the work involved in searching, integrating evidence, and supporting decisions, and are approaching expert human performance on forecasting benchmarks. This may improve judgment while also changing which cognitive operations people perform themselves. I will distinguish current evidence on skill loss under offloading from hypotheses about longer-term consequences for learning, expertise, and metacognition.

Open or download the talk PDF