VisualNarrative | From Meaning to Perception: How narratives shape visual processing

Summary
Humans constantly generate expectations of what will happen in the future. Such expectations allow us to quickly absorb incoming data. For example, the visual system takes advantage of the world’s relative stability over space and time to anticipate upcoming input. However, most of what we currently know about predictive visual processing comes from studies using arbitrary associations between simple stimuli, such as gratings or abstract objects, which do not reflect the richness of daily life events. It is largely unknown how more complex, meaning-based expectations influence visual processing. To overcome this barrier, the proposed project will use an ecologically valid paradigm, using picture books, and investigate how such more complex, narrative-based expectations shape visual processing. I hypothesise that narratives will elicit strong expectation signals that impact even the earliest levels of sensory processing. To test this hypothesis, I will first quantify narrative-based expectations derived from the picture stories using state-of-the-art computational modelling. Then, I will assess whether these expectations shape the processing in visual cortical regions of adult human volunteers, using neuroimaging. I will inspect whether such signals become anticipatory as viewers become familiar with the narrative – in line with the idea that upon construction of a world model, the brain predicts upcoming stimuli. Moreover, I will also investigate what brain regions shape such learning of narratives. In a separate eye-tracking experiment, I will explore how narratives influence gaze behaviour, providing another assay into how meaning percolates to visual processing. This project will allow me to conduct creative, original and important research, which will propel my scientific career. More broadly, these lines of inquiry will push the envelope of understanding of human perception and learning, bridging from higher-order cognition to low-level visual processing.
Unfold all
/
Fold all
More information & hyperlinks
Web resources: https://cordis.europa.eu/project/id/101106569
Start date: 01-11-2023
End date: 31-10-2025
Total budget - Public funding: - 187 624,00 Euro
Cordis data

Original description

Humans constantly generate expectations of what will happen in the future. Such expectations allow us to quickly absorb incoming data. For example, the visual system takes advantage of the world’s relative stability over space and time to anticipate upcoming input. However, most of what we currently know about predictive visual processing comes from studies using arbitrary associations between simple stimuli, such as gratings or abstract objects, which do not reflect the richness of daily life events. It is largely unknown how more complex, meaning-based expectations influence visual processing. To overcome this barrier, the proposed project will use an ecologically valid paradigm, using picture books, and investigate how such more complex, narrative-based expectations shape visual processing. I hypothesise that narratives will elicit strong expectation signals that impact even the earliest levels of sensory processing. To test this hypothesis, I will first quantify narrative-based expectations derived from the picture stories using state-of-the-art computational modelling. Then, I will assess whether these expectations shape the processing in visual cortical regions of adult human volunteers, using neuroimaging. I will inspect whether such signals become anticipatory as viewers become familiar with the narrative – in line with the idea that upon construction of a world model, the brain predicts upcoming stimuli. Moreover, I will also investigate what brain regions shape such learning of narratives. In a separate eye-tracking experiment, I will explore how narratives influence gaze behaviour, providing another assay into how meaning percolates to visual processing. This project will allow me to conduct creative, original and important research, which will propel my scientific career. More broadly, these lines of inquiry will push the envelope of understanding of human perception and learning, bridging from higher-order cognition to low-level visual processing.

Status

SIGNED

Call topic

HORIZON-MSCA-2022-PF-01-01

Update Date

31-07-2023
Images
No images available.
Geographical location(s)
Structured mapping
Unfold all
/
Fold all
Horizon Europe
HORIZON.1 Excellent Science
HORIZON.1.2 Marie Skłodowska-Curie Actions (MSCA)
HORIZON.1.2.0 Cross-cutting call topics
HORIZON-MSCA-2022-PF-01
HORIZON-MSCA-2022-PF-01-01 MSCA Postdoctoral Fellowships 2022