AVISSO | Audiovisual Speech Segmentation and Oscillations

Summary
Daily social tasks (e.g. meetings, shopping) rely on human communication with speech. Auditory only studies established that continuous speech segmentation relies on the temporal integration of rhythmic acoustic features occurring at delta-theta rates in the signal, via a possible mechanism of entrainment coupled with beta activity. In terms of neural correlates, the timing network (including the basal ganglia, supplementary motor area and cerebellum) may actively contribute to the temporal integration of auditory signal during continuous speech segmentation. However, speech is often multimodal and listeners need to integrate speaker’s body movements aligned to rhythmic acoustic features. Consequently, AV speech may generate different rhythmic patterns than auditory only speech. How the brain translates naturally concomitant AV rhythms to facilitate speech segmentation clearly remains to be addressed. The goal of this project is to investigate the oscillatory patterns and neural correlates of AV rhythms resulting from the temporal integration of body movements and salient acoustic features in speech. In a first EEG experiment, I will compare the modulations of delta-theta entrainment and coupling with beta activity, depending on the relationship between visual and auditory information (congruent, incongruent and auditory only). Secondly, in an fMRI version of the experiment, I will explore the contribution of the timing network during AV speech segmentation, and its different patterns of activations across speech conditions. Finally, to test whether the brain areas revealed by fMRI have a necessary role for the successful AV speech segmentation, I will run the same fMRI experiment with Parkinson’s disease patients with dysfunctional basal ganglia. Looking at the timing network patterns of activations will also reveal if correlated body information eventually help PD to compensate basal ganglia deficit with greater contribution of alternative path (i.e. cerebellum).
Unfold all
/
Fold all
More information & hyperlinks
Web resources: https://cordis.europa.eu/project/id/707727
Start date: 02-05-2016
End date: 01-05-2018
Total budget - Public funding: 165 598,80 Euro - 165 598,00 Euro
Cordis data

Original description

Daily social tasks (e.g. meetings, shopping) rely on human communication with speech. Auditory only studies established that continuous speech segmentation relies on the temporal integration of rhythmic acoustic features occurring at delta-theta rates in the signal, via a possible mechanism of entrainment coupled with beta activity. In terms of neural correlates, the timing network (including the basal ganglia, supplementary motor area and cerebellum) may actively contribute to the temporal integration of auditory signal during continuous speech segmentation. However, speech is often multimodal and listeners need to integrate speaker’s body movements aligned to rhythmic acoustic features. Consequently, AV speech may generate different rhythmic patterns than auditory only speech. How the brain translates naturally concomitant AV rhythms to facilitate speech segmentation clearly remains to be addressed. The goal of this project is to investigate the oscillatory patterns and neural correlates of AV rhythms resulting from the temporal integration of body movements and salient acoustic features in speech. In a first EEG experiment, I will compare the modulations of delta-theta entrainment and coupling with beta activity, depending on the relationship between visual and auditory information (congruent, incongruent and auditory only). Secondly, in an fMRI version of the experiment, I will explore the contribution of the timing network during AV speech segmentation, and its different patterns of activations across speech conditions. Finally, to test whether the brain areas revealed by fMRI have a necessary role for the successful AV speech segmentation, I will run the same fMRI experiment with Parkinson’s disease patients with dysfunctional basal ganglia. Looking at the timing network patterns of activations will also reveal if correlated body information eventually help PD to compensate basal ganglia deficit with greater contribution of alternative path (i.e. cerebellum).

Status

CLOSED

Call topic

MSCA-IF-2015-EF

Update Date

28-04-2024
Images
No images available.
Geographical location(s)
Structured mapping
Unfold all
/
Fold all
Horizon 2020
H2020-EU.1. EXCELLENT SCIENCE
H2020-EU.1.3. EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions (MSCA)
H2020-EU.1.3.2. Nurturing excellence by means of cross-border and cross-sector mobility
H2020-MSCA-IF-2015
MSCA-IF-2015-EF Marie Skłodowska-Curie Individual Fellowships (IF-EF)