Software prototypes for integration and analysis of multi-modal psychophysiological data

Task 6.2 (MS17) aims to create prototype software which will integrate and synchronize psychophysiological data acquired using the protocols developed in Task 6.1. In order to optimize the possibilities for multimodal data analysis, it is necessary that data streams in an experiment are integrated and aligned with respect to time. The software will be based on current software from the partner Noldus, considerably improved with respect to performance (to process big data), security (GDPR compliance) and analysis. Ideally, panel data and self-report measures will also be integrated. Sensor integration will be developed in an open way, to maximise the number of IoT and other sensors (eye trackers, physiology acquisition, wearables, etc) which can be used with the system. Sub-projects will be selected provide datasets labelled with relevant high-level mental constructs such as confusion, stress, pleasure, and these datasets will be used to train AI/machine learning based analysis modules.