Current Projects

SHIFT: MetamorphoSis of cultural Heritage Into augmented hypermedia assets For enhanced accessibiliTy and inclusion (#101060660)

EU Horizon 2020 Research & Innovation Action (RIA)
Runtime: 3 years
Role: Principal Investigator, Workpackage Leader, Co-Author Proposal
Partners:  Software Imagination & Vision, Foundation for Research and Technology, Massive Dynamic, Audeering, University of Augsburg, Queen Mary University of London, Magyar Nemzeti Múzeum – Semmelweis Orvostörténeti Múzeum, The National Association of Public Librarians and Libraries in Romania, Staatliche Museen zu Berlin – Preußischer Kulturbesitz, The Balkan Museum Network, Initiative For Heritage Conservation, Eticas Research and Consulting, German Federation of the Blind and Partially Sighted

The SHIFT project is strategically conceived to deliver a set of technological tools, loosely coupled that offers cultural heritage institutions the necessary impetus to stimulate growth, and embrace the latest innovations in artificial intelligence, machine learning, multi-modal data processing, digital content transformation methodologies, semantic representation, linguistic analysis of historical records, and the use of haptics interfaces to effectively and efficiently communicate new experiences to all citizens (including people with disabilities).

Machine Learning für Kameradaten mit unvollständiger Annotation

Industry Cooperation with BMW AG
Runtime: 01.01.2022 – 31.12.2023
Role: Principal Investigator
PartnersUniversity of Augsburg, BMW AG

The project aims at self-supervised and reinforced learning for analysis of camera data with incomplete annotation.

causAI: AI Interaktionsoptimierung bei Videoanrufen im Vertrieb (#03EGSBY853)

BMWi (Federal Ministry for Economic Affairs and Energy) EXIST Business Start-up Grant
Runtime: tba
Role: Mentor
Partners: University of Augsburg

causAI analysiert die Sprache, Gestik und Mimik von vertrieblichen Videoanrufen mithilfe von künstlicher Intelligenz, um die digitale Vertriebskompetenz zu verbessern. Ziel ist es, causAI als innovatives Softwareprodukt für Vertriebsgesprächsunterstützung und -schulung im Vertrieb zu etablieren.

MAIKI: Mobiler Alltagstherapieassistent mit interaktionsfokussierter künstlicher Intelligenz bei Depression

Runtime: 01.10.2021 – 31.12.2021
Role: Principal Investigator, Co-author Proposal
Partners: FlyingHealth Incubator GmbH, GET.ON Institut für Online Gesundheitstrainings GmbH, University of Augsburg

Dieses Vorhaben zielt auf einen mobilen digitalen Assistenten ab, der den Patienten interaktiv, intelligent und individualisiert darin unterstützt, seine Therapie im Alltag effektiver umzusetzen. Hierzu werden Methoden künstlicher Intelligenz (Interaktionsanalyse mit Stimmanalyse und Natural Language Processing, Artificial Empathy, Maschinelles Lernen) mit dem Ziel erforscht und entwickelt, die Patienten-Assistenten-Interaktion zu optimieren, therapeutische Interventionen fallspezifisch zu optimieren und deren Umsetzung interaktiv auf intelligente und zugleich personalisierte Art zu unterstützen. Dieser digitale mobile Therapiebegleiter geht über den derzeitigen Stand der Technik hinaus, da er a) eine dauerhafte behandlungsrelevante Kommunikation mit dem Betroffenen aufrechterhält (was bisher die face-to-face Psychotherapie nicht vermag) und b) seine Empfehlungen fortlaufend an aktuelles Erleben, Verhalten und bisherigem Therapieverlauf der Betroffenen anpasst. Auf Basis dieser digitalen Therapieindividualisierung soll der Gesundungsprozess beschleunigt und Rückfallquoten verringert werden.

Improving asthma care through personalised risk assessment and support from a conversational agent (#EP/W002477/1 )

EPSRC UK Research and Innovation fEC Grants
Runtime: 01.09.2021 – 31.08.2023
Role: Principal Investigator, Co-author Proposal
Partners: Imperial College London

Over 5.4 million people have asthma in the UK, and despite £1Billion a year in NHS spending on asthma treatment, the national mortality rate is the highest in Europe. One of the reasons for this statistic, is that risk is often dramatically underestimated by many with asthma. This leads to neglect of early care, poor control, and eventually, hospitalisation. Therefore, improving accurate risk assessment and reduction via relevant behaviour change among people with asthma could save lives and dramatically reduce health care costs. We aim to address this early-care gap by investigating a new type of low-cost, and scalable personalised risk assessment, combined with follow-up automated support for risk reduction. The technology will leverage artificial intelligence to calculate a personalised asthma risk score based on voice features and self-reported data. It will then provide personalised advice on actions that can be taken to lower risk followed by customised conversational guidance to support the process of healthy change. We envision our work will ultimately lead to a safe and engaging system where the patients are able to see their current risk of an asthma attack after answering a series of questions, akin to clinical history taking, and record their voice. They then get ongoing customised support from an automated coach on how to reduce that risk. Any progress they make will visibly lower their risk (presented, for example, as “Strengthening their shield”), in order to make their state of asthma control more tangible and motivating. The technology will be developed collaboratively with direct involvement from people with asthma and clinicians through co-design methods and regular feedback in order to ensure risk assessment, feedback and guidance are clinically sound, and delivered in a way that is autonomy-supportive, clear, useful, and engaging to patients.

Leader Humor: A Multimodal Approach to Humor Recognition and an Analysis of the Influence of Leader Humor on Team Performance in Major European Soccer Leagues (#SCHU2508/12-1)

(“Ein multimodaler Ansatz zur Erkennung und Messung von Humor und eine Analyse des Einflusses des Humors von Führungskräften auf die Teamleistung in europäischen Profifußball-Ligen”)
DFG (German Research Foundation) Project
Runtime: 36 Months
Role: Principal Investigator, Co-author Proposal
Partners: University of Passau, University of Augsburg

In this project, scholars active in the fields of management and computerized psychometry take the unique opportunity to join their respective perspectives and complementary capabilities to address the overarching question of “How, why, and under which circumstances does leader humor affect team processes and team performance, and how can (leader) humor be measured on a large scale by applying automatic multimodal recognition approaches?”. Trait humor, which is one of the most fundamental and complex phenomena in social psychology, has garnered increasing attention in management research. However, scholarly understanding of humor in organizations is still substantially limited, largely because research in this domain has primarily been qualitative, survey-based, and small scale. Notably, recent advances in computerized psychometry promise to provide unique tools to deliver unobtrusive, multi-faceted, ad hoc measures of humor that are free from the substantial limitations associated with traditional humor measures. Computerized psychometry scholars have long noted that a computerized understanding of humor is essential for the humanization of artificial intelligence. Yet, they have struggled to automatically identify, categorize, and reproduce humor. In particular, computerized approaches have suffered not only from a lack of theoretical foundations but also from a lack of complex, annotated, real-life data sets and multimodal measures that consider the multi- faceted, contextual nature of humor. We combine our areas of expertise to address these research gaps and complementary needs in our fields. Specifically, we substantially advance computerized measures of humor and provide a unique view into the contextualized implications of leader humor, drawing on the empirical context of professional soccer. Despite initial attempts to join computerized psychometry and management research, these two fields have not yet been successfully combined to address our overall research question. We aspire to fill this void as equal partners, united by our keen interest in humor, computerized psychometry, leader rhetoric, social evaluations, and team performance. 

AUDI0NOMOUS: Agentenbasierte, Interaktive, Tiefe 0-shot-learning-Netzwerke zur Optimierung von Ontologischem Klangverständnis in Maschinen
(Agent-based Unsupervised Deep Interactive 0-shot-learning Networks Optimising Machines’ Ontological Understanding of Sound) (# 442218748)
DFG (German Research Foundation) Reinhart Koselleck-Projekt
Runtime: 01.01.2021 – 31.12.2025
Role: Principal Investigator, Co-author Proposal
Partners: University of Augsburg

Soundscapes are a component of our everyday acoustic environment; we are always surrounded by sounds, we react to them, as well as creating them. While computer audition, the understanding of audio by machines, has primarily been driven through the analysis of speech, the understanding of soundscapes has received comparatively little attention. AUDI0NOMOUS, a long-term project based on artificial intelligent systems, aims to achieve a major breakthroughs in analysis, categorisation, and understanding of real-life soundscapes. A novel approach, based around the development of four highly cooperative and interactive intelligent agents, is proposed herein to achieve this highly ambitious goal. Each agent will autonomously infer a deep and holistic comprehension of sound.  A Curious Agent will collect unique data from web sources and social media; an Audio Decomposition Agent will decompose overlapped sounds; a Learning Agent will recognise an unlimited number of unlabelled sound; and, an Ontology Agent will translate the soundscapes into verbal ontologies. AUDI0NOMOUS will open up an entirely new dimension of comprehensive audio understanding; such knowledge will have a high and broad impact in disciplines of both the sciences and humanities, promoting advancements in health care, robotics, and smart devices and cities, amongst many others.

EASIER: Intelligent Automatic Sign Language Translation (#101016982)
EU Horizon 2020 Research & Innovation Action (RIA)

Runtime: 01.01.2021 – 31.12.2023
Role: Principal Investigator
Partners: Martel GmbH Martel, Athena Research & Innovation Center in Information Communication & Knowledge Technologies, Universität Hamburg, Radboud University, University of Surrey, University of Zurich, CNRS, DFKI, audEERING GmbH, nuromedia GmbH, Swiss TXT AG, European Union of the Deaf iVZW, SCOP Interpretis, University College London

EASIER aims to create a framework for barrier-free communication among deaf and hearing citizens across the EU by enabling users of European SLs to use their preferred language to interact with hearing individuals, via incorporation of state-of-the-art NMT technology that is capable of dealing with a wide range of languages and communication scenarios. To this end, it exploits a robust data-driven SL (video) recognition engine and utilizes a signing avatar engine that not only produces signing that is easy to comprehend by the deaf community but also integrates information on affective expressions and coherent prosody. The envisaged ecosystem will incorporate a robust translation service surrounded by numerous tools and services which will support equal participation of deaf individuals to the whole range of everyday-life activities within an inclusive community, and also accelerate the incorporation of less-resourced SLs into SL technologies, while it leverages the SL content creation industry. The deaf community is heavily involved in all project processes, while deaf researchers are among the staff members of all SL expert partners.

MARVEL: Multimodal Extreme Scale Data Analytics for Smart Cities Environments (#957337)
EU Horizon 2020 Research & Innovation Action (RIA)

Runtime: 01.01.2021 – 31.12.2023
Role: Principal Investigator
Partners: Idryma Technologies, Infineon, Aarhus University, Atos Spain, Consiglio Nazionale delle Ricerche, Intrasoft, FBK, audEERING GmbH, Tampereen Korkeakoulusaatio, Privanova, Sphynx Technology Solutions, Comune die Trento, Univerzitet u Novom Sadu Fakultet Tehnickih Nauka, Information Technology for Market Leadership, Greenroads Limited, Zelus Ike, Instytut Chemii Bio Organicnej Polskiej Akademii Nauk

The “Smart City” paradigm aims to support new forms of monitoring and managing of resources as well as to provide situational awareness in decision-making fulfilling the objective of servicing the citizen, while ensuring that it meets the needs of present and future generations with respect to economic, social and environmental aspects. Considering the city as a complex and dynamic system involving different interconnected spatial, social, economic, and physical processes subject to temporal changes and continually modified by human actions. Big Data, fog, and edge computing technologies have significant potential in various scenarios considering each city individual tactical strategy. However, one critical aspect is to encapsulate the complexity of a city and support accurate, cross-scale and in-time predictions based on the ubiquitous spatio-temporal data of high-volume, high-velocity and of high-variety.
To address this challenge, MARVEL delivers a disruptive Edge-to-Fog-to-Cloud ubiquitous computing framework that enables multi-modal perception and intelligence for audio-visual scene recognition, event detection in a smart city environment. MARVEL aims to collect, analyse and data mine multi-modal audio-visual data streams of a Smart City and help decision makers to improve the quality of life and services to the citizens without violating ethical and privacy limits in an AI-responsible manner. This is achieved via: (i) fusing large scale distributed multi-modal audio-visual data in real-time; (ii) achieving fast time-to-insights; (iii) supporting automated decision making at all levels of the E2F2C stack; and iv) delivering a personalized federated learning approach, where joint multi modal representations and models are co-designed and improved continuously through privacy aware sharing of personalized fog and edge models of all interested parties.

HUAWEI Joint Lab: Human-centered Empathetic Interaction
HUAWEI Joint Lab
Runtime: 01.01.2020 – 31.12.2022
Role: Lab Leader
Partners: HUAWEI, University of Augsburg

The Huawei-University of Augsburg Joint Lab aims to bring together Affective Computing & Human-Centered Intelligence for Human-centred empathic interaction.

KIrun: Einsatz Künstlicher Intelligenz in der Laufsportanalytik mit Audioanalyse/ -auswertung zur Motivation, Leistungssteigerung und Verletzungsprävention (FKZ: 16KN069402)
BMWi Zentrales Innovationsprogramm Mittelstand (ZIM) Projekt
Runtime: 01.12.2019 – 31.08.2022
Role: Principal Investigator, Co-author Proposal
Partners: Universitätsklinikum Tübingen, HB Technologies AG (HBT), University of Augsburg

Das Kooperationsprojekt KIRun verfolgt die Entwicklung eines Messsystems und eines selbstlernenden Algorithmus, der auf Basis von auditiven, biomechanischen und physiologischen Messdaten das Wohlbefinden und die Anstrengung autonom ermittelt. Der innovative Kern besteht in der Ermittlung des Wohlbefindens und der Anstrengung auf der Basis von objektiven Messdaten: Audiosignale (z.B. Atemgeräusche) werden in diesem System nicht zur Sprachsteuerung verwendet, sondern werden permanent erfasst, um daraus eigenständig Rückschlüsse auf das Wohlbefinden zu ziehen. Ein autonomes Messverfahren zur Datenerfassung, mit dem das Wohlbefinden und die Anstrengung objektiviert und zeitsynchron zum Laufen erfasst und in eine Trainingssteuerung eingebunden werden, gibt es bislang nicht. Dies stellt ein Alleinstellungsmerkmal der Technologie und eine erhebliche Verbesserung zum Stand der Technik in der Laufsportanalyse dar. Per App soll eine gezielte Beeinflussung des Läufers in Richtung Wohlbefinden möglich werden, so dass die Motivation des Läufers für das Lauftraining maximal gesteigert werden kann. Als Zielgröße des Lauftrainings wird das maximale Wohlbefinden und nicht wie bisher üblich die maximale Geschwindigkeit oder das größte Streckenpensum angestrebt. Viele Einsteiger und Gelegenheitsläufer sind aufgrund falscher Trainingsgestaltung frühzeitig demotiviert oder steigen verletzt wieder aus. KIRun stellt dagegen einen positiven Trainingseindruck für den Läufer in den Mittelpunkt. Die Steigerung des Wohlbefindens mit Hilfe der “KIRun”-Technologie ist damit der effektive Antrieb für den Sportler, um die regelmäßige körperlichen Aktivität auszuüben.

EMBOA: Affective loop in Socially Assistive Robotics as an intervention tool for children with autism
ERASMUS+ project

Runtime: 01.09.2019 – 31.08.2022
Role: Principal Investigator, Co-author Proposal
Partners: Politechnika Gdanska, University of Hertfordshire, Istanbul Teknik Universitesi, Yeditepe University Vakif, Macedonian association for applied psychology, University of Augsburg

The EMBOA project (Affective loop in Socially Assistive Robotics as an intervention tool for children with autism) aims at the development of guidelines and practical evaluation of applying emotion recognition technologies in robot-supported intervention in children with autism. Children with autism spectrum disorder (ASD) suffer from multiple deficits, and limited social and emotional skills are among those, that influence their ability to involve in interaction and communication. Limited communication occurs in human-human interaction and affects relations with family members, peers, and therapists. There are promising results in the use of robots in supporting the social and emotional development of children with autism. We do not know, why children with autism are eager to interact with human-like looking robots and not with humans. Regardless of the reason, social robots proved to be a way to get through the social obstacles of a child and make him/her involved in the interaction. Once the interaction happens, we have a unique opportunity to engage a child in gradually building and practicing social and emotional skills. In the project, we combine social robots, that are already used in therapy for children with autism with algorithms for automatic emotion recognition. The EMBOA project goal is to confirm the possibility of the application (feasibility study), and in particular, we aim at the identification of the best practices and obstacles in using the combination of the technologies. What we hope to obtain is a novel approach for creating an affective loop in child-robot interaction that would enhance interventions regarding emotional intelligence building in children with autism. The lessons learned, summarized in the form of guidelines, might be used in higher education in all involved countries in robotics, computer science, and special pedagogy fields of study. The results will be disseminated in the form of trainings, multiplier events, and to the general public by scientific papers and published reports. The project consortium is multidisciplinary and combines partners with competence in interventions in autism, robotics, and automatic emotion recognition from Poland, UK, Germany, North Macedonia, and Turkey. The methodological approach includes systematic literature reviews and meta-analysis, data analysis based on statistical and machine learning approaches, and as well observational studies. We have planned a double-loop of observational studies. The first round is to analyze the application of emotion recognition methods in robot-based interaction in autism, and especially to compare diverse channels for observation of emotion symptoms. The lessons learned would be formulated in the form of guidelines. The guidelines would be evaluated with the AGREE (Appraisal of Guidelines, Research, and Evaluation) instrument and confirmed with the second round of observational studies. The objectives of our project are matching the Social Inclusion horizontal priority with regards to supporting the actions for improvement of learning performance of disadvantaged learners (testing of a novel approach for improvement of learning performances of children with autism).

ForDigitHealth: Bayerischer Forschungsverbund zum gesunden Umgang mit digitalen Technologien und Medien
BayFOR (Bayerisches Staatsministerium für Wissenschaft und Kunst) Project

Runtime: 48 Months – 2019-31.05.2023
Role: Principal Investigator, Co-author Proposal
Partners: University of Augsburg, Otto-Friedrichs-University Bamberg, FAU Erlangen-Nuremberg, LMU Munich, JMU Würzburg

Die Digitalisierung führt zu grundlegenden Veränderungen unserer Gesellschaft und unseres individuellen Lebens. Dies birgt Chancen und Risiken für unsere Gesundheit. Zum Teil führt unser Umgang mit digitalen Technologien und Medien zu negativem Stress (Distress), Burnout, Depression und weiteren gesundheitlichen Beeinträchtigungen. Demgegenüber kann Stress auch eine positive, anregende Wirkung haben (Eustress), die es zu fördern gilt. Die Technikgestaltung ist weit fortgeschritten, sodass digitale Technologien und Medien dank zunehmender künstlicher Intelligenz, Adaptivität und Interaktivität die Gesundheit ihrer menschlichen Nutzerinnen und Nutzer bewahren und fördern können. Ziel des Forschungsverbunds ForDigitHealth ist es, die Gesundheitseffekte der zunehmenden Präsenz und intensivierten Nutzung digitaler Technologien und Medien – speziell in Hinblick auf die Entstehung von digitalem Distress und Eustress und deren Folgen – in ihrer Vielgestaltigkeit wissenschaftlich zu durchdringen sowie Präventions- und Interventionsoptionen zu erarbeiten und zu evaluieren. Dadurch soll der Forschungsverbund zu einem angemessenen, bewussten und gesundheitsförderlichen individuellen wie kollektiven Umgang mit digitalen Technologien und Medien beitragen.

ParaStiChaD: Paralinguistic Speech Characteristics in Major Depressive Disorder (#SCHU2508/8-1)
(“Paralinguistische Stimmmerkmale in Major Depression”)
DFG (German Research Foundation) Project
Runtime: 01.01.2020 – 31.12.2022
Role: Principal Investigator, Co-author Proposal
Partners: FAU Erlangen-Nuremberg, University of Augsburg, Rheinische Fachhoschule Köln

More needs to be done to improve the validity of current methods to detect depression, to improve the validity of ways to predict the future course of depression and to enhance the efficacy and availability of evidence-based treatments for depression. The work proposed in Paralinguistic Speech Characteristics In Major Depressive Disorder (ParaSpeChaD) aims to address these needs by clarifying the extent to which Paralinguistic Speech Characteristics (PSCs; i.e. the vocal phenomena that occur alongside the linguistic information in speech) can be used to detect depression and predict its future course and how recent progress in mobile sensor technology can be used to improve the detection, prediction and potentially even the treatment of depression.

Improving the specificity of affective computing via multimodal analysis
ARC Discovery Project (22% Acceptance Rate in 2nd Round of Call)
Runtime: 01.01.2020 – 31.12.2023
Role: Principal Investigator, Co-author Proposal
Partners: University of Canberra, University of Pittsburgh, CMU, Imperial College London

Being able to have computational models and approaches to sense and understand a person’s emotion or mood is a core component of affective computing. While much research over the last two decades has tried to address the question of sensitivity – the correct recognition of affect classes – the equally important issue of specificity – the correct recognition of true negatives – has been neglected. This highly inter-disciplinary project aims to address this issue and to solve the fundamental affective computing problem of developing robust non-invasive multimodal approaches for accurately sensing a person’s affective state. Of course, neither sensitivity, nor specificity should be seen in isolation. The underlying issue is one of conceptualising affective states as areas within a continuous space, of determining the affect intensity on a continuous scale and of being able to analyse very subtle expressions of affect.

ERIK: Entwicklung einer Roboterplattform zur Unterstützung neuer Interaktionsstrategien bei Kindern mit eingeschränkten sozioemotionalen Fähigkeiten
BMBF IKT2020-Grant (Forschungsprogramm Roboter für Assistenzfunktionen: Interaktionsstrategien)
Runtime: 01.11.2018 – 31.10.2021
Role: Beneficiary, Scientific and Technical Manager (STM)
Partners: Fraunhofer IIS, ASTRUM IT GmbH, Humboldt-Universität zu Berlin, Friedrich-Alexander-Universität Erlangen-Nürnberg, audEERING GmbH

Das Verstehen und Ausdrücken von sozio-emotionalen Signalen, wie z. B. Gesichtsausdruck und Stimmenmodulation, ist bei Kindern mit Autismus beeinträchtigt. Während menschliche Interaktionspartner für sie schwer einzuschätzen sind, nehmen diese Kinder Roboter als vorhersehbarer und weniger komplex wahr. Häufig sind sie zudem technisch interessiert und aufgeschlossen. Zur Entwicklung der sozio-emotionalen Kommunikationsfähigkeiten autistischer Kinder wird im Projekt ERIK eine neue Therapieform mit Hilfe eines robotischen Systems entwickelt und erprobt. Der Roboter „Pepper“ erfasst in der Interaktion mit dem Kind die Mimik und Sprache. Durch das Spielen mit dem Roboter-Ball „Leka“ kann zusätzlich über Elektroden der Puls ermittelt werden. „Pepper“ interpretiert diese Signale und leitet in Echtzeit Emotionen ab. Kombiniert mit der Therapie-App „Zirkus Empatico“ können alltagsrelevante emotionale und soziale Fähigkeiten trainiert werden. Durch das Erkennen von Interesse, Frustration und Langweile des Kindes können die Therapieszenarien individuell angepasst werden. Mittels Gesten und Augenbewegungen kann „Pepper“ lebensnah mit Kindern interagieren, wobei Ängste im Umgang mit Menschen reduziert werden können. Der innovative Therapieansatz erlaubt Therapeuten, Interaktionen genauer zu beobachten und auszuwerten, da sie selbst nicht mehr Teil der Interaktion sind. Die emotionssensitive Robotik kann außerdem erstmalig auch mit Gruppen von Kindern interagieren.

sustAGESmart environments for person-centered sustainable work and well-being (#826506)
EU Horizon 2020 Research & Innovation Action (RIA)
Runtime: 01.01.2019 – 30.06.2022
Role: Principal Investigator, Scientific and Technical Manager (STM), Workpackage Leader, Co-Author Proposal
Partners: Foundation for Research and Technology Hellas, Centro Ricerche Fiat SCPA, Software AG, Imaginary SRL, Forschungsgesellschaft für Arbeitsphysiologie und Arbeitsschutz e.V., Heraklion Port Authority S.A., Aegis IT Research UG, University of Augsburg, Aristotelio Panepistimio Thessalonikis, Universidad Nacional de Educacion a Distancia

sustAGE aims to develop a person-centered solution for promoting the concept of “sustainable work” for EU industries.
The project provides a paradigm shift in human machine interaction, building upon seven strategic technology trends, IoT, Machine learning, micro-moments, temporal reasoning, recommender systems, data analytics and gamification to deliver a composite system integrated with the daily activities at work and outside, to support employers and ageing employees to jointly increase well-being, wellness at work and productivity. The manifold contribution focuses on the support of the employment and later retirement of older adults from work and the optimization of the workforce management. The sustAGE platform guides workers on work-related tasks, recommends personalized cognitive and physical training activities with emphasis on game and social aspects, delivers warnings regarding occupational risks and cares for their proper positioning in work tasks that will maximize team performance. By combining a broad range of the innovation chain activities namely, technology R&D, demonstration, prototyping, pilots, and extensive validation, the project aims to explore how health and safety at work, continuous training and proper workforce management can prolongue older workers’ competitiveness at work. The deployment of the proposed technologies in two critical industrial sectors and their extensive evaluation will lead to a ground-breaking contribution that will improve the performance and quality of life at work and beyond for many ageing adult workers.

WorkingAge: Smart Working environments for all Ages (#210487208)
EU Horizon 2020 Research & Innovation Action (RIA)
Runtime: 36 months
Role: Principal Investigator, Co-Author Proposal
Partners:  Instituto Tecnológico de Castilla y Leon, Exodus Anonymos Etaireia Pliroforikis, University of Cambridge, Politecnico di Milano, Green Communications SAS, Brainsigns SRL, RWTH Aachen, Telespazio France SAS, audEERING GmbH, European Emergency Number Association ASBL, Fundacion Intras, Telematic Medical Applications MEPE

WorkingAge will use innovative HCI methods (augmented reality, virtual reality, gesture/voice recognition and eye tracking) to measure the user emotional/cognitive/health state and create communication paths. At the same time with the use of IoT sensors it will be able to detect environmental conditions. The purpose is to promote healthy habits of users in their working environment and daily living activities in order to improve their working and living conditions. By studying the profile of the >50 (year old) workers and the working place requirements in three different working environments (Office, Driving and Manufacturing), both profiles (user and environment) will be considered. Information obtained will be used for the creation of interventions that will lead to healthy aging inside and outside the working environment. WorkingAge will test and validate an integrated solution that will learn the user’s behaviour, health data and preferences and through continue data collection and analysis will interact naturally with the user. This innovative system will provide workers assistance in their everyday routine in the form of reminders, risks avoidance and recommendations. In this way the WorkingAge project will create a sustainable and scalable product that will empower their users’ easing their life by attenuating the impact of aging in their autonomy, work conditions, health and well-being.

ECoWeB: Assessing and Enhancing Emotional Competence for Well-Being (ECoWeB) in the Young: A principled, evidence-based, mobile-health approach to prevent mental disorders and promote mental wellbeing (#754657)
EU Horizon 2020 Research & Innovation Action (RIA)
Runtime: 01.01.2018 – 31.12.2021
Role: Principal Investigator, Innovation Manager, Innovation Management Board Chair, Workpackage Leader, Co-Author Proposal
Partners:  University of Exeter, audEERING GmbH, Vysoke Uceni Technicke v Brne, Institute of Communication and Computing Systems, Universitat Jaume i de Castellon, Fraunhofer Gesellschaft, University of Oxford, University of Geneva, LMU Munich, University of Gent, Monsenso ApS, University of Copenhagen, Deutsches Jugendinstitut eV

Although there are effective mental well-being promotion and mental disorder prevention interventions for young people, there is a need for more robust evidence on resilience factors, for more effective interventions, and for approaches that can be scalable and accessible at a population level. To tackle these challenges and move beyond the state-of-the-art, ECoWeB uniquely integrates three multidisciplinary approaches: (a) For the first time to our knowledge, we will systematically use an established theoretical model of normal emotional functioning (Emotional Competence Process) to guide the identification and targeting of mechanisms robustly implicated in well-being and psychopathology in young people; (b) A personalized medicine approach: systematic assessment of personal Emotional Competence (EC) profiles is used to select targeted interventions to promote well-being: (c) Mobile application delivery to target scalability, accessibility and acceptability in young people. Our aim is to improve mental health promotion by developing, evaluating, and disseminating a comprehensive mobile app to assess deficits in three major components of EC (production, regulation, knowledge) and to selectively augment pertinent EC abilities in adolescents and young adults. It is hypothesized that the targeted interventions, based on state-of-the-art assessment, will efficiently increase resilience toward adversity, promote mental well-being, and act as primary prevention for mental disorders. The EC intervention will be tested in cohort multiple randomized trials with young people from many European countries against a usual care control and an established, non-personalized socio-emotional learning digital intervention. Building directly from a fundamental understanding of emotion in combination with a personalized approach and leading edge digital technology is a novel and innovative approach, with potential to deliver a breakthrough in effective prevention of mental disorder.

TAPAS: Training network on Automatic Processing of PAthological Speech (#766287)
EU H2020 Marie Sklodowska-Curie Innovative Training Networks European Training Networks (MSCA-ITN-ETN:ENG)
TAPAS: Training Network on Automatic Processing of PAthological Speech
Runtime: 01.11.2017 – 31.10.2021
Role: Principal Investigator, Co-Author Proposal
Partners: IDIAP, Université Paul Sabatier Toulouse III, Universitair Ziekenhuis Antwerpen, FAU Erlangen-Nürnberg, Stichting Katholieke Universiteit, INESC ID, LMU Munich, Interuniersitair Micro-Electronicacentrum IMEC, Stichting het Nederlands Kanker Instituutantoni van Leeuwenhoek Ziekenhuis, University of Augsburg, University of Sheffield, audEERING GmbH

There are an increasing number of people across Europe with debilitating speech pathologies (e.g., due to stroke, Parkinson’s, etc). These groups face communication problems that can lead to social exclusion. They are now being further marginalised by a new wave of speech technology that is increasingly woven into everyday life but which is not robust to atypical speech. TAPAS is proposing a programme of pathological speech research, that aims to transform the well-being of these people. The TAPAS work programme targets three key research problems: (a) Detection: We will develop speech processing techniques for early detection of conditions that impact on speech production. The outcomes will be cheap and non-invasive diagnostic tools that provide early warning of the onset of progressive conditions such as Alzheimer’s and Parkinson’s. (b) Therapy: We will use newly-emerging speech processing techniques to produce automated speech therapy tools. These tools will make therapy more accessible and more individually targeted. Better therapy can increase the chances of recovering intelligible speech after traumatic events such a stroke or oral surgery. (c) Assisted Living: We will re-design current speech technology so that it works well for people with speech impairments and also helps in making informed clinical choices. People with speech impairments often have other co-occurring conditions making them reliant on carers. Speech-driven tools for assisted-living are a way to allow such people to live more independently. TAPAS adopts an inter-disciplinary and multi-sectorial approach. The consortium includes clinical practitioners, academic researchers and industrial partners, with expertise spanning speech engineering, linguistics and clinical science. All members have expertise in some element of pathological speech. This rich network will train a new generation of 15 researchers, equipping them with the skills and resources necessary for lasting success.

Sating Curiosity While Avoiding Risk in Reinforcement Learning (#2021037)
HiPEDS EPSRC / Presidential Scholarship, Imperial College, Industry integrated Centre for Doctoral Training
Runtime: 01.10.2017 – 30.09.2021
Role: Supervisor, Co-author Proposal
Partners: Imperial College London

In reinforcement learning, an agent tries to maximise some notion of expected future cumulative reward by sequentially selecting actions that affect its environment. Its applications are many and range from embodied robotics, through biomedical engineering, to human-robot interaction via speech and emotion recognition. However, in many such applications the great amounts of real data required to effectively train the agent may be difficult or expensive to obtain and an inadequate agent policy can be catastrophic. Here I propose the combination of a) environment models that make more efficient use of the agent’s past experience and b) directed exploration that encourages the exploration of more informative locations in the environment – both by means of Bayesian deep neural networks. Recent advances in the latter show them to combine the ability of deep neural networks to learn from complex, high-dimensional data and also the aversion to overfitting and learning with uncertainty of Bayesian methods. With this approach, I expect to be able to design reinforcement learning solutions that can exhibit an improved usage of data both at the beginning and at later stages of the agent’s exploration.

RADAR CNS: Remote Assessment of Disease and Relapse in Central Nervous System Disorders (#115902) – 15.8% acceptance rate in the call
EU H2020 / EFPIA Innovative Medicines Initiative (IMI) 2 Call 3

Runtime: 01.04.2016 – 31.03.2022
Role: Coauthor Proposal, Beneficiary, Principal Investigator, Workpackage Leader
Partners: King’s College London, Provincia Lombardo-Veneta – Ordine Ospedaliero di San Giovanni di Dio— Fatebenefratelli Lygature, Università Vita-Salute San Raffaele, Fundacio Hospital Universitari Vall D’Hebron, University of Nottingham, Centro de Investigacion Biomedica en Red, Software AG, Region Hovedstaden, Stichting VU-Vumc, University Hospital Freiburg, Stichting IMEC Nederland, Katholieke Universiteit Leuven, Northwestern University, Stockholm Universitet, University of AugsburgUniversity of Passau, Università degli Studi di Bergamo, Charité – Universitätsmedizin Berlin, Intel Corporation (UK) Ltd, GABO:mi, Janssen Pharmaceutica NV, H. Lundbeck A/S, UCB Biopharma SPRL, MSD IT Global Innovation Center

The general aim is to develop and test a transformative platform of remote monitoring (RMT) of disease state in three CNS diseases: epilepsy, multiple sclerosis and depression. Other aims are: (i) to build an infrastructure to identify clinically useful RMT measured biosignatures to assist in the early identification of relapse or deterioration;  (ii) to develop a platform to identify these biosignatures;  (iii) to anticipate potential barriers to translation by initiating a dialogue with key stakeholders (patients, clinicians, regulators and healthcare providers).


Comments are closed.