DESCRIPTION :
The STARS research team combines advanced theory with cutting-edge practice, with a focus on computer vision systems.
Team website: https://team.inria.fr/stars/
Scientific context
Feature extraction is a challenging computer vision problem that targets extracting relevant information from raw data in order to reduce dimensionality and capture meaningful patterns. When this needs to be done in a dataset and task-invariant way, it is referred to as general feature extraction. This is a crucial step in machine learning pipelines, and popular methods like VideoSwin and VIdeoMAE work well for the task of action recognition and video understanding. However, these works and also the datasets that they are tested on, like Something-Something and Kinetics, fail to capture information about interactions in daily life.
This work will be part of the ANR PROGRESS FSHD project.
The ANR FSHD project (21 June 2022 - 20 May 2026) aims to identify new Clinical outcomes Assessments (CoAs), digital and inflammatory biomarkers of disease severity and progression, and increase the knowledge of the key role of inflammation in facioscapulohumeral muscular dystrophy (FSHD) pathophysiology. This project will extend the already existing CTRN FSHD France database by implementing, at the Nice hospital, full-body motion video capturing of validated functional scores and at-home patient remote assessment, including digitalized score and full-body motion video capturing (PROGRESS FSHD app). Clinical and remotely collected data will be analyzed using classical multivariate statistical approaches, while full-body motion video will be analyzed with Deep Learning methods. An interconnection between patient data collected at the hospital and at home will be ensured, along with interconnection and interoperability with France Cohortes and RaDiCo.
So, the main question is:
How to extract general features from multimodal data with a lot of noise in the form of irrelevant information? Typical situations that we would like to monitor are daily interactions, responses, and reactions, and analyse cause and effect in behaviour.
The system we want to develop will be beneficial for all tasks requiring focus on interactions. Specifically, healthcare for psychological disorders -- general feature extraction will allow deep learning models to assist in various subtasks involved in the diagnosis process.
Mission confiée
In this work, we would like to go beyond existing computer vision deep learning models and introduce ways to extend them to utilise information from new modalities. Also, to identify ways to focus on relevant information for interactions in the input. The system should also take into account the long temporal duration of videos in the datasets in this domain. These have to be done in a flexible way, so that there is minimal change to the original model and hence the original model's trained weights are useful too.
Existing methods have mostly focused on modelling the variation of visual cues pertinent to the classes provided for video classification tasks. Though they perform these tasks well, changes in the recording setting or addition of noise in the form of irrelevant background information makes it hard for these models to perform well. So, for obtaining a general feature extractor, the models have to be modified to accommodate for these shortcomings.
Niveau de formation : Bac+5
Temps partiel / Temps plein : Plein temps
Type de contrat : Contrat à durée indéterminée (CDI)
Compétences : Vision par Ordinateur, C ++ (Langage de Programmation), Programmation Informatique, Bases de Données, Linux, Python (Langage de Programmation), Machine Learning, OpenCV, Données Brutes, Tensorflow, Pytorch, Deep Learning, Technologies Informatiques, Extraction des Caractéristiques, Anglais, Leadership, Enthousiasme, Stabilité Émmotionnelle, Esprit d'Équipe, Implication et Investissement, Innovation, Dossiers Médicaux, Travaux Cliniques, Soins de Santé, Mathématiques, Traitement des Maladies Mentales, Recherche Scientifique, Réalisation d'Évaluations, Biomarqueur
Courriel :
Francois.Bremond@inria.fr
Téléphone :
0139635511
Type d'annonceur : Employeur direct