2023-05783 - PhD Position F/M Object Detection from Few Multispectral Examples
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
About the research centre or Inria departmentThe Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.
ContextATERMES is an international mid-sized company, based in Montigny-le- Bretonneux (France) with strong expertise in high technology and system integration from the upstream design to the long-life maintenance cycle. It specializes in offering system solutions for border surveillance. Its flagship product BARIER(TM) (“Beacon Autonomous Reconnaissance Identification and Evaluation Response”) provides ready application for temporary strategic site protection or ill-defined border regions in mountainous or remote terrain where fixed surveillance modes are impracticable or overly expensive to deploy. As another example, SURICATE is the first of its class optronic ground "RADAR" that covers very efficiently wide field with automatic classification of intruders thanks to multi-spectral deep learning detection.
The collaboration between ATERMES and IRISA was initiated through a first PhD thesis (Heng Zhang, defended December 2021, https: // www. theses.fr/2021REN1S099/document). This successful collaboration led to multiple contributions to object detection in both mono-modal (RGB) and multi-modal (RGB+THERMAL) scenarios. Besides, this study allowed identifying remaining challenges that need to be solved to ensure multispectral object detection in the wild.
Supervision team The PhD will be co-supervised by Prof. Elisa Fromont (LACODAM team, IRISA/INRIA Rennes) and Prof. Sébastien Lefèvre (OBELIX team, IRISA Vannes). The supervision team will be completed by Dr. Minh-Tan Pham (Ass. Prof., OBELIX team) and Bruno Avignon (CSO, ATERMES).
Application Procedure Your application (CV+cover letter+academic transcripts) should be sent before 30/04/2023 (but the sooner the better) to the 4 email addresses: [email protected]; [email protected]; [email protected]; [email protected]
Applications will be treated and interviews will be conducted along the way. The candidate will be hired with a CIFRE(https: // www. anrt.asso.fr/fr/le- dispositif-cifre-7844) contract by ATERMES. The expected gross salary is around 3500€ per month for 3 years. The contract will start before the end of 2023 (ideally in October). Atermes can hire the candidate (as an engineer, CDI) before the beginning of the CIFRE contract if necessary.
AssignmentThe project aims at providing deep learning-based methods to detect objects in outdoor environments using multispectral data in a low supervision context, e.g., learning from few examples to detect scarcely-observed objects. The data consist of RGB and IR (Infra-red) images which are frames from calibrated and aligned multispectral videos. Few-shot learning 12, active learning 3 and incremental/continual learning 45 are among the frameworks to be investigated since they allow to limit the number of labeled examples needed for learning. Most developed methods 6789 based on these approaches have been proposed to perform object detection from RGB images within different weakly- supervised scenarios. They should be adapted and improved to deal with scarce object detection from multispectral images.In case of lacking objects of interest during the training, anomaly detection approaches 1011 can be also considered to detect new object classes which will be further characterized by prior semantic concepts. In addition to the (private) data from ATERMES, the PhD candidate will be able to work with public benchmarks such as KAIST (https: // soonminhwang.github.io/rgbt-ped-detection/data/), FLIR (https: // drive.google.com/file/d/1xHDMGl6HJZwtarNWkEV3T4O9X4ZQYz2Y/view), VEDAI (https: // downloads.greyc.fr/vedai/) or MIL (https: // www. mi.t.u-tokyo.ac.jp/static/projects/milmultispectral/) to benchmark the developed frameworks in the vision and machine learning communities.
Bibliography
1. Köhler, M., Eisenbach, M. and Gross, H.M., 2021. Few-Shot Object Detection: A Survey. arXiv preprint arXiv:2112.11699. https: // arxiv.org/pdf/2112.11699.pdf 2. Huang, G., Laradji, I., Vazquez, D., Lacoste-Julien, S. and Rodriguez, P., 2021. A survey of self-supervised and few-shot object detection. IEEE PAMI 2022. https: // arxiv.org/pdf/2110.14711v3.pdf 3. Brust, C.A., Käding, C. and Denzler, J., 2018. Active learning for deep object detection. arXiv preprint arXiv:1809.09875. https: // arxiv.org/abs/1809.09875 4. De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G. and Tuytelaars, T., 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7), pp.3366-3385. https: // arxiv.org/abs/1909.08383
5. Montiel, J., Halford, M., Mastelini, S.M., Bolmier, G., Sourty, R., Vaysse, R., Zouitine, A., Gomes, H. M., Read, J., Abdessalem, T. and Bifet, A., 2021. River: machine learning for streaming data in python. The Journal of Machine Learning Research, 22(1), pp.4945-4952. https: // github.com/online-ml/river 6. Fan, Z., Ma, Y., Li, Z. and Sun, J., 2021. Generalized few-shot object detection without forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4527-4536). https: // arxiv.org/pdf/2105.09491.pdf 7. Ahmad, T., Dhamija, A.R., Cruz, S., Rabinowitz, R., Li, C., Jafarzadeh, M. and Boult, T.E., 2022. Few-Shot Class Incremental Learning Leveraging Self- Supervised Features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3900-3910). https: // ieeexplore.ieee.org/document/9857447 8. Wang, X., Huang, T.E., Darrell, T., Gonzalez, J.E. and Yu, F., 2020. Frustratingly simple few-shot object detection, ICML 2020. https: // arxiv.org/pdf/2003.06957.pdf 9. Han, G., Ma, J., Huang, S., Chen, L. and Chang, S.F., 2022. Few-shot object detection with fully cross-transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 5321-5330). https: // arxiv.org/abs/2203.15021 10. Gangloff, H., Pham, M.T., Courtrai, L. and Lefèvre, S., 2022. Variational Autoencoder with Gaussian Random Field prior: application to unsupervised animal detection in aerial images. https: // hal.science/hal-03774853/document 11. Gangloff, H., Pham, M.T., Courtrai, L. and Lefèvre, S., 2022, August. Leveraging Vector-Quantized Variational Autoencoder Inner Metrics for Anomaly Detection. In 2022 26th International Conference on Pattern Recognition (ICPR) (pp. 435-441). IEEE. https: // ieeexplore.ieee.org/document/9956102
Main activitiesThe PhD candidate will work part time (80%) at IRISA (with 1 day per week in Rennes and the rest of the time in the Vannes IRISA facility) and part time (20%) in ATERMES in Paris (which corresponds to 2 days every 2 weeks). The exact schedule will be flexible: it might be preferable to spend more time in the company at the beginning of the thesis to learn about the system and understand the data and be full time in the lab while writing the PhD dissertation.
- T0-T0+8: The PhD candidate will survey the recent literature about deep learning under low supervision scenarios in the broad sense, with a specific focus on methods adapted to the (multispectral) object detection problem. - T0+9 - T0+24: During this period, the candidate will propose original contributions to tackle the problem of low supervision for multispectral object detection. We expect contributions related to few-shot learning, incremental and/or adaptive learning. - T0+24 – T0+32: During this period, the candidate will integrate its contributions to the system developed by ATERMES. - T0+33 - T0+ 36: The last period will be dedicated to writing the PhD dissertation
Skillsmonthly gross salary amounting to 2051 euros for the first and second years and 2158 euros for the third year
General InformationTheme/Domain : Data and Knowledge Representation and Processing Information system (BAP E)
Town/city : Vannes
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.
Instruction to applyPlease submit online : your resume, cover letter and letters of recommendation eventually
For more information, please contact [email protected]
Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.