Academic

Deep neural networks when trained on huge collections of labelled data have shown impressive performance, often matching human-level performance, on a wide range of supervised learning tasks such as image recognition. However, creating such large-scale labelled datasets requires significant resources, time and effort. For this reason, semi-supervised learning (SSL) has been a hot research topic in machine learning in the last decade. SSL is concerned with the use of both labelled and unlabelled data for training. The main challenge of SSL is in the design of methods that can exploit the information contained in the distribution of the unlabelled data. In this thesis, we focus on recently proposed consistency-based SSL methods such as the $\Pi$-model, temporal ensembling, the mean teacher, or the virtual adversarial training, which have advanced the state of the art in several SSL tasks. These methods can typically reach performances that are comparable to their fully supervised counterparts while using only a fraction of labelled examples. Despite these methodological advances, the understanding of these methods is still relatively limited.  In this thesis, we analyse (variations of) the $\Pi$-model in settings where analytically tractable results can be obtained. We establish links with Manifold Tangent Classifiers and demonstrate that the quality of the perturbations is key to obtaining reasonable SSL performances. Importantly, we propose a simple extension of the Hidden Manifold Model that naturally incorporates data-augmentation schemes and offers a framework for understanding and experimenting with SSL methods. In this thesis, we also propose a new clustering-based self-supervised learning method, that has recently gained much attention, for its ability to learn general image and video features from large-scale unlabelled data without using any human-annotated labels. Our method, contrarily to several recently proposed approaches, does not directly rely on contrastive learning at the instance level, does not require large batch sizes or memory banks to be trained reliably and does not rely on stop-gradient operations or the use of an auxiliary exponentially averaged network to prevent collapse. Finally, we borrow ideas from self and semi-supervised learning methods developed specifically for the image recognition task and propose a new approach for semi-supervised semantic segmentation task for medical images, which plays a crucial role in many medical imaging applications, by automating the task of finding the category of every pixel in the image. We propose a patch-based and a point-based self-supervised contrastive learning framework that can make efficient use of the information contained in the much larger unlabelled data. We empirically show that our semi-supervised algorithm achieves significant gain in performance over other baselines for several medical image datasets.