Deep neural networks are powerful, massively parameterized machine learning models that have been shown to perform well in supervised learning tasks. However, very large amounts of labeled data are usually needed to train deep neural networks. Several semi-supervised learning approaches have been proposed to train neural networks using smaller amounts of labeled data with a large amount of unlabeled data. The performance of these semisupervised methods significantly degrades as the size of labeled data decreases.

Few-shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation. Even though the idea of meta-learning for adaptation has dominated the few-shot learning methods, how to train a feature extractor is still a challenge. In this paper, we focus on the design of a training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples. We propose a two-stage training scheme, Partner-Assisted Learning (PAL), which first trains a Partner Encoder to model pair-wise similarities and extract features serving as soft-anchors, and then trains the Main Encoder by aligning its outputs with soft-anchors while attempting to maximize classification performance.

In media forensics, there has been extensive research in finding fake news using embedded imagery materials (photos or videos). Not only manipulated images can make fake news, but unaltered images can also be repurposed to make the news look real. Thus, when no evidence of manipulation can be found by using all scientific methods, one may also consider whether the materials were really taken from the event mentioned in the news.

As equality issues in the use of face recognition have garnered a lot of attention lately, greater efforts have been made to debiased deep learning models to improve fairness to minorities. However, there is still no clear definition nor sufficient analysis for bias assessment metrics. Therefore, we propose an information-theoretic, independent bias assessment metric to identify degree of bias against protected demographic attributes from learned representations of pretrained facial recognition systems.

Semantic segmentation has many applications in our life. For example, in self-driving cars, the system needs to understand the road, traffic signs, pedestrians, and so on in order to decide when and where to go. In medicine, researchers can use semantic segmentation to analyze medical images, such as magnetic resonance imaging, to find potential tumors. In Computer Vision, people usually regard semantic segmentation as a special classification problem. In the traditional classification problem, the system only needs to give one label to a given figure. While in semantic segmentation, the system classifies all pixels in a dense annotation way.

Fake news has become a familiar term in recent years. It’s infamy is superseded by the harm that it creates in society. It has the ability to manipulate election outcomes or upend a stock value. But what is fake news and what gives it this undue influence? In simple terms, it refers to manipulated multimedia (e.g. text, images, videos, etc.) containing false information. A toxic mixture of truth and falsehood usually makes it believable and persuasive. It’s a no-brainer that we have to put in our effort to mitigate its impact. This blog discusses methods to tackle a subset of the problem called Image repurposing – where a pristine image is associated with false metadata to convey misinformation. Figure 1 shows an example of image repurposing.

Virtually all aspects of modern society have been shaped by developments in science and technology. Most of the modern world invests to promote research and development and in turn utilizes the outcomes to shape a better future. At the core of this symbiotic relationship is an inherent trust in the integrity of the scientific process i.e. the experiments and findings of the scientific community are authentic. But what happens when this trust is shaken?

Face recognition systems are vulnerable to different types of presentation attacks, such as printed face images and plastic masks. To mitigate such vulnerability, face recognition systems are augmented with presentation attack detection modules. One of the most challenging attack types to detect with such modules is makeup. Makeup can substantially change the facial characteristics of a person, including, for example, their age and gender. Yet, makeup preserves the naturalness of the 3D geometry of the face, unlike flat printed images for example; and its material is normally worn every day by many people, unlike plastic masks for example.

On social media and the Internet, visual disinformation has expanded dramatically. The fake video in today’s digital age is frequently highly convincing, giving the impression that the swapped subject is the actual acting person in the video. Thanks to recent advances in data synthesis using Generative Adversarial Networks (GANs), Deep Convolutional Neural Networks (DCNN), and AutoEncoders (AE), face-swapping in videos with hyper-realistic results has become effective and efficient for non-experts with a few clicks through customized applications, or even mobile applications.

Most of the face recognition systems trained on a closed set of identities are often quite brittle to partial occlusions of the face. Some of the common occlusions we see in the wild are sunglasses, caps/hats, hands, or even scarves. Since most face biometric systems heavily depend on identification, a malicious actor could use one of these commonly found occlusion articles to evade detection or cause significant degradation of performance.

The matching problem is a basic problem in computer vision. There are region-based matching problems, such as template matching, and semantic-based matching problems, such as semantic alignment. In this blog, we will first go through classic solutions for template matching. Then we will demonstrate our deep learning-friendly matching algorithm QATM and its performance on template matching in a training-free way (only use pre-trained model weights, no other training involved). Finally, we implement QATM as a learning DNN module and show its applications in semantic matching problems (e.g. semantic alignment).

Fake news, Internet rumors, insurance fraud, extortion, and even academic publications have all been affected by image forgery, which has recently become an epidemic. Furthermore, the majority of image forgeries have yet to be recognized. Each of the retracted articles due to deliberate manipulation might account for a mean of $392,582 indirect expenses in biomedical research publications alone, implying much larger indirect costs due to misguided research. As a result, new algorithms must be developed to aid in the fight against image manipulation and forgery.

Representation learning is a key ingredient for machine learning algorithms and its effectiveness and efficiency has a direct correlation to performance of an algorithm. Supervised algorithms involve learning a mapping from a data sample x to a target variable y by estimating a conditional probability p(y|x) from the data. In the real world, often the data sample x is itself composed of many nuisance factors and noises that are irrelevant to the prediction of y and could lead to overfitting of the model. For example, a nuisance factor in the case of face recognition is images is the lighting condition the photograph was captured in, and a recognition model that associates lighting with subject identity is expected to perform poorly.

CAH is a genetic disorder that causes abnormal function of the adrenal gland. CAH is the most common adrenal insufficiency disorder in children and has serious life-long health implications, including blood pressure, diabetes, altered cognition, and obesity. Other adverse neuropsychological outcomes have also been identified over the lifespan of patients with CAH, including a heightened potential for psychiatric disorders, substance abuse, and suicide, and brain structural abnormalities have been identified in youths and adults with CAH.

Because of the rapid growth of social networks and advancements in image editing software, it is much easier to manipulate an image in a more realistic manner. Although many user manipulations are not malicious in nature, image forgery has become a more serious and widespread problem in recent years. One of the most common types of image forgery is image copy-move forgery since it is simple to accomplish using a variety of photo-editing software.

Image forgery is becoming a widespread problem as a result of the proliferation of digital content, and sophisticated image editing tools have been pushing the limits of image composition in recent years in order to produce more natural and aesthetic images. Meanwhile, new professionals, forensic experts, and legal prosecutors are finding it increasingly difficult to detect and localize image forgeries on a large scale. These new challenges necessitate the development of innovative and scalable image forensics technologies.

Face segmentation and parsing are extremely useful technologies because their output masks can be used to enable next-generation face analysis, enhanced face swapping, more complex face editing, and face completion. Face parsing differs from scene object segmentation in that faces are roughly size and translation invariant, despite being closely related to generic semantic segmentation and using the same methodology.

When learning to correlate a target with underlying aspects of data that are informative of the target, supervised machine learning models frequently learn to associate irrelevant factors. And overfitting occurs when erroneous links between the target and nuisance elements are learned. Biasing factors, on the other hand, may cause models to learn connections between the goal and factors that are linked with the target only in collected training data. Because of such biasing factors, trained models can be unfair to groups that are under-represented in the training data, raising ethical and legal issues. As a result, it’s critical to create models that are immune to both nuisance and biasing effects.

Because public safety heavily relies on all kinds of biometric identification and authentication systems, such as those used in law enforcement and border control, the Presentation Attack Detection (PAD) problem, also known as spoofing detection, attracts a lot of attention from the biometrics community. Any flaw that could expose these systems to phony/augmented presentations is unaffordable. Therefore, for high-security applications, human-in-the-loop is essential. Preventing presentation attacks (PAs) via human intervention, on the other hand, is not scalable and error-prone. However, because biometric data is more easily obtained, manufacturing or augmenting artifacts to spoof biometric systems is far easier than before. As a result, PAD solutions are needed to mitigate PA threats to current and future biometric identification systems.

A large number of deep neural network-based techniques have been developed to address the challenging problem of face presentation attack detection (PAD). Whereas such techniques’ focus has been on improving PAD performance in terms of classification accuracy and robustness against unseen attacks and environmental conditions, there exists little attention on the explainability of PAD predictions.

Advances in deep learning, combined with the availability of large datasets, have led to impressive improvements in face presentation attack detection research. However, state-of-the-art face antispoofing systems are still vulnerable to novel types of attacks that are never seen during training. Moreover, even if such attacks are correctly detected, these systems lack the ability to adapt to newly encountered attacks. In this paper, we enable a deep neural network to detect anomalies in the observed input data points as potential new types of attacks by suppressing the confidence-level of the network outside the training samples’ distribution.

In this work, we present a general framework for building a biometrics system capable of capturing multispectral data from a series of sensors synchronized with active illumination sources. The framework unifies the system design for different biometric modalities and its realization on face, finger and iris data is described in detail. To the best of our knowledge, the presented design is the first to employ such a diverse set of electromagnetic spectrum bands, ranging from visible to long-wave-infrared wavelengths, and is capable of acquiring large volumes of data in seconds, which enabled us to successfully conduct a series of data collection events.