An Overview of ASVspoof Challenge - A Community Led Effort towards Voice Spoofing Detection Research

Dr. Bhusan Chettri earned his PhD in AI and Speech Technology from Queen Mary University of London. His research focussed on analysis and design of voice spoofing detection using machine learning and AI. In this article Bhusan Chettri gives an insight on ASVspoof following his experience as a participant in two consecutive ASVspoof challenges held in 2017 and 2019 edition. See this for related publications and this for his PhD research work.

It is well acknowledged how vulnerable today’s Automatic Speaker Verification (ASV) systems, trained on vast amount of speech data using complex deep learning algorithms, are. To address the issue, the ASV community came up with the idea of promoting the research in spoofing detection by providing common evaluation protocols (which enables fair comparison of research findings), free spoofing datasets by organising a bi-annual research challenge so called automatic speaker verification and spoofing countermeasures ASVspoof challenge. See the official website for further details here.

ASVspoof is an ASV community driven effort promoting research in developing anti-spoofing algorithms for secure voice biometrics. A number of independent research studies had confirmed the vulnerability of voice biometrics to spoofing attacks, before the ASVspoof series began in 2015. However, these studies were mostly performed on small in-house datasets comprising limited speakers and spoofing attack conditions. Therefore, research results were hard to reproduce and understanding the true generalisability of the reported anti-spoofing solutions in unseen attack conditions was difficult. The main motivation of the ASVspoof series was to overcome these issues by organizing open spoofing challenge evaluations, promoting awareness of the problem, making publicly available spoofing corpora comprising sufficiently varying attack conditions with standard evaluation protocols, and further ensuring transparent research leading to reproducible results.

The first ASVspoof challenge held in 2015 focused on the detection of artificial speech generated using either speech synthesis (TTS) or voice conversion (VC) algorithms in a text-independent setting. Clean speech recorded using high quality microphones was used as bonafide speech and seven VC and three TTS algorithms were used to produce spoofed speech. The second edition of the ASVspoof challenge held in 2017 focussed on text-dependent replay spoofing attack detection. The 2019 edition, ASVspoof 2019, combined both TTS, VC and replay attacks together, using advanced state-of-the art spoofing algorithms and methods to generate spoofed speech samples. The recent edition held in 2021, ASVspoof 2021 used both LA and PA attacks but this edition also added a new track: Audio Fake Detection challenge. In the 2021 edition, the training and development data were not provided to the challenge participants. They were required to use the ASVspoof 2019 edition training and development datasets to train and tune their anti-spoofing systems. This edition only provided the fresh new evaluation set to evaluate the models.

One key observation that is worth noting from the three ASVspoof challenges is the paradigm shift in the use of modelling approaches for spoofing detection. Gaussian mixture models (GMMs), which is a generative model, were popular during the first ASVspoof challenge in 2015 as evident from the winning system of this challenge which is a GMM-based system. However, the 2017 and 2019 spoofing challenges were mostly dominated by data-driven discriminatively trained deep models. The main task, however, in all the three editions of the ASVspoof challenge was to build a standalone countermeasure model (anti-spoofing algorithm) that determines if a given speech recording is bonafide or a fake recording (spoofed). As for the performance evaluation, the equal error rate (EER) was used as a primary metric in the 2015 and 2017 edition. As for the 2019 edition, a recently introduced tandem detection cost function (t-DCF) metric [Kinnunen et al., 2018] was used as a primary metric and EER as the secondary metric.

Thanks to the ASV community, we don't have to worry about putting our own money in purchasing the spoofing datasets. These are made public and can be downloaded at no cost. The ASVspoof 2017 dataset is the first publicly available replay spoofing dataset designed by playing back bonafide audio utterances and re-recording them in real ‘wild’ acoustic conditions. It has been widely used by researchers around the globe. It has two data versions: 1.0 and 2.0. The version 1.0 was used during the ASVspoof 2017 evaluation. Post evaluation due to biases found in the dataset, a corrected version was released by the ASVspoof organisers.

Datasets and metrics: Speaking about replay attack anti-spoofing datasets, Bhusan Chettri explains that the ASVspoof 2017 dataset is the first publicly available replay spoofing dataset designed by playing back bonafide audio utterances and re-recording them in real ‘wild’ acoustic conditions. It has been extensively used in research since its release in 2017 edition of ASVspoof series. The bonafide utterances were taken from a subset of RedDots dataset – which is a dataset for speaker verification collected under wild varied acoustic conditions.

The ASVspoof 2017 dataset has two different versions: version 1.0 and version 2.0. The version 1.0 was used during the official challenge evaluation in 2017. Post evaluation data anomalies were identified that showed biased model decisions, which eventually led to the release of version 2.0 dataset. The 2019 edition combined both the replay spoofing attacks (Physical access - PA) and text-to-speech and voice-conversion attack conditions (so called Logical access – LA) and released the LA and PA datasets respectively. Also, post 2019 challenge evaluation a real replayed utterances - a small subset of real replayed speech utterances were also made publicly available to perform research on replay spoofing attacks. During the latest edition of ASVspoof evaluation, the ASVspoof 2021, no training data were released to the challenge participants. The participants were required to use the ASVspoof 2019 training and development dataset to train and tune their anti-spoofing model parameters. Only a fresh set of evaluation set was released to the participants. For more details on this please see this.

Equal error rate metric (EER) was the primary (and the only metric) used to evaluate anti-spoofing performance during the 2015 and 2017 ASVspoof evaluation. However, in the 2019 edition EER was the secondary metric used where a new metric called tandem detection cost function that jointly optimises the performance of ASV and anti-spoofing system was used to evaluate the model performances of the challenge participants.

In the next article, Bhusan Chettri will be discussing more about different corpuses and the evaluation metrics used in voice anti-spoofing research.

References

[1] Bhusan Chettri scholar  

[2] M. Sahidullah et. al. Introduction to Voice Presentation Attack Detection and Recent Advances, 2019.

[3]. Bhusan Chettri. Voice biometric system security: Design and analysis of countermeasures for replay attacks. PhD thesis, Queen Mary University of London, August 2020.

[4] ASVspoof: The automatic speaker verification spoofing and countermeasures challenge website.

Bhusan Chettri London  | Bhusan Chettri Queen Mary University of London  | Dr. Bhusan Chettri | Bhusan Chettri social | Bhusan Chettri Research

Bhusan Chettri Visits a Child Orphanage Home

 

Bhusan Chettri is a professional personality, who has made India feel proud as by contributing massively to the research based on sound and speech technology. Coming from a small town in the South Sikkim district, his journey was not easy. He overcame challenges more than one and did not allow them impede his journey from a small hilly deprived location in India to London, UK where he completed his PhD in one of the most prestigious research universities in the world. This was all made possible because of his determination and indomitable spirit attitude throughout. His kind personality, positive attitude, and professionalism have made him a successful personality. Recently, he is working as a researcher in AI and Machine learning for audio and speech.

Having gained a lot of reputation in the professional industry, he believes it’s time to give back to the    society, sharing happiness and love towards needy people. This is why he started investing some of his valued time for the welfare of the society through volunteering activities. He has opted for visiting a child orphanage home. 

Bhusan Chettri visited a child orphanage home in Assam. This child orphanage home accommodates children of different ages who either have lost their families in natural calamities or have no family at all. He is a guy, who can deeply understand the emotions and needs of children who have no parents. He knows how hard it is to spend life without loved ones. This feeling and emotion convinced him to visit this place.

He felt very lucky and blessed to invest his time in between such lovely and needy children. He helped them by donating some essential supplies and food items to more than 25 children. There are both boys and girls ranging from as small as 3 years up until 18 years accommodated in this child shelter home. He was contented to see them live together like a family. They have created their own little world within the boundaries of this home. After getting gifts from Bhusan Chettri, they all had big smiles on their faces. They thanked him for this sweet gesture of love and care. They also liked to play and shared some precious moments with him. This one-day visit to a child orphanage home by him was respected.

Not only this, Bhusan Chettri also had a thorough discussion with the management of the place to understand the working and organization. He further investigated the immediate and future needs of the children and ensured every help possible from his end towards joyful living and healthy growth and development of the kids. Furthermore he looks forward to doing something profound for such helpless people including children, youngsters, or senior citizens who are ill-fated. He is planning to take a part in many other social activities in the future and contribute to the development of the society, as a result, assisting the country to grow in every sense.

References

[1] Tumblr: https://chettribhusan.tumblr.com/post/685306610792464384/bhusan-chettri-visits-old-age-home-with-a-great

[2] Diigo: https://www.diigo.com/item/image/9gcf9/tcke

Bhusan Chettri London | Bhusan Chettri Queen Mary University of London | Bhusan Chettri PhD | Dr. Bhusan Chettri | Bhusan Chettri Research

Automatic Speaker Recognition And AI

Automatic Speaker Recognition And AI

Bhusan Chettri Gives An Overview Of The Technology Behind Voice Authentication Using Computer

What is Automatic Speaker Recognition?

Automatic Speaker Recognition is the task of recognizing humans through their voice by using a computer. Automatic Speaker Recognition generally comprises of two tasks: Speaker identification and Speaker verification. Speaker identification involves finding the correct person from a given pool of known speakers or voices. A speaker identification usually comprises of a set of N speakers who are already registered in the system and these N speakers can only have access to the system. Speaker verification on the other hand involves verifying whether a person is who he/she claims to be using their voice sample.

These systems are further classified into two categories depending upon the level of user cooperation: (1) Text dependent (2) Text independent. In text dependent application, the system has prior knowledge of the spoken text and therefore expects same utterance during test time (or deployment phase). For example, pass-phrase such as "My voice is my password" will be used both during speaker enrollment (registration) and during deployment (when the system is running). On the contrary, in text independent systems there is no prior knowledge about the lexical contents, and therefore these systems are much more complex than text dependent ones.

So how does the speaker verification algorithm work? How are they trained and deployed?

Well, in order to build automatic speaker recognition systems first thing we need is data. Big amount of speech data collected from hundreds and thousands of speakers spoken across varied acoustic conditions. It would be nice to have pictures illustrating the methodology as pictures speak louder than thousand words. The block diagram shown below summarises a typical speaker verification system. It consists of speaker enrollment phase (Fig a) and speaker verification phase (Fig b). The role of a feature extraction module is to transform the raw speech signal into some representation (features) that retains speaker specific attributes useful to the downstream components in building speaker models. The enrollment phase comprises offline and online modes of building models. During the offline mode, background models are trained on features computed from a large speech collection representing a diverse population of speakers. The online phase comprises building a target speaker model using features computed from target speaker’s speech. Usually, training the target speaker model from scratch is avoided because learning reliable model parameters requires a sufficiently large amount of speech data,which is usually not available for every individual speaker. To overcome this, the parameters of a pretrained background model representing the speaker population are adapted using the speaker data yielding a reliable speaker model estimate. During the speaker verification phase, for a given test speech utterance, a claimed speaker’s model and the background model (representing the world of all other possible speakers) is used to derive a confidence score. The decision logic module then makes a binary decision: it either accepts the claimed identity as a genuine speaker or rejects it as an impostor based on some decision threshold.

 

(a) Speaker enrollment phase. The goal here is to build speaker specific models by adapting a background model which is trained on a large speech database.

 

(b) Speaker verification phase. For a given speech utterance the system obtains a verification score and makes a decision whether to accept or reject the claimed identity.

How has the state-of-the-art changed and driven by big-data and AI?

Bhusan Chettri explains that there has been a big paradigm shift in the way we build these systems. To bring clarity on this, Mr Bhusan Chettri summarises the recent advancement in state-of-the-art in two broad categories. (1) Traditional approaches (2) Deep learning (and Big data) approaches.

Traditional methods. By traditional methods he refers to approaches driven by a Gaussian mixture model - universal background model (GMM-UBM) that were adopted in the ASV literature until deep learning techniques became popular in the field. Mel-frequency cepstral coefficients (MFCCs) were popular frame-level feature representations used in speaker verification. Using short-term MFCC feature vectors, utterance level features such as i-vectors are often derived which have shown state-of-the-art performance in speaker verification. The background models such as the Universal back-ground model (UBM) and total variability (T) matrix are learned in an offline phase using a large collection of speech data. The UBM and T matrix are used in computing i-vector (this is just a fixed length vector representing a variable-length speech utterance) representations. The training process involves learning model (target or background) parameters from training data. As for modelling techniques, vector quantization (VQ) was one of the earliest approaches used to represent a speaker, after which Gaussian mixture models (GMMs), an extension to VQ methods, and Support vector machines became popular methods for speaker modelling. The traditional approach also includes training an i-vector extractor (GMM-UBM, T-matrix) on MFCCs and using a probabilistic linear discriminant analysis (PLDA) backend for scoring.

Deep learning methods. In deep learning based approaches for ASV, features are often learned in a data-driven manner directly from the raw speech signal or from some intermediate speech representations such as filter bank energies. Handcrafted features, for example MFCCs, are often used as input to train deep neural network (DNN) based ASV systems. Features learned from DNNs are often used to build traditional ASV systems. Researchers have used the output from the penultimate layer of a pre-trained DNN as features to train a traditional i-vector PLDA setup (replacing i-vectors with DNN features). Extracting bottleneck features (output from a hidden layer with a relatively small number of units) from a DNN to train a GMM-UBM system which uses the log-likelihood ratio as scoring is also used commonly. Utterance-level discriminative features, so called embeddings extracted from pre-trained DNNs have become popular recently, demonstrating good results. End-to-end modelling approaches have also been extensively studied in speaker verification showing promising results. In this setting, both feature learning and model training are jointly optimised from the raw speech input. A wide range of neural architectures have been studied for speaker verification. This includes feed forward neural networks, commonly referred as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks, and attention models. Training background models in deep learning approaches can be thought of as a pretrainng phase where network parameters are trained on a large dataset. Speaker models are then derived by adapting the pretrained model parameters using speaker specific data, much like the same way a traditional GMM-UBM system operates.

What are the applications of these technology?

These can be used across wide-range of domains such as (a) access control - voice based access control systems (b) in banking applications for authenticating a transaction (c) personalisation: in mobile devices, lock/unlock vehicle door (engine start/off) based on specific user etc.

Are they safe and secure? Are they prone to any manipulation when they are deployed?

Bhusan Chettri further explains that although the current advancement in algorithms with the aid of big data have shown remarkable state-of-the-art results, these systems are not 100% secure. They are prone to spoofing attacks where an attacker aims to manipulate voice to sound like registered user and gain illegitimate access to their system. A significant amount of research is being promoted by the ASV community recently along this direction.

References

[1] Google scholar: https://scholar.google.co.uk/citations?user=Ht6H2WgAAAAJ&hl=en
[2] Thesis: https://theses.eurasip.org/theses/866/voice-biometric-system-security-design-and/

Bhusan Chettri London | Bhusan Chettri Queen Mary University of London | Bhusan Chettri PhD | Dr. Bhusan Chettri | Bhusan Chettri Research

Bhusan Chettri

Dr. Bhusan Chettri works as a Consultant and Researcher in AI and Machine Learning with focus on explainable AI. PhD in Machine Learning for Voice Technology from QMUL, UK.

Education:

• PhD in AI and Machine Learning for Fake Voice detection from Queen Mary University (QMUL), UK in 2020.
  
• MSc in Speech and Language Processing from University of Sheffield with a distinction in 2014 (topper of the batch).
• M.Tech in Computer Science and Engineering from Sikkim Manipal University, Sikkim, India. 2009.
• B.Tech in Computer Science and Engineering from Sikkim Manipal University, Sikkim, India, 2005.

Work Experience:

• Oct 2021 - present: AI researcher and consultant, Borac Solutions, India.
  
• Sept 2020 - May 2021: Lecturer, School of Electronic Engineering and Computer Science, Queen Mary University of London
  
• April 2020 - Feb 2021: Post-doctoral researcher, School of Computing, University of Eastern Finland, Finland
• May 2019 - Dec 2019: Project researcher, School of Computing, University of Eastern Finland, Finland
• Oct 2016 - Aug 2020: PhD researcher, School of Electronic Engineering and Computer Science, Queen Mary University of London
  
• Jan 2018 - May 2020: Part-time Lecturer, School of Electronic Engineering and Computer Science, Queen Mary University of London
  
• Oct 2014 – July 2016: Research Assistant in Speech Technology at Speech and Hearing (SPandH) Research group at the Sheffield University.
• Aug 2006 – Aug 2013: Lecturing in Computer Science department, Sikkim Manipal University, Sikkim, India as an Assistant Professor during August 2006 – August 2010 and as an Associate Professor during September 2010 – September 2013
  

 

Contact Information:

Data Analytics and AI Lab, Borac Solutions, India.

 e-mail: info@boracsolutions.com