ALBAYZIN 2014 – AUDIO SEGMENTATION EVALUATION
The Albayzin 2014 Audio Segmentation Evaluation organized by Vivolab research group (http://www.vivolab.es/) from the University of Zaragoza will be conducted as part of Iberspeech 2014 ( "VIII Jornadas en Tecnología del Habla"and IV Iberian SLTech Workshop) (http://iberspeech2014.ulpgc.es/) organized by the University of Las Palmas de Gran Canaria and supported by the Spanish Thematic Network on Speech Technology (RTTH) and the ISCA Special Interest Group on Iberian Languages (SIG-IL) that will take part in Las Palmas de Gran Canaria (Spain), 19-21 November 2014.
This evaluation consists of segmenting and labeling broadcast audio documents to indicate which segments contain speech, music and/or noise. Unlike previous editions, this evaluation aims at providing an experimental framework for segmentation systems across different databases that can be merged or even overlapped increasing the difficulty from last editions. Therefore, the main goal is to test the robustness of the participating systems against different acoustic contexts.
Data
The database for this evaluation is a combination and fusion of several databases:
The Catalan broadcast news database from the 3/24 TV channel proposed for the 2010 Albayzin Audio Segmentation Evaluation, the Aragon Radio database from the Corporacion Aragonesa de Radio y Television (CARTV) which was used for the 2012 Albayzin Audio Segmentation Evaluation, and sounds extracted from different sources (Freesound.org, HuCorpus from Ohio State University, ...). This sounds will be merged with segments from the 3/24 TV and Aragon Radio databases. All the data will be supplied in PCM format, mono, little endian 16 bit resolution, and 16 kHz sampling frequency.
Metrics
As in the NIST RT Diarization evaluations, to measure the performance of the proposed systems, the segmentation error score (SER) will be computed as the fraction of class time that is not correctly attributed to that specific class (speech, noise or music). This score will be computed over the entire file to be processed; including regions where more than one class is present (overlap regions). This score will be the ratio of the overall segmentation error time to the sum of the durations of the segments that are assigned to each class in the file.
Registration
All Research groups interested in participating in this evaluation must send (before July 15, 2014) an email to This email address is being protected from spambots. You need JavaScript enabled to view it.
, This email address is being protected from spambots. You need JavaScript enabled to view it.
(with CC to the Chairs of Iberspeech 2014, This email address is being protected from spambots. You need JavaScript enabled to view it.
), Indicating the following Information:
RESEARCH GROUP:
INSTITUTION:
CONTACT PERSON:
E-MAIL:
Evaluation Plan, Further Information and Updates
Schedule (Tentative)
· June 30, 2014: Release of the training and development data.
· July 15, 2014: Registration deadline.
· September 3, 2014: Release of the evaluation data.
· September 30, 2014: Deadline for submission of results and system descriptions.
· October 15, 2014: Results distribute to the participants.
· Iberspeech 2014 workshop (Las Palmas, November 19-21, 2014): Official public publication of the results.
For further information or to download the 2014 Albayzin Audio Segmentation Evaluation Plan in pdf format here
Contact
Alfonso Ortega
Aragon Institute of Engineering Research (I3A). University of Zaragoza.
Ada Byron Building, Office 2.05.
María de Luna 1 50018 Zaragoza, Spain.
Phone: +34-976762363.
Fax: +34-976762111.
e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
