Peng-Jen Chen inspires students at NTU to boldly venture into different subjects, research projects, and internships. He urges that embarking on such journeys would help them discover their passion and the significance of their professional knowledge. Such experiences can offer invaluable help for students to define their goals and increase their motivation to study.
Last year, Mark Zuckerberg, the CEO of Meta, demonstrated the company’s latest translation technology, a system that translates speech-to-speech in real-time between English and multiple languages, making it a real game-changer in the field of translation technology. In a video Zuckerberg posted on Facebook, he introduced Taiwanese engineer Peng-Jen Chen and they spoke in their respective language, English and Hokkien, through the speech-to-speech translation system. The successful translation of their conversation in real time showcased the creation of the world’s first-ever speech-to-speech translation system for unwritten languages, such as Hokkien.
The mastermind behind this technology is Taiwanese engineer Peng-Jen Chen, an alumnus of NTU’s Department of Computer Sciences and Information Engineering (NTU CSIE). Chen shared his journey of discovering his passion for machine learning and speech translation as a college student, which has led to his current role in developing Meta’s Universal Speech Translator (UST). He recalled that at college he was deeply interested in AI and machine learning and besides taking course in these areas he joined his professor’s laboratory.
A speech-to-speech translation system for unwritten language
Meta UST uses speech recognition, text translation, and speech synthesis to translate speech to speech. Unlike most speech translation systems that require a corresponding written text in the source language, Meta’s UST doesn’t require any written text, making it the first speech-to-speech translation system developed for an unwritten language. Chen had originally intended to develop a speech system that would help people overcome communication barriers and make translations increasingly real-time.
Using Taiwanese soap opera dialogues as the training material
Chen chose Hokkien to be the first language to test the system because it shares many translation difficulties in common with other unwritten languages. Its writing system is not commonly used or fully standardized, making it extremely difficult to find suitable training materials. To train Meta’s UST, the research team processed 30 thousand hours of Taiwanese telenovelas, 80 thousand hours of English electronic books, and 100 hours of English to Hokkien speech translation. The result proved that the system can and does translate Hokkien seamlessly into English in real-time, offering a solution to the speech translation of unwritten languages and significantly contributing to its promotion.
The demonstration video of Meta Universal Speech Translator which Meta Founder and CEO Mark Zuckerberg released on his Facebook page.
Chen urges students to boldly explore, intern, and collaborate with peers on campus; such experiences may prove crucial in shaping their future careers. (Photo credit: Peng-Jen Chen)
Chen discovered his interests and met many friends by participating in student club activities, such as coffee making, cycling, and table tennis. (Photo credit: Peng-Jen Chen)
