SPEECH-TO-SPEECH TRANSLATION BETWEEN UNTRANSCRIBED UNKNOWN LANGUAGES

Andros Tjandra, Sakriani Sakti, Satoshi Nakamura

Email: andros.tjandra.ai6@is.naist.jp

Tested on Google Chrome


ABSTRACT

In this paper, we explore a method for training speech-to-speech translation tasks without any transcription or linguistic supervision. Our proposed method consists of two steps: First, we train and generate discrete representation with unsupervised term discovery with a discrete quantized autoencoder. Second, we train a sequence-to-sequence model that directly maps the source language speech to the target language’s discrete representation. Our proposed method can directly generate target speech without any auxiliary or pre-training steps with a source or target transcription. To the best of our knowledge, this is the first work that performed pure speech-to-speech translation between untranscribed unknown languages.

ARXIV URL: https://arxiv.org/abs/1910.00795

Translation result from [ France , Japanese ] to English

Source JA Source FR Groundtruth EN FR-EN Speech2Code JA-EN Speech2Code FR-EN Topline JA-EN Topline
French (source): C'est urgent.
Japanese (source): それは緊急です。
English (target): that is urgent
French (source): Est-ce que je pourrais utiliser le téléphone?
Japanese (source): 電話をお借りしてもいいですか。
English (target): may i use the phone
French (source): C'est la première fois que vous venez ici?
Japanese (source): こちらへは初めて。
English (target): this your first time here
French (source): Comment puis-je faire un appel direct longue distance depuis ce poste?
Japanese (source): この電話で直通の市外電話をかけるにはどうすればよいのですか。
English (target): how can i make a direct long distance call from this phone
French (source): J'ai eu un accident.
Japanese (source): 事故を起こしてしまいました。
English (target): i had an accident
French (source): Mon appareil photo m'a été volé dans ma chambre.
Japanese (source): 部屋からカメラが盗まれました。
English (target): my camera was stolen from my room
French (source): De quel quai est-ce que ce train part?
Japanese (source): この列車は何番線から出ますか。
English (target): which platform does this train leave from
French (source): Quand est-ce que ce serait le plus convenable?
Japanese (source): いつが一番ご都合がいいですか。
English (target): when would be most convenient
French (source): Veuillez payer à la table.
Japanese (source): テーブルで払って下さい。
English (target): please pay at the table
French (source): La télévision ne marche pas.
Japanese (source): テレビが映りません。
English (target): the tv does not work
French (source): Combien coûte un billet aller-retour pour Boston?
Japanese (source): ボストンまで往復運賃はいくらですか。
English (target): how much is a roundtrip ticket to boston
French (source): C'était un plaisir de vous rencontrer aujourd'hui.
Japanese (source): きょうはあなたに会えて楽しかったです。
English (target): it was nice meeting you today
French (source): Les frais de réparation sont de quatre-vingt-dix-neuf dollars.
Japanese (source): 直し代九十九ドルいただきます。
English (target): the repair charge is ninetynine dollars
French (source): J'aimerais envoyer ce paquet au Japon. Quel est le moyen le plus rapide?
Japanese (source): この小包を日本に送りたいのですが、一番はやい方法は何ですか。
English (target): i would like to send this package to japan what is the fastest way
French (source): Quelle cuisson voulez-vous?
Japanese (source): 焼き方は、どうなさいますか。
English (target): how would you like it cooked
French (source): Non. J'ai dit trente dollars.
Japanese (source): いえ。 三十ドルと言いました。
English (target): no i said thirty dollars
French (source): A point, s'il vous plaît.
Japanese (source): ミディアムでお願いします。
English (target): medium please
French (source): Non, celui à côté du sac rouge.
Japanese (source): いいえ、赤いバッグの隣にある物です。
English (target): no the one next to the red bag
French (source): Faites le mien extra épicé, s'il vous plaît.
Japanese (source): 極辛にしてください。
English (target): make mine extra spicy please
French (source): Avez-vous un rendez-vous?
Japanese (source): 予約をなさっていますか。
English (target): do you have an appointment