Is VR ready for language learning?

Today we have a guest post by Ramon Brena of Avalinguo

When we see the acronym “VR”, we immediately think of headsets and gaming, as well as of fantasy dystopian worlds like in Spielberg’s Ready Player One, where everything, I mean everything, is done in VR.

Well, perhaps not everything in the real world, but could we learn to speak languages in VR? Now that travel is not possible because of COVID-19, could we engage in lively conversations with Parisians in a VR Montmartre pub?

Put like this, we have to answer “No”. To begin with, Paris is one of the worst places to practice French with the locals (as it was commented by Benny Lewis in the Language Hacking Podcast): if you want to speak French with the locals, you better go to a small town in France. Believe me on this, I lived for 6 years in France.

Then, a more basic question, Who are we going to talk to in VR? There are two broad answers to this question, and each of them takes a completely different approach to interaction in VR.

We call the first approach “simulation-based”. As you know, VR has been applied for educational purposes using the VR capabilities for simulating a world, could it be the bloodstream inside arteries in the body, or the working of complex mechanical contraptions, or… the situation inside a taxi in a foreign city.

The cab example was taken from the Mondly-VR app (headset plug-in for the Mondly app), which I bought for my Oculus. They use the synthetic worlds construction capabilities of VR to show you very compelling city views taken from inside the cab. The cab driver turns to you like expecting to have your instructions.

After that, things become a bit awkward. You have to interact with the driver using one of the options written for you on-screen and… Wait a moment, do you have to answer using predefined scripts? Here the magic starts to vanish. Dialogs are not natural, speech recognition leaves a lot to desire, and in general, the interaction doesn’t feel at all like a human conversation where only you know what you want to talk about.

To be fair, Mondly does a good job at the virtual scenario construction, but the interaction engagement wears off very quickly as the novelty fades out. Mondly-VR is available for several VR headsets and builds on top of its popular smartphone app. You can learn the language lessons on the smartphone alone, but of course, in the little cellphone screen, Mondly graphics look far less impressive than with the headset.

We call the second approach for language practice with VR “interaction-based” and it consists of putting in a virtual world the avatars of several humans. In this case, the language practice aspect is taken care of completely by those humans, who converse as they want. In this approach, the interaction is, of course, more natural just because it is done by humans, but then other less obvious problems arise.

There are several general VR platforms that can be used to have language practice conversations, like Mozilla Hubs (web browser-based), High Fidelity (audio only, I have used it for Meetup language gatherings), AltspaceVR (acquired by Microsoft some years ago), and others, which are not specifically intended to be used for language learning, but to some extent could.

I’m rather going to take as an example of this second approach the app Avalinguo, which is specifically intended for language learning (disclaimer: I’m the CEO of Avalinguo). I know, you are going to ask: In which headsets can I run Avalinguo? Right now in none. Avalinguo is a cellphone app.

Screenshot of the Avalinguo app

The value of VR in Avalinguo is, first of all, to replace the face and body of the participants by avatars in order to give privacy. You know, talking is the most stressful language learning activity, and research studies have found that many self-conscious or shy people find it difficult to deal with. Some beginners feel judged or stupid when babbling in the language they are starting to learn. Linguists such as Luca Lampariello have proposed to change the attitude and embrace making mistakes, which is a good idea, but we propose to reduce the stress of the situation in the first place, by using avatars instead of showing your babbling face.

Another advantage of VR in Avalinguo (not yet currently released) is that role-playing games, which is one of the best language-learning practice activities, is greatly enhanced by the use of avatars already using a costume like a nurse, travel agent, bell boy, you name it. Other mini-game props such as roulette can be easily implemented, and there are hundreds of ready-to-use graphic resources to plugin.

Now, the distinction between the “simulation” approach and the “interaction” one is not clear cut, because, for instance, the use of costumed-avatars and props is a form of simulation. There is plenty of room for “simulation-interactive” hybrid approaches. The tricky aspect is to build it in such a way that it makes sense from an educational point of view, while at the same time making it entertaining and coherent.

So, in the end, the question is not if VR is ready for language-learning, but rather if language-learning is ready for VR, isn’t it?

Upload a language straight to your brain

Have you ever tried to learn a language?

Maybe you’ve struggled with a language for years and never really got the hang of it.

Ever wished you could simply wave a magic wand and be able to speak another language fluently?

The lingoboffins at the secret Omniglot laboratory have been working on this problem for a long time, and believe that they have finally found a solution that will work for everybody.

It was announced today that a revolutionary new app will soon be available that will enable you to upload a whole new language directly into your brain.

apps

No more spending years learning vocabulary, trying to get to grips with grammar, and fumbling with the pronunciation. All you need to do is download the app, tune it to your brain, and upload a language. Then you will be able to understand and speak* that language almost immediately**.

*This app does not give you the ability to read and write in another language. We are working on another app which will do that.

**The actually time it takes to tune the app to your brain, and for the language to settle in, may vary from a few hours to several months, or even years.

Terms and conditions apply. This app is not currently available anywhere, but keep checking back, and one day it might be.

A Review of Mondly

Mondly screenshot

Recently I was approached by the people behind Mondly, a newish language-learning app, who wanted to advertise on Omniglot.

Before agreeing to advertise such apps, I usually try them out for myself to see if I can recommend them. So I’m currently learning Czech on Mondly. It’s a language I’ve studied on and off for years, so I have a basic knowledge of it. Mondly lets you start at beginning, or at an intermediate or advanced level – I chose the intermediate level.

The app presents you with a daily lesson, and you can choose to study other lessons as well. These have various themes such as family, travel, food & drink, and so on. There is also a chatbot, with which you can have conversations using the words and phrases you’ve learnt, and at the end of each week there’s a quiz.

It looks good, the interface works well for me, the lessons teach you a manageable number of words and phrases, and show you how to put them into sentences, and the audio is clear.

33 different languages are available on Mondly, including major European languages, and Japanese, Korean, Chinese, Turkish, Arabic, Persian, Hebrew, Vietnamese, Hindi, Indonesian and Thai.

You can get lifetime access to all of those languages for just US$99 (normal price US$1,999.99).

Disclaimer – although I will receive a commission if you subscribe to Mondly after clicking on the links in this post, I do think this app is worth a try.

Omniglot app

There is now an Omniglot app for Andriod developed by علي الساعدي (Ali al-saaedi Ali shirpaz) from Iraq. It only works online at the moment, and is available here.

Any comments or suggestions on how it might be improved are welcome.

Many thanks to علي الساعدي for doing this.

Social Media Helps Threatened Language Threatened by Social Media

Today we have a guest post by Alissa Stern of BASAbali.org

On the eve of the Balinese holy day of knowledge, learning, and wisdom (Saraswati Day), a free innovative multi-media Balinese-Indonesian-English wiki dictionary was just made available to people in Bali and throughout the world.

The wiki uses social media to save Balinese, a language threatened by, among other things, social media.

In recent years, Balinese has dwindled down to use by only about a quarter of native Balinese, the result of globalization, nationalization, and social media taking its usual toll on a minority language. With Balinese, where speakers rely on who they are, who they are speaking to, and what they are speaking about to choose the right level of words, the faceless internet presents a serious problem, encouraging Balinese posters to use the national – and status neutral – Indonesian rather than make a mistake with Balinese.

But with the new Wiki, social media is being use to re-energize Balinese by promoting pride in the language through an international web presence and by providing a tool for anyone with internet access – which these days is large portions of the island – to contribute to its well being and benefit from its information.

Nala Antara, Chair of the Linguist team from Badan Pembina Bahasa Aksara dan Sastra, Universitas Udayana, Universitats Pendidikan Ganesha and other universities within and outside of Bali who will oversee and edit the Wiki explains: “Technology will be our bridge to the future. The wiki Balinese-English-Indonesian dictionary will help everyone in Bali learn and speak Balinese alongside Indonesian, so that we two strong languages co-existing: the language of our people and the language of our nation. The wiki allows the people of Bali to actively take part in this project to take pride in their participation.”

Ayu Mandala from BASAbali which is working to connect the Linguist team with the Balinese public says “with this wiki, we can make the Balinese language well known throughout Bali and throughout the world. Wiki technology gives free access to everyone and provides an opportunity for the public to be part of the action.”

A small firm called TinyMighty, based in a remote part of Spain, which also has a threatened language, created the wiki interface. It is being supported by a Kickstarter campaign, using the same crowdsourcing for funding as the wiki uses crowdsourcing for knowledge. The wiki is particularly unique in being able to handle the different registers of Balinese – something unique to the Balinese language – but it also gives real life examples of word usage from Balinese literature, newspapers and other media, and handle the old – the endangered Balinese script – and the new – youtube videos of native speakers.

Alissa Stern of BASAbali“>BASAbali hopes that the Wiki will not only inspire people to learn and use Balinese, but that Balinese can be a model how other threatened languages in the rest of the world might benefit from a collaboration of expert linguists and the general public.

International Conference ICT for Language Learning

Here is a conference that might be of interest to you: International Conference ICT for Language Learning.

It takes place in Florence in Italy from 13-14 November 2014, and will bring together teachers, researchers, practitioners and project managers from all over the world to share findings, expertise and experience about integrating innovative technologies and solutions into language teaching and learning.

The conference will also be an opportunity for sharing results achieved in language learning projects funded by the European Commission and by other sources.

Topic areas are intentionally broad to encourage a wide range of backgrounds, ideas, and discussions.

They are currently calling for papers, and the deadline for submission is the 4th August.

Subtitles

The other day I discovered how to add subtitles and speech bubbles to my videos on YouTube. I’ve added them to my latest video in Spanish:

So you can now have the speech bubbles with the dialogue in Spanish, the subtitles in English, or both, or neither. It was quite a fiddly and laborious process to add the speech bubbles, but relatively straightforward to add the subtitles as I used the timing from the speech bubbles (annotations).

Today I saw a post on Fluent in 3 months about this very topic with suggestions on how to do it more easily. This got me thinking and I realised that another way to add the subtitles is to record the sound of a video using Audacity, and to get the timing of each bit of speech from there. You have to have a transcript of the dialogue first, of course, though that wasn’t a problem as I always prepare such transcripts when I make the videos.

I used this method to add subtitles to my video about eel-infested Austrian hovercrafts, and it worked well. I’ve also started adding speech bubbles, but haven’t quite finished that yet.

Half the dialogue is in English and half in German, and the subtitles are in English for the German bits and in German for the English bits. I thought it would be a useful exercise for me to try to translate the English bits into German. If you spot any mistakes, please let me know.

This is what the file for the subtitles (captions) looks like:

0:00:00.0,0:00:00.4
Hello

0:00:00.8,0:00:01.3
Hallo

0:00:01.6,0:00:02.7
How are you?

0:00:03.1,0:00:05.1
Es tut mir leid ich verstehe nicht.

0:00:05.5,0:00:07.0
Spechen Sie über ein Gatter?

0:00:07.4,0:00:10.8
Sprechen. Sie. Englisch?

0:00:11.2,0:00:15.2
Yes, of course. Do. You. Speak. German?

0:00:15.6,0:00:20.1
Äh, meinen Sie damit, daß Sie Englisch sprechen,
und fragen Sie mich, wenn ich Deutsch spreche?

0:00:20.6,0:00:23.4
Yes. You can understand a little German?

0:00:23.7,0:00:28.5
Ja, ich habe in der Schule Deutsch gelernt und
kann ein wenig verstehen, aber ich kann nicht sprechen.

0:00:28.8,0:00:31.8
Seit sehr viele Jahre habe ich es nicht benutzt
und ich bin etwas aus der Übung,

0:00:31.9,0:00:33.8
aber jetzt fängt es zurückzukommen.

0:00:34.1,0:00:35.9
I understand. What’s your name?

0:00:36.3,0:00:39.5
Mein Name? Ich heise Jane. Wie heißen Sie?

0:00:39.9,0:00:42.1
My name is Klaus. Where are you from?

0:00:42.5,0:00:44.9
Ich komme aus Birmingham. Und Sie?

0:00:45.3,0:00:48.0
I’m from Vienna. What do you do?

0:00:48.3,0:00:50.0
Ich bin Bibliothekarin, und Sie?

0:00:50.4,0:00:53.2
I’m a Danube hovercraft company captain.

0:00:53.5,0:00:55.6
Mann, das ist ein ziemlicher Zungenbrecher!

0:00:56.0,0:00:59.5
Genießen Sie arbeit als
Donauluftkissenfahrzeuggesellschaftskapitän
zu tun?

0:00:59.8,0:01:01.4
Yes. I enjoy it very much.

0:01:01.8,0:01:04.1
But the hovercrafts are often full of eels.

0:01:04.7,0:01:08.4
Ja, ich weiß. Diese verteufelten Aale kommen überall hin.

0:01:08.8,0:01:11.4
Yes. It was nice talking to you.

0:01:11.7,0:01:15.0
I must get back to the hovercrafts. Goodbye.

0:01:15.2,0:01:17.7
Ja, es war schön mit Ihr zu reden.

0:01:18.1,0:01:20.7
Viel Glück mit dieser Aal-verseuchten luftkissenfahrzeugen.

0:01:21.2,0:01:21.7
Auf Wiedersehen.

Txtng n N’ko

There’s an interesting article in the New York Times about how it is now possible to send text messages and emails in N’Ko, an alphabet invented in 1949 to write Mande languages of Guinea, Mali and Ivory Coast. Thanks to various iPhone apps and other software use of this alphabet is increasing. It also helps that N’Ko is included in Unicodehere are some examples.

N’ko Unicode fonts are available here and here.

Learning a Language with Flashcards

Today we have a guest post by Andrew Cohen.

Having been a reader of the Omniglot blog for quite some time now, I have seen a lot of discussion about the usefulness of flashcards for help learning a language – particularly as a tool for practicing vocabulary and verb conjugation. Spaced-repetition systems (SRSs) like Anki seem to have gained traction as the preferred vocabulary study method among the language-learning blogosphere.

Yet despite their popularity, SRSs have so far mostly remained limited to vocabulary memorization. I find it surprising that no SRS manager has yet attempted to develop a way for people to actually learn a language from scratch using audio flashcards. It’s as if language theorists are afraid of innovating too much in the flashcard realm, lest they be accused of advocating diminished human interaction.

That’s why I created Brainscape. We’re a web & mobile “smart flashcards” platform that has developed a fully flashcard-based language learning methodology called Intelligent Cumulative Exposure (ICE). (See our 30-page white paper (PDF) about the cognitive science behind ICE.) We basically took the often-overlooked body of research suggesting that adult language learners benefit most from practicing output, and we found a way for the learner to systematically practice output of full sentences.

Intelligent Cumulative Exposure is based on the following tenets:

(1) Use translation to solicit the learner to generate each foreign-language sentence in the form of a flashcard;
(2) Play a natural-sounding native recording of the full sentence on the back of the flashcard
(3) Introduce only one new concept (k + 1) per sentence, beginning at the most basic sentence possible;
(4) Provide grammatical annotation in the learner’s native language where appropriate; and
(5) Repeat previously seen sentence-generation exercises in a spaced repetition pattern driven by the learner’s own confidence levels.

We’ve spent the past two years carefully sequencing a curriculum that applies these principles on both the web and iPhone – and we are finally releasing the product today! Brainscape Spanish starts from the most basic Spanish concepts for novices, toward much more complex words, grammar, and phrases for the advanced user. The app is currently available to use for free on our website (for a limited time) and for $40 on the iTunes App Store. It contains over 6,000 flashcards and will continue to improve based on user feedback over the coming months.

Please check out the app for yourself – and tell us what you think! And thanks again to Simon for letting me share this new innovation here on the Omniglot.

Tibetan mobiles

A mobile phone which can send and receive text messages in Tibetan has been developed in China, according to this report. Such phones have apparently existed for a few years, but the new phone is a bit different as it can recognise handwritten input in Tibetan.

A useful Tibetan-related site I came across today is a English-Tibetan dictionary, which displays the Tibetan words in Wylie transliteration and in the Tibetan script.

Are any of you learning Tibetan?