The Spotify Lyrics feature was released in late 2021. It is useful for songs languages that you are already familiar with, as a clear reference to what is being played. However, it is not the case if you’re reading lyrics in a foreign language. As a super active Spotify user listening to songs that aren't in my mother tongue all the time, sometimes I find the feature a bit half a**. It only is great if you know the language inside out, otherwise, it doesn't even give you transliteration so you can sing along.

Demo

Motivation

Trying to accelerate my Thai learning, I have this huge urge to enhance the lyrics experience on Spotify. I listen to Thai songs a lot to get immersed in a Thai environment without physically being in Thailand. It has been helping me to pick up tons of vocabs enjoyably. Usually I’d listen to a song (thousand times if good), and get those melodies in. If it’s good, I’ll look up the lyrics, and then try to do a mapping of words that I’d already recognise through listening, and get translations of those. so the flow pretty much has been:

Find lyrics
Google Translate
Listen to the songs and try to sync
If there's a particular word that I don't know, copy and paste it to google translate again to get the individual translation.

The fun part of doing a project like this is that you always can discover goodies, maybe its web API, or some random CSS that you never know that existed (for ages).

Segmenting sentences in the browser with `Intl.Segmenter`

For languages that are written with no spaces between words, such as Chinese, Japanese, Thai etc, (apparently this writing style is called “Scriptio continua”), before getting anywhere near fluency in reading, it is a bit like a nightmare to guess where a particular word ends. There are a lot of rules that define the combination of characters. But when you double-click a word of this writing style in browser, it magically does the segmenting for you by highlighting the full “word”.

For example:

Chinese: 有效率的享受工作 Double-clicking 享 or 受 → highlights 享受
Thai: จากวันนั้นที่เธอได้ไปจากฉัน Double-clicking จ highlights จาก

I never really tried to understand how this "magical" double-click segmenting feature works on the browser. And actually, the browser build-in Javascript API Intl provides an object Intl.Segmenter that handles this.

To reproduce the same result for 有效率的享受工作, you can do something like:

const segmenter = new Intl.Segmenter("zh-HK", { granularity: "word" });
const segments = segmenter.segment("有效率的享受工作");

console.log(Array.from(segments));
// [{"segment":"有效率","index":0,"input":"有效率的享受工作","isWordLike":true},
//  {"segment":"的","index":3,"input":"有效率的享受工作","isWordLike":true},
//  {"segment":"享受","index":4,"input":"有效率的享受工作","isWordLike":true},
//  {"segment":"工作","index":6,"input":"有效率的享受工作","isWordLike":true}]

and as for Thai, how brilliant.

🚫

จากวันนั้นที่เธอได้ไปจากฉัน

✅

จาก วัน นั้น ที่ เธอ ได้ ไป จาก ฉัน

Select individual word and get translations of the highlighted

Spotify API

Instead of a web player, I thought it’d be a much useful application if I could stay on where I usually play music (mobile phone) and the lyrics prompter as a separate channel. When returning to the lyrics prompter, it should always jump to the line that’s being played.

Sync with current player state

The current Spotify state must be retrieved hence Get Playback State.

{
  "device": /** The device obj **/,
  "progress_ms": 0,
	"is_playing": true,
  "item": /** The currently playing item **/ 
  ...
}

From here, the play progress bar can take progress_ms as input, and the lyrics prompter can use the same variable to identity which line is being currently played.

Repeat line

This is to reproduce an existing feature of the Spotify Lyrics experience. Each line has a play button to change the progress, which essentially calls the seek to position API. The API is very straight forward. Give it the progress_ms and it’ll update the player state of the current device.

Wrap

Language learning can be very enjoyable. You get better at it as more connections grow between you and the language. Whether it being cultural connection, making friends doing casual conversations, or simply understanding better of the meaning in between the lines. Like language reactor to Netflix, we are definitely in an era with the most resources to learn a foreign language, or to rephrase, we have a lot of fun resources to design your own accelerators in language learning.

Side Note

Watch on YouTube

The first time when I listen to ถ้าเราเจอกันอีก (Until Then) by Tilly Birds, it was simply a beautiful song. Through interviews (translated) I’d understand the meaning behind the song more. The song was written by Third, the main vocal of the band, in memory of his late parents. The lyrics hit very differently when I started to understand what they mean.