Skip to main content

Thai has no spaces: how speech recognition handles word segmentation and evaluation

DataArc TeamPublished · 8 min read

Thai is written without spaces between words; a space usually appears only between phrases or sentences. That turns “where should the recognized text be split into words” into a separate problem, and it makes word error rate (WER), the usual metric, unstable for Thai. This article describes the metrics and the text normalization we use when we evaluate Thai recognition.

1. Why Thai has no spaces

Thai is a continuous script: the words in a sentence run together, and readers split them using their knowledge of the vocabulary. A space in Thai works more like a comma or a full stop. It separates phrases and sentences.

For speech recognition, the model outputs one continuous string of characters. If a downstream system needs search, translation or subtitle line breaks, it has to segment the text again. Segmentation is ambiguous in itself: the same string of characters can have more than one valid split, and different tokenizers use different dictionaries and rules.

ตากลมOne string of characters, two ways to split it
ตา | กลมround eyesตาก | ลมairing in the wind
A common example of ambiguity in Thai word segmentation.

2. Why WER is unstable on Thai

Word error rate counts substitutions, deletions and insertions per word, then divides by the number of words in the reference. For Thai, both the reference and the recognition result must be segmented before they can be compared, so the tokenizer becomes part of the evaluation: change the tokenizer and both the word count and the alignment change, and WER changes with them.

Character error rate counts per character and needs no segmentation. The result depends only on the two texts themselves. That is why CER is the more common main metric for Thai recognition.

CER = (S + D + I) / N

S  substituted characters
D  deleted characters
I  inserted characters
N  characters in the reference text

CER has limits too. It treats a wrong character and a wrong word alike, and it cannot show directly whether a sentence reads naturally. So we also report WER under a fixed tokenizer, for reference.

3. Text normalization before evaluation

In Thai, vowel and tone marks are written above or below the consonant. Text that looks the same can be encoded in different ways in Unicode:

  • The vowel ำ (U+0E33) can also be written as ํ (U+0E4D) followed by า (U+0E32). The two display almost identically.
  • If a tone mark and an upper vowel are typed in a different order, the code point sequence is different.
  • Thai digits ๑๒๓ and Arabic digits 123 are mixed, and so are full-width and half-width symbols.

Unless these are unified first, every such difference is counted as a recognition error. Before computing metrics we apply the same processing to the reference and the recognition result: Unicode normalization, [Thai diacritic order rules], unified digits and punctuation, and removal of extra whitespace.

4. What we do

  1. The main metric is CER, computed after the normalization above.
  2. The reference metric is WER, always segmented with [tokenizer name and version].
  3. Every number states the test set, model version, hardware and test date.
  4. The full evaluation methodology and raw results CSV are available on the public benchmarks page.
DataArc Team

Related articles