Word error rate (WER) is a common metric of the performance of a speech recognition or machine translation system.
The general difficulty of measuring performance lies in the fact that the recognized word sequence can have a different length from the reference word sequence (supposedly the correct one). The WER is derived from the Levenshtein distance, working at the word level instead of the phoneme level. The WER is a valuable tool for comparing different systems as well as for evaluating improvements within one system. This kind of measurement, however, provides no details on the nature of translation errors and further work is therefore required to identify the main source(s) of error and to focus any research effort.
This problem is solved by first aligning the recognized word sequence with the reference (spoken) word sequence using dynamic string alignment. Examination of this issue is seen through a theory called the power law that states the correlation between perplexity and word error rate.[1]
Word error rate can then be computed as:
where
S is the number of substitutions,
D is the number of deletions,
I is the number of insertions,
C is the number of correct words,
N is the number of words in the reference (N=S+D+C)
The intuition behind 'deletion' and 'insertion' is how to get from the reference to the hypothesis. So if we have the reference "This is wikipedia" and hypothesis "This _ wikipedia", we call it a deletion.
When reporting the performance of a speech recognition system, sometimes word accuracy (WAcc) is used instead:
Note that since N is the number of words in the reference, the word error rate can be larger than 1.0, and thus, the word accuracy can be smaller than 0.0.
^Klakow, Dietrich; Jochen Peters (September 2002). "Testing the correlation of word error rate and perplexity". Speech Communication. 38 (1–2): 19–28. doi:10.1016/S0167-6393(01)00041-3. ISSN 0167-6393.
Worderrorrate (WER) is a common metric of the performance of a speech recognition or machine translation system. The general difficulty of measuring...
rate Residual bit errorrate Soft errorrate Technique for human error-rate prediction Viterbi errorrateWorderrorrate Failure rate This disambiguation...
synchronization errors. The bit errorrate (BER) is the number of bit errors per unit time. The bit error ratio (also BER) is the number of bit errors divided...
in translation length do not impact the overall score as much. The Worderrorrate (WER) is a metric based on the Levenshtein distance, where the Levenshtein...
positive rate of a certain diagnostic device is 1%"), while type I error is a term associated with statistical tests, where the meaning of the word "positive"...
247 possibilities for each word. There are two standard evaluation metrics for language models: perplexity or worderrorrate(WER). The simpler of these...
In electronics and computing, a soft error is a type of error where a signal or datum is wrong. Errors may be caused by a defect, usually understood either...
letters stand for number, edition error and recognition error. It is an alternative to the WER model (WordErrorRate) used in several countries. The model...
computed with the help of worderrorrate (WER). Worderrorrate can be calculated by aligning the recognized word and referenced word using dynamic string...
the errorrate, then switch to ARQ when the errorrate gets too high; adaptive modulation and coding uses a variety of ECC rates, adding more error-correction...
applications in computer science and telecommunication, error detection and correction (EDAC) or error control are techniques that enable reliable delivery...
correct errors, and can detect only an odd number of bits in error. Hamming codes are perfect codes, that is, they achieve the highest possible rate for codes...
systems can be scored against reference word-level transcriptions to produce a value for the worderrorrate (WER), but because phonetic systems use phones...
first Wikimedia UK chapter Windows Error Reporting, a feature of Windows XP and later operating systems Worderrorrate, in computational linguistics, a...
Alphabet. The ASR system based on English Wiktionary has the highest worderrorrate, where each third phoneme has to be changed. Ontology engineering and...
memory maintains a memory system immune to single-bit errors: the data that is read from each word is always the same as the data that had been written...
Errors in early word use or developmental errors are mistakes that children commonly commit when first learning language. Language acquisition is an impressive...
implementation of a form of cache language model yielded a 24% drop in word-errorrates once the first few hundred words of a document had been dictated. A...
Sphinx-3. Due to the low quality of streaming audio at the time, the worderrorrate was quite high, but most searches were still able to retrieve relevant...
to complete the request 4xx client error – the request contains bad syntax or cannot be fulfilled 5xx server error – the server failed to fulfil an apparently...
Mortality rate, or death rate,: 189, 69 is a measure of the number of deaths (in general, or due to a specific cause) in a particular population, scaled...
Global Information
