Question: How can I get words alignment between ground_truth and hypothesis? about jiwer HOT 3 CLOSED

It's currently only possible to get the number of substitutions, deletions, and insertions with jiwer.compute_measures(). It should be a relatively straightforward PR to feed the Levenshtein.editops() results to the resulting dictionary, which would give you a more precise alignment, e.g to get from the hypothesis to the ground truth, insert 'x' at index 2, delete 'y' at index 5, etc.

from jiwer.

@nikvaessen: I'm looking in measures.py
I'm planning to generate 'words alignment between ground_truth and hypothesis'
should I add it to jiwer ? are there any code style or other guidance ?

from jiwer.

Hi @nikvaessen ,

I have a code version that by calling

ground_truth = ["XX hello world aa bb cc dd ee ff ", "aa bb", "aa bb cc dd", "bb dd ee"]
hypothesis = ["hello duck aa cc dd ee1 ff 11 22 33", "11 aa aa", "bb", "aa bb cc dd ee1 ff"]

error = local_wer(ground_truth, hypothesis, get_alignment=True)
print(f'error={error}\n')

words_alignment.validate()
words_alignment.collect_errors_information()
words_alignment.print_analysis()

generate the following output:

sentence 0, truth length: 9, hyp lenngth 10

0 XX - - 'DEL'
1 hello 0 hello 'COR'
2 world 1 duck 'SUB'
3 aa 2 aa 'COR'
4 bb - - 'DEL'
5 cc 3 cc 'COR'
6 dd 4 dd 'COR'
7 ee 5 ee1 'SUB'
8 ff 6 ff 'COR'
- - 7 11 'INS'
- - 8 22 'INS'
- - 9 33 'INS'

sentence 1, truth length: 2, hyp lenngth 3

- - 0 11 'INS'
0 aa 1 aa 'COR'
1 bb 2 aa 'SUB'

sentence 2, truth length: 4, hyp lenngth 1

0 aa - - 'DEL'
1 bb 0 bb 'COR'
2 cc - - 'DEL'
3 dd - - 'DEL'

sentence 3, truth length: 3, hyp lenngth 6

- - 0 aa 'INS'
0 bb 1 bb 'COR'
- - 2 cc 'INS'
1 dd 3 dd 'COR'
- - 4 ee1 'INS'
2 ee 5 ff 'SUB'

---

Detailed recognition performance analysis.

Truth words count.
bb 4
aa 3
dd 3
cc 2
ee 2
XX 1
hello 1
world 1
ff 1

Deletions words count.
XX 1
bb 1
aa 1
cc 1
dd 1

Insersions words count.
11 2
22 1
33 1
aa 1
cc 1
ee1 1

Substitutions words count.
True-word Hyp-word Count
world duck 1
ee ee1 1
bb aa 1
ee ff 1

I can merge it into the code with your/community comments and help.

Tzur

from jiwer.