The training script produce WER of 2.57% on librispeech test-clean

[pkufool]

Averaging epoch 15 - 34, (decoding with flag --epoch 34 --avg 20)
Decoding with HLG + 4gram lm rescore + attention decoder rescore.

ngram_lm_scale_1.3_attention_scale_1.2 2.57  best for test-clean                                              
ngram_lm_scale_1.1_attention_scale_0.9 2.58                                                          
ngram_lm_scale_1.1_attention_scale_1.0 2.58                                                          
ngram_lm_scale_1.1_attention_scale_1.1 2.58                                                          
ngram_lm_scale_1.2_attention_scale_1.1 2.58                                                          
ngram_lm_scale_1.2_attention_scale_1.2 2.58    

ngram_lm_scale_1.2_attention_scale_1.1 5.94  best for test-other                                              
ngram_lm_scale_1.5_attention_scale_2.0 5.95                                                          
ngram_lm_scale_1.0_attention_scale_0.7 5.96                                                          
ngram_lm_scale_1.2_attention_scale_0.9 5.96                                                          
ngram_lm_scale_1.3_attention_scale_1.1 5.96 

[danpovey]

Thanks!
LGTM, but will wait for Fangjun to check and merge.

[csukuangfj]

Could you also add a file conformer_ctc/RESULTS.md and put the WERs for test-clean and test-other in a table
into that file, e.g.,

	test-clean	test-other
WER	2.57	5.94

Please also put a link to the uploaded model after model averaging in the added file so that others can use it. See #10 (comment)

[pzelasko]

Did you observe any WER improvements due to recent changes in bucketing on either of the test sets?

[pkufool]

Did you observe any WER improvements due to recent changes in bucketing on either of the test sets?

The changes in bucketing were included in the model, but there were also some other changes. I am not sure how much it benefits from the changes in sampler.

[csukuangfj]

👍 👍

[pzelasko]

Alright -- anyway, good job :)

[danpovey]

@pkufool when you have time, would you mind telling me an earlier WER, possibly one from --epoch 23?
I want to compare it with a version with a different optimizer and learning rate scheduler. My best WER for test-clean is 2.8 and for test-other is 6.6, with --lattice-score-scale=0.5 --method=attention-decoder --epoch=23 --avg=10 --max-duration=30 .
I'm wondering whether this is worse than what I'd expect for that iteration.

[pkufool]

conformer_ctc/exp/23_10.txt:ngram_lm_scale_0.9_attention_scale_0.9      2.76    best for test-clean
conformer_ctc/exp/23_10.txt:ngram_lm_scale_1.3_attention_scale_2.0      6.07    best for test-other

I only decoded epochs greater than 30 before, will search for more epochs and update results here.

[pkufool]

Best WERs for some epochs, the *.txt filename means epoch_avg.txt, for example, 49_10.txt is the result run with --epoch 49 --avg 10.

conformer_ctc/exp/49_10.txt:ngram_lm_scale_1.5_attention_scale_1.9      2.62    best for test-clean
conformer_ctc/exp/49_10.txt:ngram_lm_scale_1.0_attention_scale_0.9      5.91    best for test-other

conformer_ctc/exp/49_12.txt:ngram_lm_scale_1.3_attention_scale_1.5      2.64    best for test-clean
conformer_ctc/exp/49_12.txt:ngram_lm_scale_1.3_attention_scale_2.0      5.94    best for test-other

conformer_ctc/exp/49_15.txt:ngram_lm_scale_1.0_attention_scale_1.3      2.64    best for test-clean
conformer_ctc/exp/49_15.txt:ngram_lm_scale_1.3_attention_scale_1.1      5.95    best for test-other

conformer_ctc/exp/49_18.txt:ngram_lm_scale_1.2_attention_scale_1.2      2.62    best for test-clean
conformer_ctc/exp/49_18.txt:ngram_lm_scale_1.0_attention_scale_0.9      5.92    best for test-other

conformer_ctc/exp/49_20.txt:ngram_lm_scale_1.7_attention_scale_2.0      2.62    best for test-clean
conformer_ctc/exp/49_20.txt:ngram_lm_scale_1.5_attention_scale_1.7      5.89    best for test-other

conformer_ctc/exp/49_8.txt:ngram_lm_scale_1.1_attention_scale_1.0       2.64    best for test-clean
conformer_ctc/exp/49_8.txt:ngram_lm_scale_1.3_attention_scale_1.3       5.96    best for test-other

conformer_ctc/exp/44_10.txt:ngram_lm_scale_1.5_attention_scale_1.7      2.66    best for test-clean
conformer_ctc/exp/44_10.txt:ngram_lm_scale_1.7_attention_scale_2.0      5.97    best for test-other

conformer_ctc/exp/44_12.txt:ngram_lm_scale_1.3_attention_scale_1.5      2.63    best for test-clean
conformer_ctc/exp/44_12.txt:ngram_lm_scale_1.7_attention_scale_2.0      6.01    best for test-other

conformer_ctc/exp/44_15.txt:ngram_lm_scale_1.2_attention_scale_1.1      2.62    best for test-clean
conformer_ctc/exp/44_15.txt:ngram_lm_scale_1.5_attention_scale_1.7      5.91    best for test-other

conformer_ctc/exp/44_18.txt:ngram_lm_scale_1.3_attention_scale_1.0      2.62    best for test-clean
conformer_ctc/exp/44_18.txt:ngram_lm_scale_1.9_attention_scale_1.9      5.93    best for test-other

conformer_ctc/exp/44_20.txt:ngram_lm_scale_0.9_attention_scale_0.6      2.63    best for test-clean
conformer_ctc/exp/44_20.txt:ngram_lm_scale_1.5_attention_scale_1.5      5.9     best for test-other

conformer_ctc/exp/44_8.txt:ngram_lm_scale_1.5_attention_scale_1.9       2.65    best for test-clean
conformer_ctc/exp/44_8.txt:ngram_lm_scale_1.1_attention_scale_1.0       5.99    best for test-other

conformer_ctc/exp/39_10.txt:ngram_lm_scale_1.1_attention_scale_1.0      2.63    best for test-clean
conformer_ctc/exp/39_10.txt:ngram_lm_scale_1.5_attention_scale_1.7      6.03    best for test-other

conformer_ctc/exp/39_12.txt:ngram_lm_scale_0.9_attention_scale_0.5      2.63    best for test-clean
conformer_ctc/exp/39_12.txt:ngram_lm_scale_1.7_attention_scale_1.9      6.0     best for test-other

conformer_ctc/exp/39_15.txt:ngram_lm_scale_1.0_attention_scale_0.7      2.64    best for test-clean
conformer_ctc/exp/39_15.txt:ngram_lm_scale_1.3_attention_scale_1.2      6.0     best for test-other

conformer_ctc/exp/39_18.txt:ngram_lm_scale_1.3_attention_scale_1.3      2.62    best for test-clean
conformer_ctc/exp/39_18.txt:ngram_lm_scale_1.7_attention_scale_1.7      5.95    best for test-other

conformer_ctc/exp/39_20.txt:ngram_lm_scale_1.5_attention_scale_1.9      2.59    best for test-clean
conformer_ctc/exp/39_20.txt:ngram_lm_scale_1.3_attention_scale_1.5      5.98    best for test-other

conformer_ctc/exp/39_8.txt:ngram_lm_scale_1.5_attention_scale_1.7       2.66    best for test-clean

conformer_ctc/exp/34_10.txt:ngram_lm_scale_1.3_attention_scale_1.0      2.67    best for test-clean
conformer_ctc/exp/34_10.txt:ngram_lm_scale_1.5_attention_scale_1.7      6.01    best for test-other

conformer_ctc/exp/34_12.txt:ngram_lm_scale_1.7_attention_scale_1.9      2.65    best for test-clean
conformer_ctc/exp/34_12.txt:ngram_lm_scale_1.1_attention_scale_0.9      6.0     best for test-other

conformer_ctc/exp/34_15.txt:ngram_lm_scale_1.5_attention_scale_1.7      2.62    best for test-clean
conformer_ctc/exp/34_15.txt:ngram_lm_scale_1.2_attention_scale_1.1      6.01    best for test-other

conformer_ctc/exp/34_18.txt:ngram_lm_scale_1.3_attention_scale_1.3      2.64    best for test-clean
conformer_ctc/exp/34_18.txt:ngram_lm_scale_1.0_attention_scale_0.9      6.07    best for test-other

conformer_ctc/exp/34_20.txt:ngram_lm_scale_1.3_attention_scale_1.3      2.59    best for test-clean
conformer_ctc/exp/34_20.txt:ngram_lm_scale_1.2_attention_scale_1.1      5.96    best for test-other

conformer_ctc/exp/34_8.txt:ngram_lm_scale_1.1_attention_scale_0.7       2.66    best for test-clean
conformer_ctc/exp/34_8.txt:ngram_lm_scale_1.2_attention_scale_1.3       6.04    best for test-other

conformer_ctc/exp/29_10.txt:ngram_lm_scale_1.2_attention_scale_1.1      2.62    best for test-clean
conformer_ctc/exp/29_10.txt:ngram_lm_scale_1.2_attention_scale_1.0      6.03    best for test-other

conformer_ctc/exp/29_12.txt:ngram_lm_scale_1.2_attention_scale_1.2      2.64    best for test-clean
conformer_ctc/exp/29_12.txt:ngram_lm_scale_1.3_attention_scale_1.5      5.96    best for test-other

conformer_ctc/exp/29_15.txt:ngram_lm_scale_1.1_attention_scale_1.1      2.62    best for test-clean
conformer_ctc/exp/29_15.txt:ngram_lm_scale_1.5_attention_scale_1.7      6.04    best for test-other

conformer_ctc/exp/29_18.txt:ngram_lm_scale_1.2_attention_scale_1.3      2.62    best for test-clean
conformer_ctc/exp/29_18.txt:ngram_lm_scale_1.3_attention_scale_2.0      6.03    best for test-other

conformer_ctc/exp/29_20.txt:ngram_lm_scale_1.2_attention_scale_1.1      2.66    best for test-clean
conformer_ctc/exp/29_20.txt:ngram_lm_scale_1.5_attention_scale_1.9      6.02    best for test-other

conformer_ctc/exp/29_8.txt:ngram_lm_scale_1.5_attention_scale_2.0       2.71    best for test-clean
conformer_ctc/exp/29_8.txt:ngram_lm_scale_1.5_attention_scale_1.7       6.09    best for test-other

conformer_ctc/exp/23_10.txt:ngram_lm_scale_0.9_attention_scale_0.9      2.76    best for test-clean
conformer_ctc/exp/23_10.txt:ngram_lm_scale_1.3_attention_scale_2.0      6.07    best for test-other

conformer_ctc/exp/23_12.txt:ngram_lm_scale_1.0_attention_scale_1.1      2.73    best for test-clean
conformer_ctc/exp/23_12.txt:ngram_lm_scale_1.3_attention_scale_1.7      6.04    best for test-other

conformer_ctc/exp/23_15.txt:ngram_lm_scale_1.0_attention_scale_0.7      2.77    best for test-clean
conformer_ctc/exp/23_15.txt:ngram_lm_scale_1.2_attention_scale_1.7      6.04    best for test-other

conformer_ctc/exp/23_18.txt:ngram_lm_scale_1.5_attention_scale_1.9      2.79    best for test-clean
conformer_ctc/exp/23_18.txt:ngram_lm_scale_1.1_attention_scale_1.0      6.1     best for test-other

conformer_ctc/exp/23_20.txt:ngram_lm_scale_1.2_attention_scale_1.3      2.89    best for test-clean
conformer_ctc/exp/23_20.txt:ngram_lm_scale_1.7_attention_scale_1.9      6.58    best for test-other

conformer_ctc/exp/23_8.txt:ngram_lm_scale_1.3_attention_scale_1.7       2.74    best for test-clean
conformer_ctc/exp/23_8.txt:ngram_lm_scale_1.3_attention_scale_1.5       6.05    best for test-other