readme: introduce mean noise span length section

README.md

@@ -19,13 +19,21 @@ Details about the training can be found [here](https://github.com/stefan-it/hmBy
 
 This model was trained with `mean_noise_span_length=20` for one epoch.
 
-# Evaluation on Downstream Tasks (NER)
+# Mean Noise Span Length
 
-We evaluated the hmByT5 model on downstream tasks:
+The previously pretrained hmByT5 models "accidentally" use a mean noise span length of 3, because this value is the
+default one for T5. But the ByT5 paper mentions, that using a length of 3 would make pretraining tasks too easy, and
+recommend a value of 20. Thus, we pretrained this model with `mean_noise_span_length=20` and fine-tuned it on English
+AjMC dataset:
 
-| Model                                                                                                                                                                           | English AjMC | German AjMC  | French AjMC  | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR  | French ICDAR | Avg. |
-|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|------|
-| [`hmbyt5/byt5-small-english`](https://huggingface.co/hmbyt5/byt5-small-english)                                                                                                 | 85.65 ± 1.21 | 87.27 ± 0.50 | 84.44 ± 0.79 |                 |                 |              |              |      |
+| Configuration                            | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Avg.         |
+|------------------------------------------|-------|-------|-------|-------|-------|--------------|
+| `wsFalse-bs4-e10-lr0.00015-poolingfirst` | 85.48 | 84.6  | 85.65 | 86.83 | 86.53 | 85.82 ± 0.79 |
+| `wsFalse-bs4-e10-lr0.00016-poolingfirst` | 85.35 | 84.5  | 86.05 | 85.1  | 85.18 | 85.24 ± 0.5  |
+| `wsFalse-bs8-e10-lr0.00016-poolingfirst` | 84.14 | 83.45 | 84.4  | 84.9  | 85.82 | 84.54 ± 0.79 |
+| `wsFalse-bs8-e10-lr0.00015-poolingfirst` | 85.27 | 85.3  | 83.33 | 85.25 | 81.7  | 84.17 ± 1.45 |
+
+For comparison the model using a length of 3 achieved 85.65 ± 1.21.
 
 # Acknowledgements