🔄 In a Training Loop

Stefan Schweter PRO

stefan-it

1806 434 242

P(doom) <0.001%

https://schweter.bayern

AI & ML interests

Flair + Flukes Library 💕, NER & PoS Tagging, LM Pretraining (mostly encoder-only & encoder-decoder), Historical Language Models, German Language Models, Bavarian NLP 🥨, xLSTM

Recent Activity

liked a model about 14 hours ago

google/embeddinggemma-2

commentedon an article about 16 hours ago

Can LLMs Correct Historical OCR Without Rewriting History?

upvoted an article about 17 hours ago

Can LLMs Correct Historical OCR Without Rewriting History?

View all activity

Organizations

liked a model about 14 hours ago

google/embeddinggemma-2

Feature Extraction • 0.7B • Updated about 16 hours ago • 364 • 663

commented on Can LLMs Correct Historical OCR Without Rewriting History? about 16 hours ago

Hey @emanuelaboros ,

very interesting topic and super interesting research question. I thought a bit about it, here's some suggestion.

Main idea is to build first a dataset that consists of sentence variants, labels and an impact description:

Label	OCR	GT (if available)	Corrected	Impact
fixed	Laud	Land	Land	improved
missed	Kricg	Krieg	Kricg	neutral
modernized	Thür, ſeyn	Thür, ſeyn	Tür, sein	unfaithful
archaized	Haus	Haus	Hauß	unfaithful
wrong fix (real word)	Wcin	Wein	Bein	harmful
number changed	1o00	1000	100	harmful
entity damaged	Schmidt	Schmidt	Schmitt	harmful
hallucinated insertion	…der König von	…der König von	…der König von Preußen	harmful
omission	ſagte er, u. ſ. w.	ſagte er, u. ſ. w.	sagte er.	harmful
paraphrase	allwo er ſtarb	allwo er ſtarb	wo er starb	unfaithful
structural	Regie- / rung	Regie- / rung	Regierung	policy-dependent

As it can be seen a potential LLM correction from "1000" to "100" is super bad.

Impact labels are:

Impact	Meaning
improved	The correction moves the text closer to the source.
neutral	Nothing changed, or the change makes no difference.
unfaithful	The meaning is kept, but the historical form of the source is changed.
harmful	The content changes: different words, numbers or names, or text added or lost.
policy-dependent	Whether the change is acceptable depends on the transcription guideline (diplomatic vs. normalized).

When an example contains several edits, its overall impact is the worst one, in this order: harmful > unfaithful > improved > neutral.

It is just my initial draft and ideas about potential labels.

I think one could construct a dataset from various input sources and train very good classifiers for it (different architectures can be possible: encoder, encoder-decodery, decoder-only and they could be a good step towards the intial "How do you check that an OCR correction improves your collection without changing the source?" question.