mahing
/

historical-narrative-generator

@@ -12,7 +12,7 @@ tags:
 ---
 **Introduction** <br />
-LLMs can be used to build out accurate and informative first-person narratives from historical periods, mimicking the language and speech of an era. This in turn can be used to create educational stories from that era to guide listeners through their journey into a specific period in history. This task for an LLM can expand our understanding of culture and language from historical eras in a fun way, which can be used for educational purposes in schools and museums.  <br />
 To successfully fine-tune an LLM for this task, I first picked a suitable base model that created passable narratives with few-shot prompting and had few enough parameters to not require massive amounts of compute for fine-tuning. I chose to use Qwen2.5-1M for this purpose. I then used Gutenberg and other sources to find historical documents that could be used as input data to train the custom Qwen model, matching a synthetically generated narrative to each historical document. This was used as the training data for LoRA, which updated the most relevant parameters for my custom task. The historical narratives generated after fine-tuning were much stronger than current LLM results and exceeded expectations. If used in schools, this model could create engaging, creative, and informative first-person narratives to build knowledge and interest in history for students.
 **Training Data** <br />

 ---
 **Introduction** <br />
+LLMs can be used to build out accurate and informative first-person narratives from historical periods, mimicking the language and speech of an era. This in turn can be used to create educational stories from that era to guide listeners through their journey into a specific period in history. This task for an LLM can expand our understanding of culture and language from historical eras in a fun way, which can be used for educational purposes in schools and museums. Using current LLMs for this task would not be very successful as current models are trained on so much data and are not tailored for this specific task, leading to possible anachronisms and inaccuracies in the language it uses and the historical information. Using current models resulted in sub-par narratives even after many different prompt engineering and few-shot prompting methods.  <br />
 To successfully fine-tune an LLM for this task, I first picked a suitable base model that created passable narratives with few-shot prompting and had few enough parameters to not require massive amounts of compute for fine-tuning. I chose to use Qwen2.5-1M for this purpose. I then used Gutenberg and other sources to find historical documents that could be used as input data to train the custom Qwen model, matching a synthetically generated narrative to each historical document. This was used as the training data for LoRA, which updated the most relevant parameters for my custom task. The historical narratives generated after fine-tuning were much stronger than current LLM results and exceeded expectations. If used in schools, this model could create engaging, creative, and informative first-person narratives to build knowledge and interest in history for students.
 **Training Data** <br />