README

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.jpg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,58 @@
+# Mahjong Vision Assistant
+
+This project uses computer vision and machine learning to provide real-time discard suggestions for the game Mahjong Soul.
+
+## Features
+
+*   **Tile Recognition:** Identifies Mahjong tiles from the Mahjong Soul game window using a fine-tuned Vision Transformer model (`pjura/mahjong_soul_vision`).
+*   **Game State Analysis:** Parses the recognized tiles to understand the current game state (player's hand, melds, discard pools).
+*   **Discard Suggestion:** Employs a neural network (`ImprovedNN`) to predict the optimal discard based on the analyzed game state.
+*   **Live Overlay:** Captures the game window and overlays suggestions directly onto the screen, highlighting the recommended discard tile.
+
+![Example of Mahjong Vision Assistant Overlay](exampleImageMahjongSoul.jpg)
+
+
+## Project Structure
+
+*   `live_feed.py`: The main script to run the live assistant. It captures the screen, performs tile recognition, predicts discards, and displays the overlay.
+*   `hf_vision_model.ipynb`: Jupyter notebook detailing the training process for the Hugging Face Vision Transformer used for tile recognition.
+*   `tools.py`: Contains utility functions for data processing, model prediction, loss calculation, MLflow interaction, and tile representation translation used by `live_feed.py`. Many cross repo functions. 
+*   `model.safetensors`: Saved weights for the discard prediction neural network (`ImprovedNN`).
+
+## Setup
+
+1.  **Environment:** Ensure you have Python installed along with necessary libraries. Key libraries include:
+    *   `torch` (with CUDA support if available)
+    *   `transformers`
+    *   `datasets`
+    *   `evaluate`
+    *   `opencv-python` (`cv2`)
+    *   `Pillow` (`PIL`)
+    *   `pygetwindow`
+    *   `numpy`
+    *   `pyautogui`
+    *   `keyboard`
+    *   `safetensors`
+    *   `mlflow` (Optional, used in `tools.py`, you can use whatever you like to serve the model)
+    *   `scipy`
+    *   `matplotlib`
+    *(A `requirements.txt` file would be beneficial here, but didn't made one at the time)*
+
+2.  **Models:**
+    *   The tile recognition model (`pjura/mahjong_soul_vision`) will be downloaded automatically by the `transformers` library.
+    *   The discard prediction model (`model.safetensors`) should be present in the root directory.
+
+## Usage
+
+1.  Ensure the Mahjong Soul game window is open and titled "MahjongSoul".
+2.  Run the main script:
+    ```bash
+    python live_feed.py
+    ```
+3.  The script will capture the game window, analyze the tiles, and highlight the suggested discard tile in the player's hand region. The color of the highlight indicates the model's confidence (Green=High, Red=Low).
+4.  Press 'q' to quit the application.
+
+## Notes
+
+*   The script relies on specific window coordinates and aspect ratios which might need adjustment depending on screen resolution and game layout.
+*   The discard prediction model (`ImprovedNN` loaded from `model.safetensors`) is based on the [pjura/mahjong_ai](https://huggingface.co/pjura/mahjong_ai) model from Hugging Face. It was trained on the `pjura/mahjong_board_states` dataset, primarily using the `tenhou_prediction_deepLearning_basic.ipynb` notebook as detailed on the model card. The local `model.safetensors` may not be the latest version available on the Hub. You can add your own logic