how to use the checkpoint
There are several PTH files. Each file seems to serve a specific purpose, and I need to understand when and how to load each one.
downstream_commentary_all_open.pth
downstream_commentary_none_open.pth
pretrained_both.pth
pretrained_classification.pth
pretrained_contrastive.pth
Could someone please provide guidance on the appropriate scenarios and methods for loading each of these PTH files?
Thanks for following our project! Here are some thought might help you around these files:
As for those start with "pretrained" are files are the backbone checkpoints of MatchVision, which you could find on our github repo. With these checkpoints you can finetune on different other soccer tasks. here classification/contrastive/both you can find clues in our paper.
As for those starts with downstream, here we provided 2 checkpoints while opening different blocks during training in MatchVoice model, the implementation details could be found on https://github.com/jyrao/UniSoccer. Also, you might be confused with all_open and none_open. Here are some explanations in detail: https://github.com/jyrao/UniSoccer/issues/3.
Good luck!
Jiayuan
Thanks.
How to set this param? visual_encoder_checkpoint = "PATH_TO_VISUAL_BACKBONE_CHECKPOINT", https://github.com/jyrao/UniSoccer/blob/main/model/matchvoice_model_all_blocks.py#L42
Is this parameter used during training? using one of the pretrained_* pth?
For inference, does this parameter need to be set? Is it overwritten by the downstream_commentary* pth?
If you want to replace "VISUAL_BACHBONE_CHECKPOINT", as I said, use those start with "pretrain".
If you want to train a (for example, commentary generation) model, you can initialize it with pretrain backbone files. If you want to continue training something, you might get some hint here: Line 168, https://github.com/jyrao/UniSoccer/blob/main/task/downstream_commentary.py