Skip to content

Turn model management to XPM-Torch

Basically, what I've in mind is to change this XPMIR-based block:

monomlm_scorer: CrossScorer = CrossScorer.C(
        encoder=TokenizedTextEncoder.C(
            tokenizer=HFStringTokenizer.from_pretrained_id(cfg.base),
            encoder=HFCLSEncoder.from_pretrained_id(cfg.base),
        )
    ).tag("scorer", cfg.id)

into XPM-Torch.

The objectives are:

  1. Ease saving/loading of the model's checkpoints (avoid the init_task thing for example at evaluation and instead instantiate the model using .from_pretrained()). I've no problem with turning experiment.py into a train.py only file to do so, and then having a separated file for evaluation (I think that resembles what you suggested).
  2. In the next steps, leverage the inheritance from torch.Module to build more easily the masked versions of the model, instead of having to build "XPMIR-adapters" on top of each others to change some specific behaviors.