14 lines
722 B
Plaintext
14 lines
722 B
Plaintext
[jev] device=cuda base=/tmp/base tokenizers=0.22.2
|
|
Some weights of ModernBertForSequenceClassification were not initialized from the model checkpoint at /tmp/base and are newly initialized: ['classifier.bias', 'classifier.weight']
|
|
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
|
|
[jev] LoRA targets: ['Wqkv', 'Wo']
|
|
trainable params: 2,298,625 || all params: 309,829,634 || trainable%: 0.7419
|
|
[jev] train=27954 calib=5291
|
|
[jev] canaries: ['en', 'zh']
|
|
[jev] ep0 step100/1746 loss=0.2554
|
|
[jev] ep0 step200/1746 loss=0.4286
|
|
[jev] ep0 step300/1746 loss=0.4042
|
|
[jev] ep0 step400/1746 loss=0.3410
|
|
[jev] ep0 step500/1746 loss=0.3232
|
|
[jev] ep0 step600/1746 loss=0.2050
|