27 lines
1.4 KiB
Plaintext
27 lines
1.4 KiB
Plaintext
[jev] device=cuda base=/tmp/base tokenizers=0.22.2
|
|
Some weights of ModernBertForSequenceClassification were not initialized from the model checkpoint at /tmp/base and are newly initialized: ['classifier.bias', 'classifier.weight']
|
|
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
|
|
[jev] LoRA targets: ['Wqkv', 'Wo']
|
|
trainable params: 2,298,625 || all params: 309,829,634 || trainable%: 0.7419
|
|
[jev] train=37182 calib=5291
|
|
[jev] ep0 step100/2322 loss=0.2687
|
|
[jev] ep0 step200/2322 loss=0.3022
|
|
[jev] ep0 step300/2322 loss=0.3244
|
|
[jev] ep0 step400/2322 loss=0.2449
|
|
[jev] ep0 step500/2322 loss=0.2033
|
|
[jev] ep0 step600/2322 loss=0.2433
|
|
[jev] ep0 step700/2322 loss=0.2197
|
|
[jev] ep0 step800/2322 loss=0.1367
|
|
[jev] ep0 step900/2322 loss=0.1337
|
|
[jev] ep0 step1000/2322 loss=0.1400
|
|
[jev] ep0 step1100/2322 loss=0.1538
|
|
/usr/local/lib/python3.10/dist-packages/torch/_inductor/compile_fx.py:282: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.
|
|
warnings.warn(
|
|
[jev] epoch 0 calibration: {'auc': 0.9947, 'ece': 0.0182, 'n': 5291}
|
|
[jev] live-checkpoint 已写 /tmp/jd/model_live
|
|
[jev] ep1 step1200/2322 loss=0.2625
|
|
[jev] ep1 step1300/2322 loss=0.0523
|
|
[jev] ep1 step1400/2322 loss=0.0902
|
|
[jev] ep1 step1500/2322 loss=0.0737
|
|
[jev] ep1 step1600/2322 loss=0.1239
|