bert-base-multilingual-cased-IDMGSP-danish

This model is a fine-tuned version of bert-base-multilingual-cased on the on the ernlavr/IDMGSP-danish dataset. It achieves the following results on the evaluation set:

• Loss: 1.0123
• Accuracy: {'accuracy': 0.8289043068464459}
• F1: {'f1': 0.842473183078221}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.4692	1.0	480	0.3779	{'accuracy': 0.8519439717240477}	{'f1': 0.84845236500067}
0.3267	2.0	960	0.5350	{'accuracy': 0.7896321508050792}	{'f1': 0.8138538167496815}
0.5149	3.0	1440	0.7051	{'accuracy': 0.7510145306977353}	{'f1': 0.7911267296288161}
0.2823	4.0	1920	0.6520	{'accuracy': 0.7317711742374656}	{'f1': 0.7837010450754776}
0.2107	5.0	2400	0.3335	{'accuracy': 0.8785181306453724}	{'f1': 0.8759689922480619}
0.1868	6.0	2880	0.8269	{'accuracy': 0.8175153815944496}	{'f1': 0.8349123638086214}
0.0969	7.0	3360	0.4585	{'accuracy': 0.877470873150936}	{'f1': 0.872200983069361}
0.1116	8.0	3840	1.0309	{'accuracy': 0.7993192826286163}	{'f1': 0.8236106316879531}
0.0386	9.0	4320	0.9517	{'accuracy': 0.8294279355936641}	{'f1': 0.8426898466739103}
0.0204	10.0	4800	1.0123	{'accuracy': 0.8289043068464459}	{'f1': 0.842473183078221}

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.1
• Datasets 2.14.6
• Tokenizers 0.14.1