wav2vec2-common_voice_20-hy-mms-finetune

This model is a fine-tuned version of facebook/mms-1b-all on the COMMON_VOICE_20_ARMENIAN dataset. It achieves the following results on the evaluation set:

• Loss: 0.1588
• Wer: 0.2465

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 4.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2137	100	0.2269	0.3076
No log	0.4274	200	0.1915	0.2824
No log	0.6410	300	0.1993	0.2964
No log	0.8547	400	0.1832	0.2735
0.9965	1.0684	500	0.1764	0.2649
0.9965	1.2821	600	0.1733	0.2625
0.9965	1.4957	700	0.1725	0.2592
0.9965	1.7094	800	0.1706	0.2581
0.9965	1.9231	900	0.1681	0.2585
0.2922	2.1368	1000	0.1694	0.2591
0.2922	2.3504	1100	0.1701	0.2575
0.2922	2.5641	1200	0.1701	0.2614
0.2922	2.7778	1300	0.1654	0.2535
0.2922	2.9915	1400	0.1644	0.2517
0.2788	3.2051	1500	0.1636	0.2540
0.2788	3.4188	1600	0.1616	0.2512
0.2788	3.6325	1700	0.1600	0.2470
0.2788	3.8462	1800	0.1587	0.2464

Framework versions

• Transformers 4.51.0
• Pytorch 2.2.1+cu121
• Datasets 3.5.0
• Tokenizers 0.21.1