TheBug95/sumarize_model_pegasus_v2_finetuned_spanish

sumarize_model_pegasus_v2_original

This model is a fine-tuned version of google/pegasus-xsum on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0660
• Rouge1: 0.6881
• Rouge2: 0.5187
• Rougel: 0.6489
• Rougelsum: 0.6488
• Gen Len: 44.9812

Model description

This model has been trained with a large dataset with data in Spanish to summarize financial texts that are difficult to understand. For more information refer to the following paper https://arxiv.org/abs/2312.09897

Intended uses & limitations

This model is used to summarize financial text in Spanish.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3.419313942464226e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	239	1.2247	0.687	0.5129	0.6465	0.646	44.2425
No log	2.0	478	1.1818	0.6865	0.5145	0.646	0.6458	44.4135
1.2142	3.0	717	1.1477	0.6853	0.5141	0.6459	0.6455	44.203
1.2142	4.0	956	1.1233	0.6863	0.5148	0.647	0.6466	44.2801
1.2426	5.0	1195	1.1101	0.6868	0.517	0.6473	0.6473	44.7425
1.2426	6.0	1434	1.0830	0.6889	0.5193	0.6495	0.6493	44.8064
1.1652	7.0	1673	1.0713	0.6874	0.5172	0.6468	0.6469	44.8252
1.1652	8.0	1912	1.0708	0.688	0.5189	0.649	0.6486	44.9962
1.1176	9.0	2151	1.0664	0.688	0.5186	0.6488	0.6485	45.0357
1.1176	10.0	2390	1.0660	0.6881	0.5187	0.6489	0.6488	44.9812

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2