fewabu/1e-04_AmpGPT2

1e04_output_dir_clean_df_10-100_noX_100_50_epoch_cluster

This model is a fine-tuned version of nferruz/ProtGPT2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.9866
• Accuracy: 0.5809

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	148	4.7445	0.3307
No log	2.0	296	4.2080	0.3912
No log	3.0	444	3.9412	0.4242
4.5166	4.0	592	3.7429	0.4491
4.5166	5.0	740	3.6093	0.4675
4.5166	6.0	888	3.5137	0.4819
3.2167	7.0	1036	3.4691	0.4945
3.2167	8.0	1184	3.4144	0.5048
3.2167	9.0	1332	3.3772	0.5134
3.2167	10.0	1480	3.3649	0.5210
2.5704	11.0	1628	3.3601	0.5281
2.5704	12.0	1776	3.3302	0.5337
2.5704	13.0	1924	3.3532	0.5391
2.127	14.0	2072	3.3311	0.5439
2.127	15.0	2220	3.3248	0.5481
2.127	16.0	2368	3.3493	0.5524
1.8084	17.0	2516	3.3751	0.5545
1.8084	18.0	2664	3.3950	0.5577
1.8084	19.0	2812	3.4425	0.5591
1.8084	20.0	2960	3.4472	0.5610
1.559	21.0	3108	3.4676	0.5636
1.559	22.0	3256	3.4823	0.5658
1.559	23.0	3404	3.5218	0.5678
1.3724	24.0	3552	3.5351	0.5689
1.3724	25.0	3700	3.5680	0.5700
1.3724	26.0	3848	3.6106	0.5713
1.3724	27.0	3996	3.6143	0.5726
1.2295	28.0	4144	3.6293	0.5743
1.2295	29.0	4292	3.6705	0.5742
1.2295	30.0	4440	3.7098	0.5753
1.1165	31.0	4588	3.7151	0.5757
1.1165	32.0	4736	3.7225	0.5773
1.1165	33.0	4884	3.7667	0.5777
1.0351	34.0	5032	3.7980	0.5771
1.0351	35.0	5180	3.8173	0.5779
1.0351	36.0	5328	3.8312	0.5787
1.0351	37.0	5476	3.8549	0.5789
0.9701	38.0	5624	3.8660	0.5791
0.9701	39.0	5772	3.8984	0.5793
0.9701	40.0	5920	3.9071	0.5796
0.9209	41.0	6068	3.9172	0.5805
0.9209	42.0	6216	3.9471	0.5799
0.9209	43.0	6364	3.9458	0.5804
0.8868	44.0	6512	3.9573	0.5807
0.8868	45.0	6660	3.9646	0.5804
0.8868	46.0	6808	3.9689	0.5812
0.8868	47.0	6956	3.9685	0.5809
0.8627	48.0	7104	3.9845	0.5812
0.8627	49.0	7252	3.9856	0.5810
0.8627	50.0	7400	3.9866	0.5809

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.2.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0