fewabu/1e-03_AmpGPT2

1e03_output_dir_clean_df_10-100_noX_100_50_epoch_cluster

This model is a fine-tuned version of nferruz/ProtGPT2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.8960
• Accuracy: 0.6032

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	148	3.5661	0.4613
No log	2.0	296	3.1192	0.5279
No log	3.0	444	3.0594	0.5626
2.9531	4.0	592	3.1790	0.5783
2.9531	5.0	740	3.5376	0.5831
2.9531	6.0	888	3.7871	0.5859
1.0805	7.0	1036	3.8639	0.5897
1.0805	8.0	1184	4.0745	0.5902
1.0805	9.0	1332	4.1780	0.5920
1.0805	10.0	1480	4.3679	0.5898
0.6958	11.0	1628	4.4148	0.5898
0.6958	12.0	1776	4.6758	0.5894
0.6958	13.0	1924	4.7395	0.5891
0.5158	14.0	2072	4.7986	0.5902
0.5158	15.0	2220	4.9618	0.5907
0.5158	16.0	2368	5.1031	0.5900
0.3438	17.0	2516	5.1599	0.5891
0.3438	18.0	2664	5.5223	0.5899
0.3438	19.0	2812	5.7151	0.5920
0.3438	20.0	2960	5.7640	0.5929
0.1561	21.0	3108	5.7294	0.5943
0.1561	22.0	3256	5.7996	0.5943
0.1561	23.0	3404	5.9189	0.5954
0.0862	24.0	3552	5.9626	0.5961
0.0862	25.0	3700	6.0168	0.5964
0.0862	26.0	3848	6.1294	0.5979
0.0862	27.0	3996	6.0221	0.5970
0.0624	28.0	4144	6.2514	0.5978
0.0624	29.0	4292	6.1634	0.5969
0.0624	30.0	4440	6.2918	0.5977
0.0486	31.0	4588	6.3516	0.5983
0.0486	32.0	4736	6.4406	0.5980
0.0486	33.0	4884	6.2788	0.5990
0.041	34.0	5032	6.4498	0.5986
0.041	35.0	5180	6.4954	0.5995
0.041	36.0	5328	6.4053	0.5992
0.041	37.0	5476	6.5243	0.5999
0.0362	38.0	5624	6.4976	0.6010
0.0362	39.0	5772	6.6286	0.6008
0.0362	40.0	5920	6.6397	0.6007
0.0321	41.0	6068	6.6746	0.6016
0.0321	42.0	6216	6.6910	0.6012
0.0321	43.0	6364	6.7029	0.6020
0.0282	44.0	6512	6.7490	0.6027
0.0282	45.0	6660	6.7538	0.6032
0.0282	46.0	6808	6.8012	0.6024
0.0282	47.0	6956	6.7953	0.6028
0.0252	48.0	7104	6.8655	0.6031
0.0252	49.0	7252	6.8806	0.6032
0.0252	50.0	7400	6.8960	0.6032

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.2.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0