train_record_789_1767900827

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.7648
• Num Input Tokens Seen: 825662432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3146	2.0	55542	0.3481	82574560
0.365	4.0	111084	0.3042	165148064
0.1601	6.0	166626	0.3144	247703680
0.111	8.0	222168	0.3599	330243296
0.1308	10.0	277710	0.3919	412811488
0.1705	12.0	333252	0.4655	495364848
0.1237	14.0	388794	0.5298	577939272
0.1882	16.0	444336	0.5840	660518800
0.1484	18.0	499878	0.7128	743088984
0.1426	20.0	555420	0.7648	825662432

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1