Snowflake/Arctic-Text2SQL-R1-7B Fine-tuned for NL2SQL++ v8

This model is a fine-tuned version of Snowflake/Arctic-Text2SQL-R1-7B on the NL2SQL++ v8 dataset with code-with-thought reasoning.

Model Details

Base Model: Snowflake/Arctic-Text2SQL-R1-7B
Task: Text-to-SQL generation
Dataset: NL2SQL++ v8 with code-with-thought reasoning
Fine-tuning Method: LoRA (Low-Rank Adaptation) with Unsloth
Quantization: 16-bit merged weights
Training Dataset Size: (15506, 1) examples
Validation Dataset Size: (1000, 1) examples

Training Configuration

output_dir: trainer_output
overwrite_output_dir: False
do_train: False
do_eval: True
do_predict: False
eval_strategy: IntervalStrategy.STEPS
prediction_loss_only: False
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 8
eval_accumulation_steps: 10
eval_delay: 0
torch_empty_cache_steps: None
learning_rate: 0.0002
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 3.0
max_steps: -1
lr_scheduler_type: SchedulerType.COSINE
lr_scheduler_kwargs: None
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_dir: trainer_output/runs/Jan23_15-05-43_ip-172-31-44-246.ap-southeast-2.compute.internal
logging_strategy: IntervalStrategy.STEPS
logging_first_step: False
logging_steps: 0.004
logging_nan_inf_filter: True
save_strategy: SaveStrategy.BEST
save_steps: 0.04
save_total_limit: 2
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 3407
data_seed: None
jit_mode_eval: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
eval_steps: 0.04
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
run_name: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
metric_for_best_model: eval_loss
greater_is_better: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: AcceleratorConfig(split_batches=False, dispatch_batches=None, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False)
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: OptimizerNames.PAGED_ADAMW
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
report_to: ['wandb']
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: None
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: HubStrategy.EVERY_SAVE
hub_token: None
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: True
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
push_to_hub_token: None
_n_gpu: 1
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
model_init_kwargs: None
chat_template_path: None
dataset_text_field: text
dataset_kwargs: None
dataset_num_proc: None
eos_token: None
pad_token: None
max_length: 1024
packing: False
packing_strategy: bfd
padding_free: False
pad_to_multiple_of: None
eval_packing: None
completion_only_loss: None
assistant_only_loss: False
loss_type: nll
activation_offloading: False
vllm_sampling_params: None
unsloth_num_chunks: -1
unsloth_logit_chunk_multiplier: None
unsloth_grpo_mini_batch: None
max_seq_length: 15000
model_name: Snowflake/Arctic-Text2SQL-R1-7B
train_batch_size: 8
val_batch_size: 1
num_epochs: 2.5
lora_r: 64
lora_alpha: 128

Train Dataset Example

<|im_start|>system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>
<|im_start|>user

You are an expert in SQL++ query generation. You will be given a document schema and a natural language query. You need to generate a valid SQL++ query equivalent to the natural language query.

Bucket Name: `travel-sample`
Scope Name: `inventory`

Use the given document schema to generate the SQL++ query.

Document Schema:
{"landmark": {"properties": {"activity": {"samples": ["buy"], "type": "string"}, "address": {"samples": [["21 Elizabeth St, SW1W 9RP"]], "type": "string"}, "alt": {"samples": [["Kings Head"]], "type": "string"}, "city": {"samples": [["Hornchurch"]], "type": "string"}, "content": {"samples": ["All manner of Middle Eastern f..."], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "directions": {"samples": [["Located just south of Hornchur..."]], "type": "string"}, "email": {"samples": [["contact@noevalleybakery.com"]], "type": "string"}, "geo": {"properties": {"accuracy": {"samples": ["APPROXIMATE"], "type": "string"}, "lat": {"samples": [50.120496], "type": "number"}, "lon": {"samples": [-5.535092], "type": "number"}}, "type": "object"}, "hours": {"samples": [["M-F 7:30AM-4PM, Sa 8AM-4PM, Su..."]], "type": "string"}, "id": {"samples": [11965], "type": "number"}, "image": {"samples": [["https://en.wikivoyage.org/wiki..."]], "type": "string"}, "image_direct_url": {"samples": ["https://upload.wikimedia.org/w..."], "type": "string"}, "name": {"samples": ["Friar's Inn"], "type": "string"}, "phone": {"samples": [["+44 1708 450894"]], "type": "string"}, "price": {"samples": [["Prices and opening hours set i..."]], "type": "string"}, "state": {"samples": [["California"]], "type": "string"}, "title": {"samples": ["London/Hammersmith and Fulham"], "type": "string"}, "tollfree": {"samples": [["+1 800 546-2070"]], "type": "string"}, "type": {"samples": ["landmark"], "type": "string"}, "url": {"samples": [["http://www.farmtablesf.com"]], "type": "string"}}, "type": "object"}, "route": {"properties": {"airline": {"samples": ["AS"], "type": "string"}, "airlineid": {"samples": ["airline_1316"], "type": "string"}, "destinationairport": {"samples": ["ATL"], "type": "string"}, "distance": {"samples": [960.5866347178], "type": "number"}, "equipment": {"samples": [["73G"]], "type": "string"}, "id": {"samples": [11746], "type": "number"}, "schedule": {"items": {"properties": {"day": {"type": "number"}, "flight": {"type": "string"}, "utc": {"type": "string"}}, "type": "object"}, "samples": [[{"day": 0, "flight": "AS298", "utc": "06:09:00"}]], "type": "array"}, "sourceairport": {"samples": ["CMN"], "type": "string"}, "stops": {"samples": [0], "type": "number"}, "type": {"samples": ["route"], "type": "string"}}, "type": "object"}, "airport": {"properties": {"airportname": {"samples": ["Harrisburg Intl"], "type": "string"}, "city": {"samples": ["Harrisburg"], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "faa": {"samples": [["JFK"]], "type": "string"}, "geo": {"properties": {"alt": {"samples": [8], "type": "number"}, "lat": {"samples": [25.79325], "type": "number"}, "lon": {"samples": [-80.290556], "type": "number"}}, "type": "object"}, "icao": {"samples": [["EGEO"]], "type": "string"}, "id": {"samples": [3542], "type": "number"}, "type": {"samples": ["airport"], "type": "string"}, "tz": {"samples": ["America/Anchorage"], "type": "string"}}, "type": "object"}, "airline": {"properties": {"callsign": {"samples": [["AIRLINAIR"]], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "iata": {"samples": [["A5"]], "type": "string"}, "icao": {"samples": ["CCM"], "type": "string"}, "id": {"samples": [10], "type": "number"}, "name": {"samples": ["40-Mile Air"], "type": "string"}, "type": {"samples": ["airline"], "type": "string"}}, "type": "object"}, "hotel": {"properties": {"address": {"samples": [["39 Abbey Street, Armagh, BT61 ..."]], "type": "string"}, "alias": {"samples": [["Inn Exile"]], "type": "string"}, "checkin": {"samples": [["3PM"]], "type": "string"}, "checkout": {"samples": [["11AM"]], "type": "string"}, "city": {"samples": [["Argyll and Bute"]], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "description": {"samples": ["28 bed SYHA hostel, with beds ..."], "type": "string"}, "directions": {"samples": [["M\u00e9tro: Concorde"]], "type": "string"}, "email": {"samples": [["info@Lajoyainn.com"]], "type": "string"}, "fax": {"samples": [["+1 415 665-5440"]], "type": "string"}, "free_breakfast": {"samples": [false], "type": "boolean"}, "free_internet": {"samples": [false], "type": "boolean"}, "free_parking": {"samples": [false], "type": "boolean"}, "geo": {"properties": {"accuracy": {"samples": ["APPROXIMATE"], "type": "string"}, "lat": {"samples": [52.44779], "type": "number"}, "lon": {"samples": [-6.6584], "type": "number"}}, "type": "object"}, "id": {"samples": [1097], "type": "number"}, "name": {"samples": ["Armagh City Youth Hostel"], "type": "string"}, "pets_ok": {"samples": [false], "type": "boolean"}, "phone": {"samples": [["+44 121 236 4031"]], "type": "string"}, "price": {"samples": [["$12 a night"]], "type": "string"}, "public_likes": {"items": {"type": "string"}, "samples": [["Caden Schinner"]], "type": "array"}, "reviews": {"items": {"properties": {"author": {"samples": ["Imelda Renner"], "type": "string"}, "content": {"samples": ["Dont be mislead by some hotel ..."], "type": "string"}, "date": {"samples": ["2012-11-10 17:29:26 +0300"], "type": "string"}, "ratings": {"properties": {"Business service": {"samples": [-1], "type": "number"}, "Business service (e.g., internet access)": {"samples": [1], "type": "number"}, "Check in / front desk": {"samples": [-1], "type": "number"}, "Cleanliness": {"samples": [-1], "type": "number"}, "Location": {"samples": [-1], "type": "number"}, "Overall": {"samples": [1], "type": "number"}, "Rooms": {"samples": [-1], "type": "number"}, "Service": {"samples": [-1], "type": "number"}, "Sleep Quality": {"samples": [2], "type": "number"}, "Value": {"samples": [-1], "type": "number"}}, "type": "object"}}, "type": "object"}, "samples": [[{"author": "Bart Simonis PhD", "content": "I booked two rooms for Saturday so my wife and I could take our two young children and her parents t...", "date": "2013-12-18 22:51:08 +0300", "ratings": {"Cleanliness": 5, "Location": 5, "Overall": 3, "Rooms": 3, "Service": 5, "Sleep Quality": 5, "Value": 5}}]], "type": "array"}, "state": {"samples": [["California"]], "type": "string"}, "title": {"samples": ["Armagh"], "type": "string"}, "tollfree": {"samples": [["+1-800-962-0186"]], "type": "string"}, "type": {"samples": ["hotel"], "type": "string"}, "url": {"samples": [["http://www.cpbirminghamnechote..."]], "type": "string"}, "vacancy": {"samples": [false], "type": "boolean"}}, "type": "object"}}

Natural Language Query
Which routes in the route collection rank first by the shortest distance within each destination airport when limited to seven results?

SQL++ Query:
<|im_end|>
<|im_start|>assistant

```sql++
SELECT d.id, d.destinationairport, ROW_NUMBER() OVER (PARTITION BY d.destinationairport ORDER BY d.distance NULLS FIRST) AS `row` FROM route AS d LIMIT 7;

<|im_end|>