agents-course (Hugging Face Agents Course)

FunctionGemma Tuning Lab is a new no-code tool by @google that lets you fine-tune a model directly from the browser, with no coding knowledge required, using TRL behind the scenes.

blog: https://developers.googleblog.com/a-guide-to-fine-tuning-functiongemma/

try it out: google/functiongemma-tuning-lab

This example builds on a more advanced one for learning fine-tuning with SFT using TRL: https://ai.google.dev/gemma/docs/functiongemma/finetuning-with-functiongemma

1 reply

·

sergiopaniego

posted an update 9 days ago

Post

717

TRL v0.27.0 is out!! 🥳

It includes GDPO, the latest variant of GRPO for multi-reward RL ✨
GDPO decouples reward normalization to avoid reward collapse and improve per-reward convergence — developed by
@sliuau @SimonX et al.

Explore the paper: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization (2601.05242)

Explore the full set of changes here:
https://github.com/huggingface/trl/releases/tag/v0.27.0

sergiopaniego

posted an update 12 days ago

Post

2939

New REPL environment in OpenEnv available! ✨
Used in the Recursive Language Models (RLM) paper by Alex Zhang.

Ready for inference & post-training using trajectories. Handles long contexts:

> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result

Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py

sergiopaniego

posted an update 13 days ago

Post

407

Recursive Language Models (RLM) is a new interface for LLMs with cool ideas by Alex Zhang!

⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)

We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282

More resources:

> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm

2 replies

·

sergiopaniego

posted an update 17 days ago

Post

2223

New GRPO + TRL free Colab notebook out! 🔥

Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO

7B model uses only 9.2 GB VRAM (~7× reduction) 🤯

Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb

pcuenq

posted an update 21 days ago

Post

2942

👉 What happened in AI in 2025? 👈

We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!

Play with it here:
2025-ai-timeline/2025-ai-timeline

Here's my personal quarterly TL;DR:

1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.

Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)

2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.

Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4

3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.

Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5

4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!

Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯

Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline

🫡 @reach-vb for the original idea, design and recipe

🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition

🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂

1 reply

·

sergiopaniego

posted an update 23 days ago

Post

2574

The list of hands-on notebooks (some beginner-friendly!) to get started with fine-tuning using TRL keeps growing!!

• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!

https://github.com/huggingface/trl/tree/main/examples/notebooks

sergiopaniego

posted an update 26 days ago

Post

469

As the year comes to an end, it’s a good moment to catch up on some of the best long-form pieces published by the @huggingface team.

I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts

sergiopaniego

posted an update 28 days ago

Post

2297

This super detailed tutorial by @Paulescu is pure gold 🪙 "Fine-tuning a Small Language Model for browser control with GRPO and OpenEnv"

LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝

https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser

qgallouedec

submitted a paper to Daily Papers about 1 month ago