🛠️How to Train a Coding Model to Paint With RL and TRL
TL;DR
Hugging Face walks through training a coding LLM to generate watercolor-style art by writing p5.js code, using TRL's GRPO trainer with LoRA on Qwen 3.5-35B. The reward stack layers gate checks, a vision-model judge, and the HPSv3 aesthetic model, and every adapter, rollout, and dataset is published open.
Hugging Face walks through training a coding LLM to generate watercolor-style art by writing p5.js code, using TRL's GRPO trainer with LoRA on Qwen 3.5-35B. The reward stack layers gate checks, a vision-model judge, and the HPSv3 aesthetic model, and every adapter, rollout, and dataset is published open.
Key Points
Reproduces Surya Narreddi's original approach using fully open tools: TRL, OpenEnv, and Qwen 3.5-35B with LoRA
The model writes JavaScript through p5.brush, a drawing library layered on p5.js, to produce each painting
Reward signal combines gate validation, a length bonus, a vision-model pairwise judge, and the HPSv3 aesthetic scorer
Three training runs with different reward weightings show the pairwise judge sharpens style and pigment coverage most
All trained adapters, rollouts, the reference pool, and the training environment are released publicly
Why It Matters
It's a clean, reproducible template for RL-training any coding model against a subjective reward — swap the aesthetic judge for a different scorer and this same recipe works outside watercolor painting.
Quick Facts
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,507 builders reading daily.