A pretrained robot policy that works half the time needs more reliable behavior before deployment. Even 99% success needs a declared task scope, quality bar, speed and evaluation protocol. This guide explains how people improve reliability: the hill-climbing methods, the textbook RL ideas under each one, and recent developments since September 2025.
The Awesome list on GitHub · The hands-on course · Search all papers
If you read seven things from the past year, read these. Each one is a different way to climb, and together they illustrate several major approaches.
Sunday's framing of hill climbing: scale pretraining until in-house gains carry over to unseen homes, then climb reliability in-house. Proposes the "Solve" reporting standard.
Pretrains ACT-2 on sensorized human data from Sunday's own capture hardware, then post-trains on repeated in-house Memo runs and recoveries (update rule undisclosed). Laundry: 99.1% over 785 autonomous attempts in unseen homes, one fixed checkpoint, zero per-home adaptation.
Advantage-conditioned retraining of a VLA: a value model tags chunks positive or negative, the model is retrained with its supervised loss on that tag, and at test time it is conditioned on positive (optionally with classifier-free guidance).
Fits a distributional value, binarizes n-step advantages into text conditioning, and supervises the whole flow VLA; CFG is optional. Same-data real laundry: about 2x AWR throughput; success ~96% versus AWR ~91%, PPO ~75% (chart reading).
Off-policy RL around a VLA with no RL gradient through it: propose chunks, apply a small bounded edit, execute the best under a Q-ensemble. The VLA itself keeps training on its own supervised loss.
Samples 8 pi0.5 chunks, adds a tanh-bounded learned edit to each, runs the argmax-Q of 16; the VLA keeps training on its supervised loss, with human interventions. Real, 8 tasks: SFT 20.5/30 → 30/30 each, 19.1 min average (HG-DAgger 22.1/30).
Black-box search (random search, CEM, zeroth-order) for one fixed input-noise vector for a frozen diffusion or flow policy, training no new networks. Simple enough to run before RL: it beat Gaussian noise sampling on 46 of 51 sim and real tasks.
Replaces per-chunk Gaussian noise in a frozen diffusion or flow policy with one fixed noise vector found by black-box search. Real Franka block/cube pick: 80 → 98% success on 50 held-out episodes after 150 search episodes (~30 min).
Noise-space steering constrained to what the base policy can already do, trained only in a scanned digital twin, transferred with zero real RL.
Freezes a real-demo flow policy; in a scanned sim twin, PPO learns to steer its input noise plus a small residual, staying near the base's action support. Real Franka/LEAP, 8 tasks, no real-world RL: 37.8% → 89.9% vs Residual-RL 59.5%.
Train a verifier on your own evaluation rollouts and rerank N samples at deploy time. No policy weight updates.
Trains a small verifier on labeled evaluation rollouts of the deployed policy, then reranks 10 samples from the frozen diffusion or flow policy. Real ALOHA, 5 tasks, 20 trials each: mean success 38% → 87% (chart reading).
A controlled study of advantage-guided (RECAP-style) post-training that separates how the advantage is built, calibrated and used, screens each choice with offline diagnostics, then tests the recipe on four real bimanual tasks.
Ablates separately how critic-derived advantages are built, calibrated and used; n-step TD, per-value-bin normalization and soft exponential weights on the pi0.5 loss worked best. Real bimanual, 4-task mean success: SFT 0.11, source-weighted DAgger 0.45, continuous weighting 0.74.
183 papers and posts dated in the last 14 days (2026-09-21 to 2026-10-05). The list has a documented weekly-update workflow; this page is rebuilt from its data.
In 2025-26, most last-mile improvement of robot policies can be read as one loop: deploy, measure, find failures, improve, re-measure. The families mostly differ in which step they put effort into and which part of the system they change. Pretrained VLA and BC policies, once fine-tuned on a task, usually start well above zero, so in many recent papers the optimizer is not the main bottleneck. The bottlenecks are the signal (success checks, rewards, advantages, verifiers), trustworthy evaluation, and human and robot time. Many recipes that work keep the large model frozen and climb with something small: a noise vector, a residual or edit head, a critic, a verifier or a curated data slice. Others update the whole model but limit how far it moves: they restart each round from the pretrained checkpoint (RECAP), or keep the supervised loss and change only which data or advantage label it copies. Full-model RL with no KL anchor also works when the starting policy already succeeds often enough (SimpleVLA-RL). Under the new names, the classic ideas of perturb-and-score search and reward-weighted averaging are still there.
Narrative drawn from the linked papers and posts.
Begin from a pretrained policy that already succeeds sometimes: a BC diffusion or flow policy, ACT, or a VLA such as pi0, pi0.5 or OpenVLA-OFT. If it never succeeds (0%), most methods have nothing to climb: on five RoboTwin 2.0 tasks, SimpleVLA-RL on an OpenVLA-OFT base model given no task demonstrations (0-trajectory SFT) stays at 0% after RL, because every rollout earns zero reward. The exceptions change how they explore instead: PDE has a VLM rewrite the instruction and climbs LIBERO-PRO from a 0% start.
Run the policy to collect episodes. They can come from the real robot, a fleet, a GPU simulator, a scanned or generated digital twin (the sim-to-real family), or a learned action-conditioned video world model (the world-model family). Each has a cost: real episodes need resets and supervision, and simulated ones can be wrong in ways the policy will exploit.
Turn each episode into a number. The options are a binary success check (a human keypress, a hand-written detector, a classifier), a dense progress or value model (the reward family), or a VLM judge. For comparing checkpoints, you want an evaluator that ranks policies the same way reality does, reported as Pearson r and MMRV against paired real trials (the world-model/evaluation family). Offline validation action MSE is a poor proxy: when SIMPLER ranked 6 checkpoints on 3 Google Robot tasks, MSE averaged Pearson r 0.308 against real success, vs 0.924 for visual-matching sim.
Locate where and why the policy goes wrong. The tools are dips in a progress model (Dyna), runtime monitors, either calibrated on successful runs (CoRe) or triggered when the critic's Q jumps away from its recent moving average (JITI, which targets lapses in execution quality rather than outright failures), human takeovers (the HITL family), VLM diagnosis of failed videos (F4R, PDE), and per-stage predicates (Temporal GRPO). This step decides where the next batch of data, search or corrections goes.
Apply an update. Each family is a different update operator:
Evaluate on held-out episodes and start states that were not used for search or training. Report trial counts and confidence intervals, and track speed, intervention rate and per-task results, not just average success. Then repeat. Many loops also distill what worked back into the base (PLD, VERITAS); Q-Planning instead refits only its critic each round and leaves the policy frozen. Others restart each round from the pretrained checkpoint to avoid drift (RECAP). Restarting does not fix a skewed data mix, though: DEED, a RECAP adaptation on a humanoid, restarted every round and still fell from 42% (21/50) to 22% (11/50) in round 2 (its round-1 gain over the 32% starting policy was not statistically conclusive); its authors attribute the drop to self-generated rollouts crowding out the teleoperation demos.
Reward and value models serve steps 2-3 for everyone else. World models and twins make steps 1-2 cheap. HITL methods put a person into steps 3-4. Industry recipes are mostly this loop built as infrastructure: fleet logging, auto-grading, targeted retraining.
Near the top, the hard part is knowing whether you actually climbed. An evaluation with only 20–50 trials cannot reliably distinguish 95% from 99% success.
gives a 95% Wilson interval of [88.6%, 100%]. A perfect observed score does not prove perfect reliability.
successes at a fixed evaluation size give a Wilson interval whose lower bound reaches 95%.
successes at a fixed evaluation size give a lower bound of 99%. Correlated attempts or changing task conditions need additional analysis.
Sunday’s laundry result gives a 95% interval of [98.2%, 99.6%]. Sunday’s “Solve” label rests on more than the interval: a declared scope and zero per-home adaptation.
Wilson score interval at 95%, assuming independent trials with a constant success probability and a fixed evaluation size. Report it next to the observed rate, keep search separate from evaluation, and use paired start states when comparing methods.
Many 2025–26 methods recombine a common set of textbook ideas. This matrix shows which ideas each family is built on, with the classic references. The standard text is Sutton & Barto (free online).
| Idea · core equation · classic methods | Black-box search | On-policy PG | Off-policy critics | Residual & steering | Advantage-weighted BC | Human corrections | Test-time search | Rewards & progress | World models & eval | Sim-to-real | Industry loops |
|---|---|---|---|---|---|---|---|---|---|---|---|
Perturb and scoreθ' = θ + σε; keep the best (hill climbing) or step along Σᵢ R(θ+σεᵢ)·εᵢ (ES, ARS)Finite differences, ES, CEM, CMA-ES, Bayesian optimizationKohl & Stone 2004 · CMA-ES tutorial · OpenAI ES · ARS | |||||||||||
Reward-weighted averagingπ_new(a) ∝ π_old(a) · exp(R(a)/λ); AWR puts the advantage A in place of R, PI² the negative path cost −SRWR, PI² / MPPI, AWR. Filtered BC is the 0/1 versionPoWER · PI² · AWR | |||||||||||
Likelihood-ratio policy gradient∇J = E[∇log π(a|s) · A]REINFORCE → TRPO/GAE → PPO; GRPO group baselinesREINFORCE · TRPO · GAE · PPO · GRPO | |||||||||||
Bellman backups on replayQ(s,a) ← r + γ Q(s', a'), a' ~ π(·|s') (DQN: max over a')Q-learning, DDPG/TD3, SAC, RLPD; chunk criticsDQN · DDPG · TD3 · SAC · RLPD | |||||||||||
Conservative / in-sample criticsdon't trust Q on actions the data never showsCQL, IQL, Cal-QL, anti-explorationCQL · IQL · Cal-QL | |||||||||||
Residual learning on a frozen basea = a_base + f_θ(s); bounded variant: a_base + α · tanh(u)Residual RL, ResiP, Policy Decorator; noise steeringResidual RL (Johannink) · Residual Policy Learning (Silver) | |||||||||||
On-policy labelslabel the states the learner actually visitsDAgger, HG-DAgger, intervention-weighted regression, RLIFDAgger · HG-DAgger | |||||||||||
Greedy selection & guidancea* = argmax over samples of Q(s,a); guidance: (1+w)∇log p(a|s,c) − w∇log p(a|s)Best-of-N, V-GPS, classifier-free guidance, MPC/MCTSClassifier-free guidance · IDQL · V-GPS | |||||||||||
Reward shaping & learned judgesr' = r + γΦ(s') − Φ(s)Potential shaping, preferences, success classifiers, progress modelsPotential-based shaping · Human preferences · VICE-RAQ | |||||||||||
Model-based RL / Dynalearn p(s'|s,a); practice or plan in imaginationDyna, Dreamer, MBPO, action-conditioned video modelsDyna · MBPO · Dreamer | |||||||||||
Domain randomization & twinsmake reality one more sample of the simulatorDR + PPO, real-to-sim scans, BC-anchored sim RLDomain randomization · ADR (Rubik's cube) |
Pick by what you have: how often the base succeeds, whether you have a simulator, how much robot time and human time you can spend, and whether you can touch the weights.
Narrative drawn from the linked papers and posts.
Each family answers one question. For each: where it sits in the loop, the ideas it is built on, when to use it, what changed in the last year, and its must-know works with their headline numbers.
How do I improve a policy when all I can do is run it and get a score, for example a success flag, and I cannot or do not want to backprop through it?
Treat the policy, or a few knobs around it, as a vector. Try perturbed copies, score each with one number per episode, and move toward the copies that scored well. No backprop through the policy, no critic, no action log-probs.
All 40 entries on GitHub → (26 from Sep 2025 on, 8 must-know) · filter the index
Use it when (1) you only have a scalar, trajectory-level score such as success, success times time, or pass/fail, (2) the policy's log-probs are unavailable or expensive (diffusion, flow, action chunks, deterministic controllers, closed APIs), and (3) you can shrink the search to a small space: controller gains, impedance, a noise vector, a residual or last-layer head, a prompt. Pick BO for tens of expensive real trials over a few knobs.
The biggest change is scale: evolution strategies went from small networks to full-parameter fine-tuning of billion-parameter models, and Evolution Strategies at Scale (Qiu et al., Sep 2025, ICML 2026) used a population of 30 to beat grid-tuned PPO and GRPO on the Countdown reasoning task for all seven LLMs it tried (0.5B-8B), a language-model result with no robot experiments. EGGROLL (Nov 2025) made large populations cheap, reaching up to 91% of pure batch-inference throughput with low-rank perturbations. On robots the search space got smaller and smarter: the Golden Ticket (Mar 2026) searches for one fixed initial-noise vector for a frozen diffusion or flow policy and improved 46 of 51 simulated and real tasks, while ENPIRE (Jun 2026) has coding agents search over training recipes using real-robot rollouts. Black-box search is also used as a second stage after gradient training, as in EvolSAC and TD-ES.
Survey framing controller tuning as episodic policy search where a GP surrogate picks each trial; reviews 110 hardware papers and releases TuneControl. 57 of 110 use vanilla BO; sim 4-parameter cart-pole: LogEI 4310±48 vs Sobol 4647±190 (median±IQR cost, lower better).
Coding agents edit training recipes, train, run real rollouts scored automatically, and keep what raises success. Pin insertion: 8 agent/robot pairs reached near-perfect success in about 40 min vs over 1.5 h with one (success counts up to 8 retries).
Replaces per-chunk Gaussian noise in a frozen diffusion or flow policy with one fixed noise vector found by black-box search. Real Franka block/cube pick: 80 → 98% success on 50 held-out episodes after 150 search episodes (~30 min).
Shares matrix multiplication across low-rank evolution-strategies perturbations. With pre-generated noise, one GH200 benchmark (8192-dimensional bf16 layer, batch 1024) reaches 91% of maximum batch-inference throughput, versus PPO 34% and OpenES 0.41% (chart reading).
Continues from a PPO checkpoint with antithetic evolution strategies whose bounded triangular per-coordinate noise keeps candidates in a small box around the weights. Sim IQM success, 3 Isaac Lab Franka tasks: PPO 67.2%, Gaussian ES 73.7%, TD-ES 85.0%.
Perturbs all weights of 30 model copies with tiny Gaussian noise, scores each with an outcome-only reward, and steps toward high-scoring noise, with no backprop or log-probs. Countdown, Qwen-2.5-3B (language-only): ES 60.5 vs Dr.GRPO-v 43.8 (base 10.0).
Pretrains SAC on a dense surrogate reward, then fine-tunes the policy with SNES using the true task score as fitness. Sim cartpole swing-up time (10-agent mean): 2.085 s → 1.135 s, vs 1.565 s after 1000 more SAC episodes.
Tutorial framing derivative-free robotics methods as one loop: sample Gaussian perturbations, score them, move toward better samples; the sample weighting recovers MPPI, CMA-ES or CEM-like updates. Plots only: on 4 Hydrax sim tasks, MPPI-CMA consistently outperforms MPPI.
How do I take a BC or SFT policy (diffusion, flow, ACT or a 7B VLA) that succeeds 50-90% of the time and push it toward 95-100% using only a success signal, a resettable simulator, and fresh on-policy rollouts?
Roll out the pretrained policy as a stochastic policy, raise the log-probability of rollouts that beat a baseline, and clip each step so the pretrained skill survives.
All 70 entries on GitHub → (58 from Sep 2025 on, 11 must-know) · filter the index
Use it when you have (1) a pretrained policy that already succeeds sometimes, roughly 10-90%, (2) a simulator or digital twin with cheap resets and an automatic success check, and (3) GPUs for large-scale parallel simulated rollouts. It is the right tool for pushing a competent SFT policy past its imitation plateau: in RL4VLA, SFT plateaued at about 16k demos with 0.781 in-distribution success, and PPO reached 0.938 (appendix Table 1). Do not run it directly on a real arm with a small rollout budget; there, prefer the off-policy, residual or human-in-the-loop families.
A year ago (May to July 2025) the open question was whether PPO or GRPO could train a VLA at all: RIPT-VLA, VLA-RL, RL4VLA and TGRPO showed it could in simulation, while ReinFlow and FPO showed how to run PPO-style updates on flow-matching policies. From September to October 2025 this became standard recipes and infrastructure: SimpleVLA-RL applied critic-free GRPO with binary success rewards to OpenVLA-OFT, πRL made PPO work on the pi0 and pi0.5 flow VLAs, and RLinf-VLA provided shared training infrastructure. With LIBERO close to saturated (several methods report about 97-99% average), 2026 work focuses on where the learning signal comes from, avoiding forgetting, and real-world transfer.
Post-trains a reposing-pretrained PPO arm-hand policy: BC-distill the actor into new observations, warm up a fresh critic with the actor frozen, then PPO at actor LR 1e-5. Sim peg insertion: 5/5 seeds reach ADR 50 vs 0/5 at LR 1e-3.
Keeps the flow decoder deterministic and adds exploration noise only to the output action, AR(1)-correlated within a chunk with learned, group-bounded scales for position, rotation and gripper. ManiSkill pi0.5 OOD average: SFT 25.9, piRL 49.3, action-space Gaussian 65.9, StructRL 70.3.
A frozen VLM proposes checkable stages compiled into predicates; group advantages are computed per stage over rollouts that reached it, so good early actions escape blame for later failures. Sim RoboTwin 2.0, OpenVLA-OFT, macro-average: 75.8 vs SimpleVLA-RL 68.8 (SFT 38.3).
Chunk-level PPO on a pretrained ACT (CVAE removed, learnable Gaussian noise), with a KL term between consecutive updates and a reward penalty for leaving the frozen BC policy. Sim Contour and Square Assembly: ACT 60.0%/51.2% → 100.0%/98.2%.
A VLM rewrites task prompts to discover successful rollouts; PPO trains with a mixture including the deployment prompt. Sim LIBERO-PRO, pi0.5 zero-start Regime 3 task, step 120: 81.8% vs action-noise PPO 0%.
Fine-tunes a VLA on each new task in sequence with GRPO on sparse 0/1 success and a rank-32 LoRA adapter, with no replay or regularizer. Sim LIBERO-Spatial: AVG 81.2, forgetting 0.3, vs 7.3 and 40.9 with full fine-tuning.
Flow policy gradient using exp(old CFM loss - new CFM loss) as the PPO ratio, clipped per (tau, eps) sample, with zero-noise test-time sampling. Zero-sampling alone lifts the sim Robomimic Can base policy from 10.00% to 71.35% before RL.
Makes pi0/pi0.5 flow chunks PPO-trainable by adding a learned per-step noise net (Flow-Noise) or an equal-marginal SDE (Flow-SDE) for exact log-probs, dropped at deployment. Few-shot LIBERO (sim): pi0 57.6% → 97.6%, pi0.5 77.1% → 98.3%.
BC-trains a diffusion policy on 29-400 teleop demos per task, then applies PPO-clip at every denoising step, offline with IQL advantages and an OPE gate, then online with GAE. Real, 8 tasks: 100 mean vs DP3 67.8; 1000/1000 trials.
Infrastructure plus recipe for VLA RL: simulators, rollout and trainer share GPUs, split them or run pipelined, with chunk-aware PPO and GRPO tricks. ManiSkill 25 tasks, ID/OOD: OpenVLA 53.91/39.10 → GRPO 84.38/75.15 → PPO 96.09/81.93.
GRPO on a tokenized-action OpenVLA-OFT: 8 rollouts per scene, binary success reward on every action token, group-normalized advantages, no critic, and all-agree groups dropped. LIBERO average 91.0 → 99.1; with one demo per task, 48.9 → 96.9.
How do I take a pretrained robot policy that works 40-80% of the time to 95-100% with minutes-to-hours of real robot data, reusing every demo and rollout I already have?
Learn a Q-function from every transition you have (demos, human corrections, old rollouts, new rollouts), then use it to pick, edit or push the policy's actions toward higher value.
All 123 entries on GitHub → (70 from Sep 2025 on, 12 must-know) · filter the index
Use this family when you already have a decent policy (roughly 40-80% on one task), each real episode is expensive, and you can get a binary success signal. Off-policy RL reuses every demo, correction and failed rollout, so it usually needs far fewer robot minutes than on-policy PPO/GRPO.
The target policy changed: the 2023-24 lineage (RLPD, SERL, HIL-SERL) trained small task-specific SAC policies, while 2026 work wraps the same off-policy machinery around pretrained flow VLAs such as pi0.5 and pi0.6 (EXPO-FT, RL Token, ALOE, LWD, RAPolicy). Critics increasingly score whole action chunks, following Q-chunking (Jul 2025), and follow-ups such as DQC, AQC and ACH decide how long to commit to a chunk. Other work questions how much offline critic pretraining and offline-data replay really help (WSRL, Three Regimes), and the human-in-the-loop HIL-SERL loop is being automated or scaled: AutoSERL replaces the live intervener with a one-demo scripted corrector, and LWD runs the loop on a 16-robot fleet.
A rollout copy of pi0.5, re-synced after each small update group, collects data while the learner trains an IQL-style chunk critic and one-step flow actor only on stored actions. Real, 4 tasks, 20 trials each: 86.3% average vs EXPO-FT 50%.
Post-trains pi0.5's action-expert LoRA with flow-matching BC on demos, successes and human corrections, plus ensemble-critic advantages relative to a frozen reference instead of absolute-Q maximization. 9 real chemistry-lab tasks: 98.3% vs demo-only pi0.5 67.8%, ConRFT (Octo) 7.8%.
Shows demo-pretrained critics barely beat random ones, then rolls out an ensemble of diverse policies trained on the same data so the critic sees contrasting actions. Six Robomimic/OGBench sim tasks: reported 1.26x average fine-tuning improvement over naive Q pretraining.
Replaces HIL-SERL's human interventions with one-demo rules: a planner corrects drift beyond 2 cm; stalls trigger partial demo replay. Real, no start randomization: 50/50 on all 6 tasks in 8-45 min; SERL (20 demos, same time): 0-20/50.
Predicts the clean action from a noisy flow point, nudges it along the critic's gradient, and regresses velocity toward it alongside BC, without ODE backprop. Real dual-arm AirPods assembly: 88% offline vs 80% for RLT/QAM (BC 64%).
Samples 8 pi0.5 chunks, adds a tanh-bounded learned edit to each, runs the argmax-Q of 16; the VLA keeps training on its supervised loss, with human interventions. Real, 8 tasks: SFT 20.5/30 → 30/30 each, 19.1 min average (HG-DAgger 22.1/30).
A 16-robot fleet logs rollouts and takeovers with sparse rewards; off-policy RL bootstraps the critic from an uncertainty-adaptive quantile of a distributional value model, with QAM policy extraction. 8 real tasks, mixed success/stage score: 0.95 vs SFT 0.76, RECAP 0.85.
Compresses frozen pi0.6 embeddings into one 'RL token'; a tiny TD3-style actor-critic then edits the VLA's own sampled action chunk under an L2 penalty, only in each task's critical phase. Screw critical-phase success: 20% → 65%.
Learns a chunk-level Q of the current pi0.5 policy by TD bootstrapping with a pessimistic ensemble, instead of Monte Carlo state values, to weight policy updates. Phone Assembly, 60 trials, 10.0% BC start: ALOE 95.0, AWR 80.0.
Trains a second flow toward pi_beta(a|s)·exp(tau·Q(s,a)) via adjoint matching, using the critic's action gradient without backpropagating through the sampler. Offline RL over 50 OGBench tasks (sim): QAM 44 vs ReBRAC 40, FQL 36, FBRAC 11.
Compares the pretrained policy's return with the dataset's average: if the policy is better, protect it with warm-up rollouts; if the data is better, replay offline data. Regime predictions matched outcomes in 45/63 D4RL settings (71%).
Critic and policy act on h-step action chunks with an unbiased h-step TD target, while the policy stays near a flow-matching BC model of the data. OGBench, 25 tasks, offline → online: QC 52→86 vs FQL 37→58.
How do I push a big imitation, diffusion or VLA policy from 50% to 95%+ when I can't cheaply fine-tune or backprop through it, without breaking what it already does well?
Freeze the big pretrained policy and train a small add-on with RL: one that corrects its actions (residual/edit) or one that picks the noise it is fed (steering).
All 81 entries on GitHub → (72 from Sep 2025 on, 12 must-know) · filter the index
Use it when you already have a decent base (roughly 20-80% success) that you can't or won't fine-tune: a VLA too big to backprop through with RL, a closed checkpoint that allows only forward calls, or a chunked diffusion or ACT policy whose likelihood is awkward for policy gradients. It also fits when you want the base's generality and recovery behaviors kept intact, and when your budget is minutes to hours of real robot time with sparse success labels.
Over the last 12 months, bases moved from task-specific diffusion or ACT policies to frozen generalist VLAs (pi0, pi0.5, GR00T, SmolVLA, OpenVLA-OFT). Freezing is common because RL through a multi-billion-parameter model is costly, and an untouched base keeps behaviors the new task data never exercise, such as recovery from failure (the case PPS makes for steering a frozen base). Real-robot add-ons are usually trained off-policy (ResFiT, DICE-RL, EXPO-FT, MiDAS); ResFiT reports about 200x the sample efficiency of a PPO residual in simulation. The limits of each branch became clearer: noise steering stays inside the base's support (on one-demo LIBERO-Long, DSRL stays at the BC start's 33.5 while MiDAS's residual actor reaches 91.2), and RFS pairs noise steering for global exploration with a residual for local correction. The EXPO line pairs a large proposer with a small editor and picks the argmax-Q action: Real-Time EXPO-FT reports 42% → 97% on four dynamic real-world tasks with online data capped at 10 minutes.
Finds noise directions that most move a frozen generative policy's actions using finite differences at 64 states; SAC steers those directions. Sim RoboMimic Diffusion-Square, final-five-evaluation mean, one seed: 0.878 vs 0.716 for full-latent steering.
Slow pi0.5 proposes 32 chunks from a stale observation; a fast policy adds bounded edits from the fresh one; a Q-ensemble picks one. Real, 4 dynamic tasks, 10 min online data: 12.5/30 → 29/30 (EXPO-FT w/ RTC 25/30).
LoRA-tunes pi0.5 on one demo, then freezes it and trains a sparse-reward off-policy edit actor-critic that outputs the full executed chunk, so it can leave the base's action support. LIBERO-Long, 1 demo: 91.2 vs DICE-RL 89.3, DSRL 33.5.
Trains a PPO residual on a frozen per-phase pi0 for each subtask, multiplying each phase's reward by a learned predictor of next-phase success. Sim wrench-on-nut full task: 85.6±3.9% vs 54.5±3.6% with plain per-phase residuals.
Freezes a real-demo flow policy; in a scanned sim twin, PPO learns to steer its input noise plus a small residual, staying near the base's action support. Real Franka/LEAP, 8 tasks, no real-world RL: 37.8% → 89.9% vs Residual-RL 59.5%.
A noise-steering policy on a frozen diffusion policy, conditioned on a per-episode code, learns distinct modes via a DIAYN-style classifier reward, kept as an RL fine-tuning bonus. ManiSkill sim Avoid, success ~1.00: DSRL keeps 2/24 modes, DPPO 6.33/24, DSRL[BMD] 10/24.
Trains a SAC-style noise actor for frozen pi0, inverting the flow decoder by fixed-point iteration so human teleop corrections become noise-space targets. Real AgileX Piper, 4 tasks: average success 20% → 90% (~66 min training) vs DSRL 55%, DAgger 60%.
Freezes a diffusion/flow BC prior, trains a residual on the same noise with TD3+BC, then executes the best-Q sample. Real UR5 BeltAssembly, 30 trials/checkpoint: 56.67% → 93.33% after ~420 training episodes (chart reading).
Speeds up residual SAC with 20K base-policy replay transitions and LayerNorm in the critic. ManiSkill sim PegInsertionSide reaches 90% success around 0.3M environment steps versus roughly 2.3M for Policy Decorator (chart reading).
One PPO policy outputs both the starting noise for a frozen flow-matching base and an additive residual, so it can switch modes and leave the demo manifold. Sim, 6 IsaacLab dexterous tasks: 0.861 vs base 0.250, DSRL-only 0.483.
Trains a bounded Gaussian residual with SAC on a frozen VLA, lets the VLA drift before the residual recovers, and distills successes into it. LIBERO Spatial/Object/Goal avg: OpenVLA-OFT 91.8 → 99.2; real Franka cube pick-up: 30/30 vs 10/30 (human data).
Keeps an ACT chunked BC policy frozen and trains a small per-step residual with a tuned TD3-style off-policy recipe (demos in each batch, LayerNorm critic, n-step returns). Real 29-DoF humanoid: WoollyBallPnP 14% → 64% after 134 rollouts.
How do I push a big imitation policy (a diffusion policy or a flow VLA) up on one task using its own rollouts, human corrections and success labels, without running policy-gradient RL through it?
Keep the supervised imitation loss, but decide which actions (or which demos) the policy copies and how much, using a success label, a value function or a data-influence score.
All 63 entries on GitHub → (46 from Sep 2025 on, 12 must-know) · filter the index
Use this family when you have a decent BC or VLA policy (roughly 20-60% success) on a specific task, a way to label episodes success or failure, and ideally a teleop device for takeovers, but you do not want to backprop RL through a large flow or diffusion sampler. Start with curation (RoboDrop, QoQ or CUPID-style audits, or cheap smoothness filters) when your demos are mixed-quality.
The big shift came with pi*0.6/RECAP in November 2025. Advantage-conditioned BC, where the policy is conditioned on a binarized advantage token and classifier-free guidance is optional, suits flow VLAs whose likelihood is awkward to weight, and it joined advantage-weighted BC as a main option; RECAP names CFGRL (May 2025) as the closest prior formulation. In the next ten months at least a dozen groups adopted or adapted the recipe: GigaBrain RAMP, chi0, DexPIE, DEED, ROVE, STEAM, Robo-ValueRL, PACE, DistAL, Facet-0, CLIFT and a prizewinning LeHome Challenge entry (1st in the simulation round, 2nd in the real-world final).
Ablates separately how critic-derived advantages are built, calibrated and used; n-step TD, per-value-bin normalization and soft exponential weights on the pi0.5 loss worked best. Real bimanual, 4-task mean success: SFT 0.11, source-weighted DAgger 0.45, continuous weighting 0.74.
In one warm-up epoch, compares each sample's compressed gradient with one from the most similar clean validation frames, and drops conflicting episodes before re-post-training pi0.5. Real, 4 tasks, 20 rollouts each: 35.0% → 67.5% with automatic filtering.
Tunes closed-weight Gemini Robotics On-Device via an SFT API only, tagging the top 30% of chunks by dense-reward advantage over similar-start chunks as 'advantage positive'. Real Unitree G1, 2 cycles of 100 rollouts: Plate Handover 53% → 96%.
Clusters action chunks by estimated progress and joint state; within a cluster, positive-scored chunks (progress gain plus episode outcome) form correction targets for negative chunks, which are also suppressed. Real pi0 clothes folding: 36 → 67/100 (AWR 48).
Data-efficient GR00T N1.6 post-training plus a RECAP-adapted refinement (text advantage prefix, VLM value function) for chip restocking on a Unitree G1, ~108 min of data. 50 real episodes each: SFT 32%, iteration 1 42% (CIs overlap), iteration 2 22%.
An ensemble of small VLMs predicts time offsets between expert frame pairs (forward and reversed); the ensemble minimum gives per-frame advantages for CFGRL training of pi0. Real towel folding: 92.3% vs one-iteration RECAP-style baseline 55.6%, BC 33.3% (trials unstated).
Splits each human intervention into autonomous rollout, human adaptation and human recovery, placing the failure penalty after adaptation; an optimistic expectile critic drives RECAP-style conditioning. Iteration 1, same robot data, Erase/Bread: 60.0%/73.3% vs RECAP 50.0%/66.7%.
Trains pi0.5 with 60 demos per task, offline advantage-weighted flow matching from a learned critic, then human-in-the-loop correction rounds with the same weighting. Four real long-horizon tasks, 20 trials each: 18.8% → 40.0% → 85.0%.
Fits a distributional value, binarizes n-step advantages into text conditioning, and supervises the whole flow VLA; CFG is optional. Same-data real laundry: about 2x AWR throughput; success ~96% versus AWR ~91%, PPO ~75% (chart reading).
Keeps episodes above a rising return threshold, including human-corrected successes, and trains pi0 with reward-weighted flow matching. Three real tasks, two robots: 23.3 percentage points over offline BC (chart reading); about 1.5 hours of real training.
Scores each demo's effect on closed-loop success by chaining influence functions with a REINFORCE estimate from labeled eval rollouts, then drops negative-influence demos and retrains. Real TuckBox mass shift, 66% filtered: base 0%, CUPID 84%, oracle 88% (bar readings).
Trains one flow policy by BC with an optimality token; classifier-free guidance samples reference policy times a non-decreasing advantage function. Sim ExORL walker-stand, same IQL critic: 782±8 vs AWR 603±8. OGBench: mixed vs AWR, beats goal-conditioned BC on most tasks.
My imitation policy works most of the time but fails in states the demos never covered. How do I spend a small amount of human time so the next version fixes exactly those failures?
Run the policy, let a person (or now a machine) step in where it goes wrong, and train on those on-policy fixes so the policy learns to handle the states it actually reaches.
All 80 entries on GitHub → (47 from Sep 2025 on, 12 must-know) · filter the index
Use this family when you have a decent imitation policy (roughly 30% to 80% success) that fails in specific, repeatable places, and a person who can recognize and fix those failures. It is the default last-mile tool when you have no reward function, when the task is long-horizon or contact-rich, and when you want the policy to stay close to its pretrained behavior.
Over the last 12 months, the policy being corrected became a VLA: pi0/pi0.5-class models replace small BC-RNNs, and the loops are built as systems. SOP streams fleet corrections to a cloud learner, and HELP splits the human work between a teleoperator and a floor operator who together supervise twelve robots. Methods also get more out of each correction: FlowPRO trains on paired failed and corrected segments with a DPO-style loss, and FlowDAgger maps corrections into a frozen flow policy's noise space instead of retraining it. Machines increasingly take the intervener's role (UniIntervene, Skill-Space Shooting), and how and when a correction is collected is now studied directly (RaC's recover-then-correct rule, TimelyDAgger's takeover timing).
Keeps the policy running during corrections, blending human and policy actions per DoF group and giving the human more control when they push hard, disagree, or the policy is uncertain. Final round, real Almond Scooping: 50% vs HG-DAgger 15%.
A learned value model halts a pi0.5-class policy when its score drops below zero; Gemini 2.5 Pro picks and checks a repair skill; successful repairs become training data. Real Coffee mean progress: 71.25% vs 46.25% with extra human demos.
Robot-gated DAgger: asks for takeover when a PCA residual on VLA features exceeds a threshold that the expert's first post-takeover actions raise or lower. pi0.5 StackCube-Green (ManiSkill sim, oracle expert): 81 vs HG-DAgger 29, offline BC 49.
Operators correct HIL-SERL-style real-world RL by hand-guiding the arm via admittance control sharing the policy's action space, limits and impedance tracker. Vs HIL-SERL, four real insertion tasks: 62%-86% fewer interventions; 23%-48% lower cycle time, mostly from the new controller stack.
Learns the return distribution with a truncated-quantile critic and acts on its lower tail, because takeover timing is inconsistent (over half of repeated marks differ by 10+ frames). Four sim and two real tasks: 22% higher success than RLIF.
Inverts the flow sampler to find the noise that would have produced the operator's correction and trains a small noise policy on it, VLA frozen. MetaWorld, pi0.5: 0.78 vs SFT 0.71; held-out skills 0.88 (base 0.96) vs SFT 0.02.
A progress critic (VLAC-CUT) cuts autonomous pi0.5 rollouts into progress, recovery, idle and failure-inducing segments; only progress and recovery segments join HITL recovery data for retraining. Real round 2, Microplate: 42 tasks/hr at 90% vs HITL-only 25 at 60%.
Adds an intervention agent to HIL-SERL: a VLM-based proxy value model and a temporal value-risk critic detect stagnation, then steer back to a remembered high-value state. 5 real UR7e tasks: HIL-SERL 81% success at 34.3% intervention → 88% at 14.6%.
Operators roll back and teleoperate fixes, giving paired failed/corrected segments for a critic-free DPO-style flow-matching loss with proximal and SFT terms. Real Pack, pi0 base, 3 rounds: 99% vs 94% for a RECAP-style advantage-conditioning baseline, DAgger 88%.
A human corrects failure-prone policy rollouts inside a learned action-conditioned world model, with rollback and branching from cached states; the corrections post-train the policy. 3 real tasks, DP and pi0: +37.9 points over base, +19.0 over world-model self-training without humans.
Ten real AgiBot G1 robots run pi0.5 with human takeovers; a cloud learner mixes on-policy data with old demos, publishing weights every 25 steps. 3 h on-policy interaction: 0.571 → 0.800; 80 more demo hours: 0.576 → 0.612.
Changes HG-DAgger takeovers: the human first recovers to a familiar state, then finishes the sub-task, and the episode ends with the takeover. Shirt-hanging: 78.3% with 5 hours of data vs ALOHA Unleashed's 75.0% with ~89 hours (RaC's estimate; cross-paper).
My policy sometimes does the right thing but not reliably. Can I raise its success rate without retraining it, just by choosing better among its own samples at run time?
Leave the policy's weights alone and spend extra compute at deployment: sample several candidate actions, score them with a critic, verifier, world model or monitor, and run the best one (or nudge the sampler toward it).
All 143 entries on GitHub → (122 from Sep 2025 on, 12 must-know) · filter the index
Use it when (1) your policy is stochastic (diffusion, flow or sampled tokens) and pass@k is clearly above pass@1, i.e. it sometimes does the right thing, (2) fine-tuning the policy is expensive or risky (a large VLA, a shared checkpoint), and (3) you can afford some extra latency or can batch samples off a shared prefill.
Verifiers moved from generic to policy-specific: V-GPS (2024) reranked samples with a generic offline-RL Q-function and RoboMonkey (2025-06) with a 7B VLM verifier trained on synthetic preferences, while much 2026 work trains a small critic on the deployed policy's own labeled rollouts (UF-OPS, Q-Planning). For flow policies, gradient guidance started to replace sampling many candidates (QGF, Jun 2026), and gating schemes such as VLA-ATTC and JITI spend extra compute only when it looks useful. Runtime failure monitors became part of the steering loop (CoRe, RL2-VLA). Many of these are single-paper results with few real-robot trials per task, and few methods have been compared head to head.
Pairs each human takeover window with a frozen pi0.5 chunk from the same state, trains a small Bradley-Terry scorer on the pairs, and uses it to guide sampling. Four real Franka insertion tasks: 90.5% vs frozen 69.0%, intervention BC 84.5%.
Trains a critic that only values finishing the active subtask (reward at subtask end, no bootstrapping across boundaries) and reranks 8 samples from a frozen pi0.5 with it. Real bimanual, 24 trials per task: 35.4% → 66.7% average.
Keeps the policy frozen as a proposal, scores 32-64 candidate chunks with a separate ~1B Q-function, runs their softmax-Q-weighted average, and retrains only the critic. Real robot, 5 iterations: stack-cups 40% → 90% vs 55% for SFT on successful rollouts.
Flags out-of-distribution states via a Mahalanobis test on the action-expert latent, rewinds to a recovery anchor, imagines the frozen policy's moves, and plans minimal object restorations; training-free. Sim LangSwitch (ManiSkill) switch average: pi0 18.3 → 87.5, pi0.5 35.5 → 92.7.
Freezes a BC flow policy and IQL critic; each flow step adds grad Q, evaluated at a one-step estimate of the finished action, to the velocity. OGBench sim, charted 20-task mean: beats tested test-time methods, slightly edges EDP.
Samples two chunks; when their DTW distance exceeds a threshold, samples 16 and picks one by knockout tournament under a relative action critic trained on auto-generated preference pairs. LIBERO-LONG, 50 trials/task: pi0 82.8% → 90.6% gated (92.2% ungated); RoboMonkey 56.5%.
A critic predicts action Q-values from raw noise, so only the best seed gets denoised. pi0.5 on five libero_90 tasks versus EXPO: inference FLOPs 37.55 TF → 4.70 TF, comparable success (chart reading).
Trains a small verifier on labeled evaluation rollouts of the deployed policy, then reranks 10 samples from the frozen diffusion or flow policy. Real ALOHA, 5 tasks, 20 trials each: mean success 38% → 87% (chart reading).
Keeps pi0 frozen, samples K instruction rephrasings times M action chunks, and picks one with a CLIP-style contrastive verifier trained on BridgeV2. SIMPLER red-teaming, ID/OOD: pi0 41.5/29.7, pi0 fine-tuned on augmented data 44.0/48.7, pi0 (rephrase) + CoVer 65.5/62.0.
Training-free: a VLM names task objects, SAM/DINOv2 features plus depth give 3D keypoints, and VLM-generated reward gradients steer frozen diffusion/flow denoising. CALVIN (sim) movable objects: 0.94 versus DynaGuide 0.26 and base 0.13 (chart reading).
Samples M chunks from a flow VLA and runs the one with the highest pseudo-count in the fine-tuning data, from a small Coin Flipping Network on internal features. RoboTwin 2.0 (pi0.5, 41 tasks, M=50): 59.3 → 64.0.
Trains an offline Cal-QL critic on frozen VLA features; reranks chunks only when Q deviates from its moving average. LIBERO-Elegant, SmolVLA: ESR 49.8 → 67.2, versus 53.8 with guidance every step (chart reading).
My robot only gets a success bit at the end of an episode, or a human has to watch every rollout. How do I get a dense, trustworthy score so RL, filtered BC or advantage conditioning can push the policy from about 60-80% toward 95%+?
These methods leave the policy alone and build the score it climbs against: a learned or zero-shot judge that says how far along a rollout is, whether it succeeded, and where it went wrong.
All 88 entries on GitHub → (61 from Sep 2025 on, 12 must-know) · filter the index
Use this family when success is sparse or expensive to judge and the task is long or multi-stage. Examples: cloth folding, insertion sequences, anything where a rollout that gets 80% of the way looks the same as one that fails at once under a success bit.
Per-task rankers (Rank2Reward) and pretrained visual-embedding rewards (VIP, trained on Ego4D human video) gave way to VLM reward models trained on large pooled datasets: Robo-Dopamine on over 3,400 hours, Robometer on RBM-1M (over 1M trajectories), RynnValue on over 7,000 hours. Training data now deliberately includes real or synthesized failures, so models can score failed rollouts and not only expert ones (RoboReward's counterfactual relabeling of successful episodes, Robometer's trajectory-comparison loss over failed trajectories). Shaping got more careful: Robo-Dopamine feeds progress in through policy-invariant shaping instead of as raw reward. In RECAP-style loops the value model itself became a lever: with the same RECAP pipeline, ViVa reports 80.0% average real success with its value vs 63.3% with a VLM value. The 'No Free Checker' survey notes that how far a policy can be optimized against a learned robot verifier is still largely unstudied, and that no shared benchmark for verifier exploitability exists yet.
A learned video progress model monitors deployment: progress dips flag errors and steer targeted recovery-data collection and retraining. Dyna-1 napkin folding: 99.4% success over a 24-hour run; Dyna says conventional pipelines stall near 80% (talk, no paper).
Predicts seconds until task completion from an instruction and a few frames, trained on over 7,000 hours of video with no preference labels. As reward for real online DSRL on frozen pi0.5: 72.5% vs Robometer 52.5%, sparse 48.8%.
Trains a classifier on the policy's own successful vs failed rollouts and adds its clipped log-odds to the sparse success reward. Real WidowX with DSRL, Pick and Place: about 0.93 vs about 0.2 for outcome-only RL (plot reading).
A comparator trained on non-robot videos (DAVIS, MOSE) matches rollout clips to one successful demo, rewarding only confident forward progress. Real pi0 with DSRL, cloth folding after 90 rollouts: 25/30 vs binary reward 7/30, Robometer 9/30.
Classifies the active action primitive to gate a mixture-of-experts value head regressing normalized remaining steps, used as reward for a residual TD3+BC policy on frozen pi0.5. Real, Round 3 Whiteboard: 18/20 vs fine-tuned Robometer 13/20, sparse 11/20.
Builds the RECAP value model from the Wan2.2 video generator, jointly predicting future proprioception and a return separating successes [0,1) from failures [1,2). Real, three tasks, two rounds: 80.0% average success vs 63.3% with a VLM value.
An 8B video-language model writes chain-of-thought at each timestep, then outputs progress in [-100, 100] as the only reward for off-policy RL from a random policy. 40 sim and real tasks: >=50% success on 24 vs 7 for GPT-5.
Qwen3-VL-4B reward model with progress, success and preference heads, trained on RBM-1M including failures. Real DSRL on frozen pi0: 20% → 85% in ≤45 min (10k steps) vs 55% with RoboReward; 0 false-positive successes vs 45.
Reads a frozen Qwen3-VL-8B's log-probability of 'True' that a video prefix completes the task as a training-free progress curve. Real ManiRewardBench progress subset (113 tasks, 497 episodes): VOC 0.942 vs GVL 0.316 on the same model, trained Robometer-4B 0.858.
Fine-tunes Qwen3-VL to score rollouts 1-5 on a rubric, adding counterfactual-relabeled and clipped negatives; in sim, reward accuracy tracks RL success (r=0.83). Real WidowX DSRL, open drawer, 20 trials: base 10%, RoboReward 8B 80%, human-reward oracle 90%.
A VLM reward model predicts signed progress 'hops' from multi-view images, fused into a potential used as policy-invariant shaping, not raw reward. 8 real tasks: 95.2% final success vs 68.0% with sparse reward (ConRFT) and 9.8% for BC.
How do I tell whether checkpoint k+1 really beats checkpoint k, and how do I improve a policy, when every real robot trial is slow, noisy and needs a reset?
Build a stand-in for the real robot, either a learned video or latent world model or a scanned real-to-sim twin, then use it to score checkpoints cheaply and to run the try-score-improve loop without spending robot time.
All 104 entries on GitHub → (93 from Sep 2025 on, 12 must-know) · filter the index
Use evaluation infrastructure first, almost always. If you will compare more than a handful of checkpoints, build a twin or learned evaluator and validate it on 5-10 checkpoints with paired real trials (report Pearson r and MMRV) before trusting it.
In the last 12 months the family went from 'world models as evaluators' to 'world models as RL environments for VLAs', and then to 'making those environments trustworthy'. From September to November 2025, World-Env, VLA-RFT and WMPO ran GRPO/PPO-style RL for VLAs inside learned world models, scored by a VLM judge, a learned success classifier or similarity to a reference rollout; the draw was on-policy RL without resets. By February-March 2026 the focus had shifted to hallucination and compounding error: WoVR (keyframe-initialized rollouts, model-policy co-evolution), VLAW and PlayWorld (grounding the model in real policy rollouts or failure-rich play data) and VLA-MBPO (short imagined branches) each target ways an imperfect world model misleads the policy. On real robots, RISE reports 30-35% → 85-95% (20 trials per task) and WoVR 51.1 → 80.0 on a Franka, and evaluators were validated at larger scale (Veo against 1600+ real trials; RoboWorld at r = 0.989 with the RoboArena leaderboard).
Adds a Dyna-like stream to real-robot SAC/RLPD: jittered actions are rendered by a post-trained Cosmos-Predict2.5 model, reward-labeled, and gated into the critic loss. Real robot, 5 tasks: 82% in 64 min vs HIL-SERL 56% in 83 min.
Converts Wan2.1-T2V-1.3B into a causal, KV-cached, 4-step autoregressive video model on DROID, trained with Step Forcing to limit drift, and scores rollouts with GPT-4o. Across 8 VLAs (4,186 rollouts): Pearson r = 0.989 with the real RoboArena leaderboard.
Fine-tunes Bagel on ~50 collected rollouts per task to predict multi-view frames 10 steps ahead plus task success, then runs PPO on short imagined branches from dataset states. Sim LIBERO average 76.8 → 85.9, vs 82.6 for equal-budget online RL.
Trains a video world model on 30 hours of autonomous robot play, then improves a diffusion policy inside it with DSRL. Real towel-folding success peaks at 75% from 10% (2000 steps); human-demo-model peak: 40% (chart reading).
Runs GRPO entirely inside an action-conditioned Wan2.2-5B world model, starting imagined episodes near failure keyframes and refreshing the model with policy-aligned data to curb hallucination. Real Franka: 51.1 → 80.0 average (WMPO 64.5), no online robot RL.
Grounds a world model on 50 real pi0.5 rollouts per task each iteration, labels synthetic rollouts with a strict Qwen3-VL-4B, and fine-tunes on real plus synthetic successes. 5 real DROID tasks: 0.460 → 0.868 vs filtered BC 0.752.
Replaces online physical rollouts with a learned action-conditioned video model and a pi0.5-based value model that scores imagined chunk outcomes, driving advantage-conditioned training of pi0.5. Real dual-arm, 20 trials per task: box closing 35% → 95% vs RECAP 60%.
Turns a 2-5 minute monocular video into a splat-rendered IsaacSim twin for policy evaluation, with brief co-training to align visuals. Average Pearson r = 0.9 across 6 unseen scenes; in their tests the Ctrl-World video model mis-ranked policies (MMRV 0.22).
Runs Gemini Robotics On-Device inside an action-conditioned, multi-view Veo 2 world model; humans score the videos. Eight nominal checkpoints: Pearson 0.88, MMRV 0.03 against real success, with under-predicted absolute rates (chart reading).
Trains flow VLAs with FA-GRPO and FlowScale inside a 2B Cosmos-initialized action-to-video world model, rewarded by a VLM majority vote. Real UR30e, 100 in-model RL updates, 3 runs x 20 trials: pi0.5-3B 52.1 → 82.1 (+30.0).
Runs critic-free GRPO for OpenVLA-OFT inside a pixel-space video world model fine-tuned on the policy's own rollouts, failures included, with a VideoMAE success classifier. MimicGen (sim), 4 tasks, 128-rollout budget: 33.6 → 47.1 vs online GRPO 33.2, offline DPO 37.3.
Fine-tunes a 1.5B multi-view video diffusion model on DROID so a VLA runs closed-loop inside it, then fine-tunes the policy on human-labeled successful imagined rollouts. pi0.5-DROID: 38.7% → 83.4% on novel instructions and objects, imagined rollouts only.
How do I get the trial-and-error gains of RL for a robot policy without paying for thousands of risky, hand-reset real-world episodes?
Uses a simulator, often a scanned or generated copy of the real scene, as a safe practice gym where the policy can hill-climb with RL before or alongside a short real-robot stage. Real demos or real replay data keep it tied to reality.
All 53 entries on GitHub → (42 from Sep 2025 on, 12 must-know) · filter the index
Use this family when real-world trial and error is slow, unsafe or needs human resets, and you can build a simulator that gets the coarse task structure right: rigid objects, known geometry, a calibratable controller. With a pretrained BC or VLA policy, a few real demos and a rough simulator, run sim RL with a real-data BC anchor (RL-Co, TacCoRL, DLS); for recurring deployment failures, rebuild the failing scenes in simulation (F4R); for a small actor-critic you can run on hardware for minutes, pretrain in sim and continue online. Avoid it for deformables, fluids or contact physics your simulator cannot fake, or when building the twin costs more than collecting real data.
The simulator changed jobs: earlier sim-to-real trained small policies from scratch and hoped for zero-shot transfer, while now it is also a practice gym for already-pretrained BC and VLA policies, often with real demonstrations kept as a BC anchor during sim RL (RL-Co, Feb 2026; TacCoRL; DLS). Building the simulation got cheaper and more automatic, from phone-scanned twins (TwinRL) to generated 3D scenes (Generative 3D Worlds, Mar 2026), which makes it possible to rebuild only the scenes where the real policy failed (F4R, Sep 2026). Sim-real hybrids also matured: TwinRL runs RL in the twin to seed the real replay buffer, and What Matters for Sim-to-Online RL sets out how to continue sim-pretrained SAC on a real robot.
A VLM diagnoses real pi0.5 failures, the scene is rebuilt in Isaac Sim, and the policy is co-trained on failure-conditioned sim data, then PPO-refined and redeployed. Real OOD success: 26.25% → 90.0%, vs 71.25% for budget-matched Targeted BC.
Finds pi0/pi0.5 failures in parallel digital twins, turns hand-defined task phases into a dense potential, and applies a critic-free contrastive velocity loss plus a real-demo SFT anchor. pi0.5, 3 real tasks, ID/Unseen: 84.4/72.2 vs PPO 81.1/67.8, 100-demo SFT 78.9/54.4.
Trains PPO from scratch in sim on a capsule-chain cable for single-clip routing, then deploys through a 10 Hz twin whose sim joint angles become real targets. Real 3-clip routing, zero demos: 18/24 vs 12/24 copied from the hierarchical-IL paper.
Adds contact-gated tactile tokens to a pi0.5-style VLA, co-trains on real plus sim demos, then runs sim PPO with a real-data BC anchor. Real, 4 bimanual tasks after sim RL: 72.5% visuo-tactile vs 50.0% vision-only.
Builds a sim twin from one RGB-D human video, re-optimizes robot poses at object-motion keyframes, distills them into a point-cloud policy, adds a PPO finger residual. Real, 7 tasks, no robot demos: 67/70 (95.7%) vs 11/70 with FoundationPose poses.
Controlled study of zero-shot sim-to-real for a VLA trained only in RoboTwin 2.0, isolating which randomization factors matter and adding binary-success GRPO in sim. Real average success: SFT 5.6% → SFT+RL 33.4% → SFT+RL+DR 42.8%.
Generates 100 physical ManiSkill 3 scenes with a GPT-4o designer and EmbodiedGen, runs PPOFlow on a BridgeV2-pretrained pi0 across them, then transfers zero-shot. Real WidowX, 12 scenes, 240 trials: 21.7% → 75% success, compared only with the imitation starting policy.
Freezes a sim piano rollout, corrects lateral finger joints from MIDI errors, then trains bounded residual RL with MIDI rewards. Five real songs: mean best-validation key-press F1 77.6 versus open-loop simulation trajectory 41.8 (chart reading).
Empirical study of continuing sim-pretrained SAC on real robots: keep or warm-start replay, update the actor once per 20 critic steps at LR 1e-5, restore optimizer state. Franka vision grasping reaches near-perfect success in about ten real minutes.
Co-trains the VLA on real plus MimicGen sim demos, then runs PPO in a rough ManiSkill3 task copy with a weighted real-demo BC loss; no real-robot RL. Real average success, pi0.5: SFT co-training 45.9 → 66.2.
SFTs Octo-Small on 30 real demos plus twin trajectories, runs RL in a phone-scanned twin to seed the real buffer, then real HIL-SERL-style RL. 4 real Franka tasks: near-100% success, at least 30% less real interaction than HIL-SERL.
BC-trains a visuo-tactile diffusion policy on 30 real demos per task, fine-tunes it with DPPO and sparse success reward in an Isaac Gym twin simulating the same tactile pad, and redeploys. Real tabletop, 5 parts, visuo-tactile mean: 0.50 → 0.85.
How do robot companies say they climb from ~60-80% to 99% on a paying task, and what do they actually disclose?
Companies take a big pretrained robot model and push one real task from "works in a demo" to "works all shift" with deploy, grade, curate and retrain loops, sometimes with RL folded in.
All 28 entries on GitHub → (20 from Sep 2025 on, 7 must-know) · filter the index
Use these case studies as a template when you have one fixed, economically defined task, a pretrained base policy that already works sometimes (roughly 50-80%), and the ability to run many long evaluation sessions. A common order, pieced together from partial company disclosures: (1) define success with the customer's own quality bar and log everything; (2) automate grading and failure localization (progress model or auto-labeler); (3) collect targeted recovery or correction data where the policy actually fails (DAgger-style); (4) if success plateaus or speed matters, add a value function and advantage-weighted or advantage-conditioned retraining in the RECAP style; (5) report streaks, successes per hour and graded quality over hundreds of trials with a declared scope.
RL entered industry recipes in supervised-friendly forms: π*0.6/RECAP (Nov 2025) used a value function and advantage-conditioned retraining and outperformed PPO and AWR baselines on laundry folding with π0.6 (a Gemma 3 4B backbone plus an 860M action expert), and Generalist lists learning from experience (RL) among GEN-1's ingredients. Large-scale human data is now presented as what makes post-training cheap: GEN-1 is pretrained on over 500,000 hours of wearable-device data and adapted with about an hour of robot data per task, and Sunday pretrains ACT-2 on sensorized human data. Reporting moved toward production metrics, such as ACT-2's 99.1% success over 785 laundry attempts and Dyna-2's 95 napkins per hour with 93% meeting the quality bar, but these posts give few details of the training updates behind the numbers. A Siemens factory case study is a rare published account of the effort and recurring failure modes of an iterative fine-tuning loop on π0.5.
Previews RL post-training via self-play: in simulated robot soccer the policy's one stated objective is to score, playing recent versions of itself, then is transferred to a real humanoid. No metrics; 140 simulated years of play, real match on video.
Fleet workflow: an auto-labeler splits each deployed episode into SOP steps with outcome and failure mode, humans review uncertain cases, and engineers query the fleet to retrain. Napkin folding: 35/hour, 75% passing (Dyna-1) → 95/hour, 93% (Dyna-2).
Pretrains ACT-2 on sensorized human data from Sunday's own capture hardware, then post-trains on repeated in-house Memo runs and recoveries (update rule undisclosed). Laundry: 99.1% over 785 autonomous attempts in unseen homes, one fixed checkpoint, zero per-home adaptation.
Runs a pure-SFT loop on pi0.5 in a real Siemens factory: teleop demos, manual review, re-fine-tuning, and targeted recovery demos, relaxing constraints each round. 2,535 factory episodes; no final success rate given; authors say it fell short of expectations.
Pretrained on 500K+ hours of human wearable-device data, adapted per task with about 1 hour of robot data; RL is one of five listed ingredients, not detailed. Three charted tasks: 99% average vs 64% for fine-tuned GEN-0 (whole-system comparison).
Post-trains a VLA on 'only tens of hours' of office data to stay reliable in unseen venues; no objective or update rule is given. 30-minute shirt-folding trials (~40/hour): 22 consecutive failure-free folds in the office, 20 in unseen venues.
Improves one deployed parcel-sorting job with more in-domain demos, richer System-1 inputs (stereo, vision memory, state history, force) and a 50% larger decoder head; the post describes no RL. Over three months, barcode-orientation success rose ~70% → ~95%.
Which general RL results from the last year change how you should hill-climb a robot policy?
Not robot-manipulation papers: scaling laws for value functions and RL compute, critic and actor-critic design studies, flow/diffusion policy theory, RL for language models (group baselines, off-policy and stale-data training, process rewards, test-time compute), and last-mile RL for driving policies. Robot work keeps borrowing from all of them.
All 79 entries on GitHub → (59 from Sep 2025 on, 0 must-know) · filter the index
Read these when you are choosing an algorithm family or a compute budget, or when a robot paper cites an idea from language-model RL and you want the original.
RL for language models and RL for robot policies increasingly share a toolkit in 2025-26: critic-free group baselines, KL-anchored fine-tuning from a pretrained model, learned verifiers, and spending compute at test time. Real-robot work still leans heavily on off-policy critics (SAC or TD3 with RLPD-style demo mixing), which language-model RL rarely uses.
716 papers and posts from September 2025 to October 2026, one dot each, sorted into family lanes. Accented dots are the must-know works. Hover over a dot for its title and click to find it in the index; on a touch screen, tap once for the title and again to jump to it.
Narrative drawn from the linked papers and posts.
What each company says its loop is, the evidence it gives, and what it has not disclosed. Most industry numbers bundle new pretraining with the post-training loop, so none of them isolates how much hill climbing itself contributed.
Narrative drawn from the linked papers and posts.
pi*0.6 / RECAP. Deploy the VLA, have humans label every episode success or failure, and let teleoperators intervene (their actions are forced to 'positive'). Fit a distributional value on negative steps-to-success and compute n-step advantages (N=50). Binarize them into an 'Advantage: positive/negative' text token, retrain the whole flow VLA with its supervised loss (token dropped 30% of the time for CFG), and restart each round from the pretrained checkpoint. Also RL Token: a small TD3-style actor-critic on a compressed token from pi0.6 (task-fine-tuned on 1-10 h of demos, then frozen), penalized for straying from the VLA's own chunk.
T-shirt folding, same data, different policy-extraction methods: RECAP about 60/hr and 96% vs AWR about 30/hr and 91% vs PPO about 22/hr and 75% (chart readings; AWR and PPO start from pi0.6, not the offline-RL pi*0.6 checkpoint). Espresso: about 40% after offline RL + SFT → about 92% with RECAP (about 11% for pretrained pi0.6 with no task fine-tuning); the paper reports 13 hours straight of espresso making. RL Token: screw critical-phase success 20% → 65%.
Algorithm, ablation and hyperparameters are published (arXiv 2511.14759). No code. Numbers are mostly bar-chart readings. The text's iteration gains (laundry +50% throughput, box assembly 2x after the second iteration) do not name their reference point, and Fig. 9 shows box assembly first dropping and then improving, so our readings against the initial policy (box about 10 → 13/hr, laundry about 33 → 60/hr) differ from them; treat these readings as approximate. The method relies on human labeling, interventions and resets, which the authors list as a limitation: the system is not fully autonomous.
A learned video progress/reward model whose score dips when the robot makes a mistake acts as evaluator, failure finder and data filter. Flagged failure states direct targeted recovery-data collection, then retraining (human-in-the-loop active learning). At customer sites, 'deployment is the eval': fleet-wide logging (>1 TB/day), an auto-labeler that grades each SOP step against the customer's quality bar, human review of new tasks, ambiguous or low-confidence cases, suspected regressions and a sampled slice that checks label quality, and retraining on targeted slices. A separate hardware-health monitor keeps gripper wear from being mistaken for policy regression.
DYNA-1: 850+ napkins in one 24h run with zero interventions at 99.4% success; Dyna reports 98% of folds at grade 3 or above and 75% at its production grade (4-5 of 5). Talk (Jason Ma, posted 2026-09-24): standard post-training stalls near 80%, and the reward-model loop reached 99.4% over 24 hours. Din Tai Fung: 35/hr at 75% quality (Dyna-1) → 95/hr at 93% (Dyna-2). DYNA-1i: about 40 shirts/hour, 20 consecutive failure-free folds per 30-minute window in unseen venues.
No update rule, no reward-model accuracy, no RL objective and no ablation isolating the loop. Dyna credits Dyna-2 gains to pretraining, new capabilities, tooling and systems together. The talk's trial protocol (how many 24h runs stand behind 99.4%) is not stated in its public description.
GEN-1: trained from scratch on 500k+ hours of human wearable interaction data with no robot data in pretraining, then about 1 hour of robot data per task. Five named ingredients: pretraining advances, post-training techniques, learning from experience (RL), multimodal human guidance and inference-time techniques.
99% average on three charted tasks vs 64% for fine-tuned GEN-0 and 19% for training from scratch without pretraining. Box folding in 12.1 s vs about 34 s for GEN-0 and pi0 (folding time only; baseline times taken from their earlier published sources).
Blog only; GEN-1 is offered to early-access partners. No algorithm details and no ablation of GEN-1's own components (the 19% baseline is a from-scratch GEN-0 with no pretraining). No trial counts behind the averages (only consecutive-success streaks, such as over 200 box folds in a row). The 64 → 99 comparison bundles new pretraining, data, compute and algorithms, so the post does not show how much of it comes from RL.
ACT-2: pretrain on sensorized human data from Sunday's proprietary capture hardware, then hill-climb reliability in-house on Memo robots. Run repeatedly, surface failures, post-train so the model learns from recoveries. Proposes the 'Solve' reporting standard: performance, declared scope, adaptation cost.
Laundry: 99.1% ± 0.3% SE over 785 attempts in unseen homes with one fixed checkpoint and no per-home adaptation. Generalization gap after identical post-training: 82 points with 0% pretraining, 0 with 100% (50 evals each).
The post-training update rule is undisclosed (deferred to an unpublished technical post). No ablation isolates how much of the 99.1% comes from the hill-climbing loop (the pretraining-scale study holds post-training fixed). Some garment categories have small samples (n=19, n=7).
Helix logistics: no RL described. More in-domain demos plus architecture changes aimed at observed failures (stereo, vision memory, state history, force input, a 50% larger decoder).
Over three months, barcode-orientation success went from about 70% to about 95% and handling time from about 5.0 s to 4.05 s. 10h → 60h of demos: 6.84 s / 88.2% → 4.31 s / 94.4%.
Blog only. Ablations appear in a figure with no trial counts or error bars. It serves as the demos-plus-architecture baseline that post-training loops must beat.
A pure-SFT loop on pi0.5 via OpenPI in a real factory: teleoperate under constraints, hand-review every episode, re-fine-tune the base on all data, evaluate, collect targeted recovery demos, and relax constraints as a curriculum (a 70% gate per round).
2,535 factory episodes (about 10 h) plus about 900 lab episodes. Final success 'did not meet expectations'. In unconstrained trials, 65% of failed episodes left bag contents on the product.
A 5-page deployment case study with named root causes (wrist-camera angle, transparent bags, occlusion, contact). No overall success rate is reported.
SOP: 10 robots run pi0.5, humans take over on failure, episodes stream to a cloud learner that publishes new weights every 25 steps (HG-DAgger or RECAP as the loss). LWD: a 16-robot fleet logs everything with sparse success labels, and a learner runs offline-to-online RL (distributional implicit value learning plus adjoint-matching policy extraction) over offline plus fleet data.
SOP+HG-DAgger: Grocery 0.61 → 0.94, Laundry 0.58 → 0.96, Box Assembly 0.70 → 0.98. LWD over 8 real tasks with about 60 fleet robot-hours, averaging binary success on grocery tasks and partial-credit sub-step scores on long-horizon tasks: SFT 0.76 → 0.95 (RECAP 0.85, HG-DAgger 0.85).
arXiv papers with methods. No public code for SOP was found (the project page listed in the paper returned 404 when checked in October 2026). LWD reports its 652.5-hour offline buffer and a roughly 60 robot-hour online budget, but not intervention counts or learner compute. SOP's Fig. 5 and text disagree on one number (0.895 vs 0.800).
Physical Self-Play: competitive self-play RL in NVIDIA Isaac Sim (robot soccer; the post gives 'a single objective: score', and opponents are recent versions of the same policy), transferred to a real humanoid. Pitched as post-training to get past the ceiling of human-data pretraining.
140 simulated years of play, then a real humanoid match shown on video. No metrics.
No quantitative results, baselines or algorithm. On rewards, the post gives only the stated objective and says behaviors such as dribbling, shielding and tackling were not hand-rewarded. The post says 'a strong base model like S1' can learn this way but does not state how the soccer policy was initialized; it describes the policy as barely able to walk in its first simulated months.
A learned action-conditioned video world model with a success-value head, used as a repeatable evaluator to rank checkpoints and architectures before real trials. The stated motivation: fully evaluating one humanoid checkpoint in real homes takes days to weeks and hundreds to thousands of trials, while the world model can replay identical starting conditions for every policy.
A technical report states that a world model with 70% alignment picks the better of two checkpoints 15 points apart 90% of the time, and reports alignment rising from 63.06% to 71.17% with cross-task data. The 2024 challenge reports only token-level baselines (GENIE_138M at 8.79 CE loss on its GitHub leaderboard).
A blog plus a 12-page technical report. The report plots world-model scores next to real Arcade task scores for a few checkpoint sweeps and architecture pairs, but gives no trial counts or error bars for the real-world points and no correlation coefficient, and it does not show how the 70% / 15-point / 90% figure was derived.
Perturb-and-score search and reward-weighted averaging did not disappear. Most of it now runs as an inner loop inside newer systems.
Narrative drawn from the linked papers and posts.
Best-of-N with a verifier is one step of random search. You sample N candidates from the policy, score them and keep the argmax. This is the core of IDQL, V-GPS, UF-OPS, SeeQ, CoVer, RoboMonkey and TACO. EXPO-FT and Real-Time EXPO-FT use the same argmax over base and edited proposals, both for acting and inside the TD target (EMaQ-style). QT-Opt picks actions with CEM over a critic, and VLA-ATTC replaces argmax with a knockout tournament.
MPPI/PI2 exponential weighting shows up as 'softmax-Q-weighted averages'. Q-Planning executes a softmax-Q-weighted average of 32-64 proposal chunks, which is one MPPI step. SAILOR does MPPI residual search in a learned world model, and Simulator-in-the-loop cloth refinement runs MPPI in a GPU cloth simulator around the base plan. The Zero-order optimization primer shows MPPI is a natural-gradient step with step size 1/lambda, and that choosing exponential, rank or top-K weights gives MPPI, CMA-ES-like or CEM-like updates.
The KL-regularized closed form pi_ref * exp(A/beta) is reward-weighted regression, and PI2's weights are the same softmax. AWR, CRR, ARFM, IG-RFT and the soft exponential weights of Dissecting advantage-guided post-training use it directly. Filtered BC and rejection sampling (Hi-ORS, SOAR, RoboCat) are RWR with 0/1 weights. RECAP and CFGRL reach pi_ref * p(I|A)^w through an advantage token plus classifier-free guidance. QAM trains a flow toward pi_beta * exp(tau*Q).
ES-style fitness shaping appears in critic-free policy gradients. GRPO's group z-score, (R - mean)/std, is the same normalization ES at Scale applies to population rewards. ARS divides by return std, and OpenAI-ES and TD-ES use centered ranks. The difference is that GRPO perturbs actions per token or step and needs log-probs, while ES perturbs weights once per episode.
Noise-space search is black-box search with the generative policy as the decoder. Golden Ticket runs random search or CEM over one noise vector. Diffusion-ES mutates by re-noising and denoising. DSRL, SCORE, PSS and UniSteer replace search with an RL actor over the same noise space. PSS keeps only the top-k sensitive directions found by finite differences, and INSPO and SDN pick noise without training.
Black-box search at the outer level. ENPIRE runs evolutionary keep-what-works search over whole training recipes on real robots, similar in spirit to Eureka's evolutionary search over reward code, which it cites. Refinery uses a GP plus Bayesian optimization to pick PPO start states. VLA Grounder is compared against the evolutionary prompt optimizer GEPA. CMA-ES-IG learns user preferences from rankings. HandelBot's per-finger refinement is a non-learned, error-driven iterative correction applied before residual RL.
Black-box search as a second stage and for small knobs. EvolSAC runs SNES on the actor weights with the true competition score as fitness after SAC is trained on a shaped reward, and TD-ES continues from a PPO checkpoint with bounded-noise ES once policy-gradient gains become unreliable. BO remains common for small real-world tuning problems: of 110 hardware papers reviewed, 57 use vanilla BO, and most hardware examples tune fewer than 10 parameters (Decade of BO for control).
Shooting and tree search with world models and critics. GPC, IRASim, Latent Policy Steering and DreamSteer rank sampled chunks by predicted outcomes (random shooting). VLA-Reasoner runs MCTS with a frozen VLA as the prior. SVA runs MCTS offline in simulation and distills the returns into a critic that picks among frozen-VLA samples at deployment. tau0-VLA beam-searches over candidate subtasks, scored with a world model and a value model.
Patterns that show up across families, with the evidence behind each.
Narrative drawn from the linked papers and posts.
SimpleVLA-RL: an OpenVLA-OFT base with no task demos scores 0% on five RoboTwin 2.0 tasks and stays at 0% after RL. PDE: on a LIBERO-PRO task where action-noise PPO starts at 0%, action noise, Robometer and RND all stay at 0 for 120 steps; PDE reaches 81.8%; the authors suggest successful exploratory rollouts on related tasks teach transferable skills. From a 1-demo BC base at 33.5% on LIBERO-Long (zero-shot 0%), noise steering (DSRL) stays at 33.5, while MiDAS, which trains a residual actor with value-based RL and reaches actions outside the base's effective support, gets to 91.2. The Golden Ticket paper notes its noise search works only if the frozen policy is steerable. Exceptions exist: ADEPT's RL learned a plate flip-and-regrasp the pretrained policy never succeeded at, and SimpleVLA-RL reports an emergent 'pushcut' strategy.
ADEPT (sim, Kuka-Allegro peg task): standard PPO at LR 1e-3 gets 0/5 seeds to ADR 50 (all collapse at ADR 20), while the low-LR recipe gets 5/5. PAC-ACT (sim, sparse-reward ablation): without its KL penalty between successive PPO updates, the policy drifts from the pretrained ACT's trajectories and the final training window drops to 72.7% (22.0% cumulative), vs 100% with it. Simple Recipe Works (continual VLA RL, LIBERO-Spatial): sequential GRPO with LoRA gives AVG 81.2 vs 7.3 with full fine-tuning. FlowDAgger keeps held-out skills at 0.88 (frozen base 0.96) vs 0.02 for SFT. TacCoRL (real Assembly #2): with no real-data anchor (beta=0) it reaches 45%, below co-training alone (55%); beta=0.1 or 1.0 reaches 80%. Counterpoint: SimpleVLA-RL updates all OpenVLA-OFT parameters with GRPO and no KL term and lifts the LIBERO average from 91.0 to 99.1.
SARM2: same residual RL from the same checkpoint (4/12 on Shorts Crumpled), and the reward source alone decides whether the loop climbs (SARM2 8/12), plateaus (Robometer-FT 5/12) or regresses (sparse 2/12). ViVa: 80.0% average real success with its value vs 63.3% with a VLM value in the same RECAP pipeline. Dissecting advantage-guided post-training: continuous advantage weights 0.74 vs hard filtering 0.50 mean success over 4 real bimanual tasks. RoboReward: in simulated Robomimic, offline reward accuracy correlates with RL success at r=0.83.
ResFiT (sim BoxCleanup): 200k vs 40M steps against a PPO residual (about 200x), the only controlled comparison here. EXPO-FT: 30/30 on every task after an average of 19.1 min of online data, starting from a pi0.5 SFT policy. AutoSERL: 50/50 in 8-45 min from one demo (without start-position randomization). For scale only (different tasks and goals, not a controlled comparison): RL-100, which applies a PPO-style objective to iterated offline and then online rollouts, logged about 14 h of data collection per task (about 1.9 h demos, 6.8 h offline-RL rollouts, 5.3 h online), and FPO++ locomotion trains on roughly 147M-197M sim steps (4096 envs x 24 steps x 1500-2000 updates).
SOP: 3 h of on-policy interaction took a model from 0.571 to 0.800, while 80 extra hours of offline demos took it from 0.576 to 0.612. RL4VLA (sim): SFT plateaued at 0.781 in-distribution and PPO reached 0.938. PAC-ACT: doubling demos from 800 to 1600 moved BC only from 60% to 66% on Contour, where RL reached 100%. VT-Refine (visuo-tactile, evaluated in sim): going from 30 to 50 demos moved BC only from 0.48 to 0.57, while sim RL from the 30-demo policy reached 0.94; on the real robot, RL raised the visuo-tactile policy from 0.50 to 0.85.
EvolSAC: SAC on a surrogate reward, then SNES directly on the target score. On the cartpole task, swing-up time fell from 2.085 s to 1.135 s, where 1000 more SAC episodes reached 1.565 s; the same recipe was then applied to the AI Olympics competition score. Q-Planning: on real stack-cups, critic-only rounds on a frozen BC policy go from 40% to 90%, where SFT on successful rollouts stalls at 55%. Related, but without a metric switch: TD-ES refines PPO checkpoints with bounded ES on the same return, lifting IQM success from 67.2% to 85.0% (Gaussian ES: 73.7%) at a fixed total budget.
Robo-Dopamine: without its policy-invariant shaping, the raw progress reward drops a 3-task real average from 85.0% to 41.3%. FlowPRO: plain DPO on correction pairs collapses PACK success to 13% from an 80% SFT base. VLA-MBPO: full-horizon world-model rollouts compound model error and fall below SFT on LIBERO-Long (52.8 vs 54.6), while 2-chunk branched rollouts reach 66.8. PlayWorld: a policy fine-tuned inside a world model trained only on human demos peaked at 45% and then fell to 25% on carrot removal, below its 30% base (chart reading, 20 trials per checkpoint). GPT-5 as reward: in the SOLE-R1 authors' review of failed rollouts, 42% of failures come from hallucinated success, vs 9% for SOLE-R1.
TimelyDAgger (sim OpenDrawer) at a matched expert budget: 42% with immediate takeover vs 78% with takeover at step 80. UniIntervene vs HIL-SERL on 5 real tasks: intervention rate 14.6% vs 34.3%, success 88% vs 81%. PAKT reports 62%-86% fewer cumulative interventions than HIL-SERL. Skill-Space Shooting's machine corrector scored above duration-matched extra human demos on all 4 tasks: clearly on Coffee (71 vs 46 progress) and Sweeping (84 vs 63), narrowly on Stack-3 (0.83 vs 0.80) and Drawer (0.60 vs 0.50), one run each.
RECAP's ablation (laundry, T-shirts and shorts): AWR keeps about 91% success (RECAP about 96%) but about 30/hr vs about 60/hr. BMD (Avoid, 24 modes): residual RL keeps 0.98 success but solves only 1 of 24 modes at 80% or better, vs 20 for the pretrained policy. Uncertainty-gated exploration noise (one small-compute LIBERO-10 study with a 450M SmolVLA): PPO with fixed noise collapsed individual tasks in 2 of 3 seeds and learned noise in every seed, and no arm beat the BC baseline of 0.648. Foresight: residual RL lifts oracle-initialized subtasks to 98.4/92.2/99.2, but the chained full task stays at 54.5% (85.6% with foresight rewards). HELP: round-1 training on recovery data raised Test Tube success from 35% to 55% but slowed execution from 93 s to 105 s, as the policy made extra recovery moves (round 2 reached 95% at 93.4 s).
EXPO-FT: 30/30 is still consistent with a true success rate of roughly 88-90% at 95% confidence. Golden Ticket: the SimplerEnv 63.0 → 89.5 figure takes the best of 9 tickets by their held-out success (base numbers from NVIDIA's public evaluation), while picking by search reward gives +14.8 points. DEED's round-1 gain (32% → 42%, 50 trials each) has overlapping 95% CIs, as the authors note, and round 2 fell to 22%. R2S-Eval notes that real-world success-rate rankings can change across repeated evaluations, and that policies with similar success rates can differ substantially in execution quality.
Narrative drawn from the linked papers and posts.
From textbook ideas to classic methods to this year’s papers.
TEXTBOOK RL IDEAS → classic methods → 2025-26 methods
|
+-- Perturb, score, reweight (finite differences, ES, CEM, CMA-ES, PI2/MPPI, BO, QD)
| +-- classic: Aibo gait 2004, PoWER 2008, PI2 2010, IT&E 2014, OpenAI-ES 2017, ARS 2018
| +-- 2025-26: ES at Scale, EGGROLL, TD-ES, Golden Ticket, ENPIRE, Decade of BO for control
| `-- hidden inside: best-of-N verifiers, CEM/MPPI planners, noise-space search
|
+-- Likelihood-ratio policy gradient (REINFORCE → TRPO/GAE → PPO; group baselines)
| +-- classic: DAPG 2017, DDPO 2023, DPPO and FLaRe 2024
| `-- 2025-26: SimpleVLA-RL, piRL, RL-100, RLinf-VLA, FPO++, Temporal GRPO, PDE, ADEPT
|
+-- Bellman backups + off-policy replay (Q-learning, SAC, CQL/IQL, RLPD)
| +-- classic: DDPGfD 2017, QT-Opt 2018, AWAC/CQL 2020, IQL 2021, RLPD/Cal-QL 2023
| +-- classic: SERL and HIL-SERL 2024
| +-- 2025-26: Q-chunking, QAM, EXPO-FT, RL Token, LWD, ALOE, RAPolicy, AutoSERL
| `-- frozen-base add-ons (residual / edit / latent-noise steering)
| +-- classic: Residual RL 2018, PLAS 2020, ResiP and Policy Decorator 2024
| `-- 2025-26: DSRL, ResFiT, PLD, RFS, DICE-RL, SCORE, MiDAS, Real-Time EXPO-FT
|
+-- KL-regularized improvement as supervised learning (RWR/AWR, filtered BC, CFG)
| +-- classic: RWR 2007, AWR 2019, CRR 2020, RoboCat 2023, SOAR 2024
| `-- 2025-26: CFGRL, pi*0.6/RECAP, STEAM, CLIFT, DEED, Dissecting advantage-guided post-training; curation CUPID, RoboDrop
|
+-- Imitation with on-policy labels (DAgger, HG-DAgger, IWR, RLIF, PVP)
| `-- 2025-26: RaC, SOP, FlowPRO, FlowDAgger, HELP, TimelyDAgger, UniIntervene, PAKT
|
+-- Greedy selection at test time (argmax Q over samples, guidance, MPC, MCTS)
| +-- classic: IDQL 2023, V-GPS 2024, ITPS 2024
| `-- 2025-26: RoboMonkey, CoVer, UF-OPS, QGF, FASTER, Q-Planning, SeeQ, CoRe
|
+-- Reward shaping and learned judges (potential shaping, preferences, classifiers)
| +-- classic: Ng 1999, human preferences 2017, VICE-RAQ 2019, VIP 2022, GVL 2024
| `-- 2025-26: VLAC, Robo-Dopamine, RoboReward, Robometer, SOLE-R1, SARM2, SVM, RynnValue
|
+-- Model-based RL / Dyna and evaluation (Dreamer, MBPO, real-to-sim twins)
| +-- classic: Dyna 1991, DayDreamer 2022, UniSim 2023, SIMPLER 2024
| `-- 2025-26: WMPO, Ctrl-World, Veo eval, PolaRiS, WoVR, RISE, VLAW, RoboWorld
|
+-- Sim-to-real (domain randomization, digital twins, BC-anchored sim RL)
| +-- classic: DR + PPO 2017, RialTo 2024, SGFT 2025-02
| `-- 2025-26: VT-Refine, RL-Co, TwinRL, What Matters for Sim-to-Online RL, generative worlds, TacCoRL, F4R
|
`-- Industry loops (deploy → grade → curate → retrain, some RL)
+-- classic: DAgger 2011, self-play (AlphaGo Zero), 1X World Model 2024
`-- 2025-26: DYNA-1 to Dyna-2 flywheel, pi*0.6, GEN-1, Sunday ACT-2, SOP/LWD, SkildThis guide is the map for lastmile, an open-source course in the same repo: one robot, one task, a starting policy that works about half the time (a behavior-cloned flow policy; Chapters 1 and 2 tune a mis-tuned scripted controller instead), and a chapter plan spanning twelve approaches to improvement. Implemented chapters report success and costs in robot-minutes and human-minutes; several approaches remain planned.
A low-cost SO-101 arm (the AgileX PiPER is supported in simulation too) picks up a cube and drops it in a cup. Everything runs in MuJoCo on a laptop without an NVIDIA GPU. Each chapter also describes a real-arm step for the SO-101; none has been run on hardware yet, and each carries a safety note to read first. Published full runs report a Wilson interval on fixed held-out start states, robot-minutes (search, training and evaluation kept separate) and human-minutes. Quick runs use search states for their diagnostic scores.
Benchmark limitation: CupDrop-v1 can count a successful drop after the cup was moved or tipped and recovered. These scores do not establish a no-shoving benchmark; the measurements are in the design contract (§2).
9 of 13 chapters are available. The result column is the headline each chapter leads with, on the held-out set with its 95% Wilson interval: pooled over seeds, or a single run in Chapter 0 (one base) and Chapter 3 (whose method is to pick one ticket on the search set), and never a seed picked by its held-out score. Every seed is scored on the same 256 held-out start states, so a pooled interval, which treats the seeds as independent, is narrower than the evidence supports. The chapter notes have every run and what went wrong.
| Ch | Method | Base idea it teaches | Papers it mirrors | Status | Headline result [95% CI] |
|---|---|---|---|---|---|
| 0 | The base policy | Behavior-clone a flow policy from mixed-quality demos | Rectified flow, ACT, Diffusion Policy | available | 50.4% [44.3, 56.5]base_v1 · 129/256 |
| 1 | Hill climbing & ARS | Finite differences on a smoothed objective | Kohl & Stone 2004, ARS, TD-ES | available | 99.7% [99.1, 99.9]ARS, pooled over 3 seeds · 766/768 · base 48.0% |
| 2 | CEM, CMA-ES, PI², BO | Reward-weighted averaging; BO: a surrogate model | Zero-order primer, BO review | available | 99.7% [99.1, 99.9]CMA-ES, pooled over 3 seeds · 766/768 · base 48.0% |
| 3 | The golden ticket | Black-box search in noise space | Golden Ticket | available | 85.5% [80.7, 89.3]peek-free pick (racing), chosen on the search set · 219/256 · base 50.4% |
| 4 | PPO/GRPO for a flow policy | Likelihood-ratio PG + anchors | SimpleVLA-RL, πRL, PAC-ACT | planned · v0.2 | — |
| 5 | SAC → RLPD → Q-chunking | Bellman backups on replay | Q-chunking, Three Regimes, IPE | available | 93.9% [92.0, 95.4]Q-chunking best-of-32, pooled over 3 seeds · 721/768 · base 50.4% |
| 6 | Residual RL | Frozen base + bounded add-on | ResFiT, DAWN, Res-HIL | available | 89.7% [87.4, 91.7]residual TD3, pooled over 3 seeds · 689/768 · base 50.4% |
| 7 | DSRL noise steering | RL in latent-noise space | DSRL, SCORE, PSS, RFS | available | 81.0% [78.1, 83.6]steering the top 8 PSS noise directions, pooled over 3 seeds · 622/768 · base 50.4% |
| 8 | Propose, edit, select | Edit policy + argmax-Q | EXPO-FT, Real-Time EXPO-FT | available | 93.8% [91.8, 95.3]EXPO edit (β = 0.1, chosen on search), pooled over 3 seeds · 720/768 · base 50.4% |
| 9 | Filtered BC, AWR, RECAP-lite | Improvement as supervised learning | π*0.6/RECAP, CFGRL | planned · v0.2 | — |
| 10 | DAgger, HG-DAgger, RaC | On-policy labels | RaC, SOP, FlowDAgger | planned · v0.2 | — |
| 11 | Rewards & progress models | Shaping and learned judges | Robometer, TOPReward | planned · v0.2 | — |
| 12 | Test-time best-of-N | Greedy selection with a verifier | UF-OPS, Q-Planning, SeeQ | available | 93.9% [92.0, 95.4]best-of-32 with a learned verifier, pooled over 3 seeds · 721/768 · base 50.4% |
212 resources from the list: code to start from, simulators, benchmarks, courses, surveys, talks and related Awesome lists. Open a group to see them.
Every paper and post in the list, newest first. Filter by family or period, or search titles, labs and descriptions. “Checked” and “corrected” record a second pass against the source; they do not mean independent experimental replication.
| Date | Paper or post | Lab | Family | Status |
|---|---|---|---|---|
| 2026-10-02 | Self-Repairing Recurrent Ensembles for Real-Time Recovery from Distribution ShiftAn ensemble of recurrent controllers on masked observation subsets is Kalman-fused, and at deployment each member is fine-tuned every step toward the consensus action; recovers close to original performance after sensor shifts in simulated continuous control. | Lemmel et al. | Test-time search | read once |
| 2026-10-02 | Probe the Harness: Setup Checks for Stale-Data RL Comparisons in Language ModelsShows four harness details (ratio reference, unseeded arm, replayed first batch, loss normalisers) reversed a stale-data RL comparison between SAN and truncated importance sampling; once checked, TIS matches SAN on verl. Releases the PTH checklist. | Pan | General RL | read once |
| 2026-10-02 | MobiAgent: Dual-Loop Recursive Policy Self-Improvement for Long-Horizon Mobile ManipulationOuter loop segments and verifies deployment rollouts, clusters them into atomic skills and fine-tunes flow-matching skill experts without annotations; autonomous data recycling lifts success 7.50% to 27.50% on RoboCasa and 32.5% to 57.5% on Astribot S1. | Liu et al. | Advantage-weighted BC | read once |
| 2026-10-02 | Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot ManipulationA Proposer-Verifier-Governor loop uses privileged simulation evidence to learn and validate executable skills behind a shared robot API, transferred to real without updates; LIBERO-Pro Long success 2.0% to 56.3%, 78.75% mean completion on four real tasks. | He et al. | Sim-to-real | read once |
| 2026-10-02 | UniIntervene++: An Adaptive Intervention Agent for Efficient Real-World Reinforcement LearningLearns online values for the RL policy, trajectory correction and a task-structured CodePolicy as semi-MDP options to allocate control, periodically probing unassisted competence; five real tasks: 89.67% average success, at least 6 points over baselines, with 0.77% human intervention. · code | Lin et al. | Human corrections | read once |
| 2026-10-02 | RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real TransferTreats sim-to-real as continued adaptation of executable task skills: interaction feedback proposes skill revisions that are verified before being kept, while a pretrained VLA serves as a frozen action tool with inference-time guidance; LIBERO-PRO and matching physical tasks. | Li et al. | Sim-to-real | read once |
| 2026-10-01 | eRLT: Efficient VLA Reinforcement Learning via Action-Relevant Token RoutingOnline RL on a frozen VLA; a learned router pools action-relevant tokens across layers into an RL token, refined by critic feedback; real USB and ribbon-cable insertion learning-curve AUC +108.9% and +46.7% over the strongest baseline. | Huang et al. | Off-policy critics | read once |
| 2026-10-01 | Continue, Abort, or Fall: Viability-Aware Policy Selection (VAPS) for Safe Humanoid AcrobaticsLearned predictors estimate short-horizon viability of a tracking policy and an abort policy, and each step keeps the most ambitious viable behavior, else a fall policy; Pareto-dominates single-network alternatives in success and head impact; side flips on LimX Oli. | Ju et al. | Test-time search | read once |
| 2026-10-01 | Closed-Loop Refinement and Execution for Learned Driving PlannersRefines a frozen planner's trajectory with multi-start finite-horizon optimal control, keeps candidates passing a predicted-clearance check, and executes the cheapest; VAD on 126 Bench2Drive routes: driving score 43.41 to 56.42, collisions 70 to 53. | Hu et al. | Test-time search | read once |
| 2026-10-01 | Completion Aware Guidance for World Action ModelsCAG, a training-free sampling guidance for world action models, steers generation toward task-completing transitions; success 64% to 70% on a RoboTwin 2.0 subset, with task-incomplete imagination cut from 79% to 40%. | Kim et al. | Test-time search | read once |
| 2026-10-01 | Towards Optimal Policy ImprovementFrames greedification under approximate evaluation as decision-making under uncertainty and derives an operator optimal for that objective; it and its gradient-based approximations improve aggregate performance across GumbelAlphaZero, SAC, ReBRAC and Generalized Policy Iteration. | Oren et al. | General RL | read once |
| 2026-10-01 | SocialVLA: A Social Perception Gateway for Human-Reaction-Based Failure Detection and Recovery in VLA ManipulationRuntime monitor that fuses bystanders' vocal, facial and verbal reactions to halt a VLA and take spoken corrections; on Unitree G1 with an unseen participant, the frozen system reaches 59.5% recall and 91.7% precision. | Konstantinova et al. | Test-time search | read once |
| 2026-10-01 | Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement LearningMFPG-PPO uses cheap simulator rollouts only as a control variate for scarce target-domain policy gradients; matches PPO with 16x more high-fidelity data on the hardest task at the smallest budgets; learns on a real Franka with 4 episodes per update. | Liu et al. | Sim-to-real | read once |
| 2026-10-01 | Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied AgentsBuilds simulated practice tasks from an offline dataset, diagnoses failures with privileged state, and revises skills and the system prompt without weight updates; 22 tasks: 28.6% after the first round to 95.0% after 15, vs ASPIRE 75.5%; 30/30 real trials. · code | Wang et al. | Sim-to-real | read once |
| 2026-10-01 | Recova: Agent-Guided Failure Recovery for Autonomous Robotic ManipulationAn agent diagnoses failures and tests corrective programs in a reconstructed twin, then routes real rollouts and human demonstrations into DAgger training of task and recovery policies; four real workstations: mean success 23.8% to 77.5%, 87.5% with recovery skills. · code | Liu et al. | Human corrections | read once |
| 2026-10-01 | InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-ManipulationAn LLM agent revises staged reward programs and an optimizer tunes their constants, each verified in parallel simulation, to steer a frozen object-aware forward-backward behavioral foundation model to new loco-manipulation tasks without retraining; evolved skills run on a Unitree G1. | Lin et al. | Black-box search | read once |
| 2026-10-01 | Filter-Aware Fine-Tuning for Safe Humanoid Whole-Body TrackingCoFiT fine-tunes pretrained humanoid trackers with the CBF safety filter in the loop, closing dynamics, objective and information mismatches; cuts violation time 91% (TWIST2) and 21% (SONIC) vs filter-only training, and 83% for TWIST2 on Unitree G1 hardware. | Mohnot et al. | Sim-to-real | read once |
| 2026-10-01 | CriticHack: Evaluating Visual Rewards Under Robot Policy OptimizationFine-tuning a diffusion policy on the Robometer reward raises drawer-task success 10.2 points but wrong-object failures 10.9 points, unlike the simulator's success signal; a tilt model predicts shifts across 26 settings (Spearman .89), and a frozen outcome verifier fixes it. | Luo et al. | Rewards & progress | read once |
| 2026-09-30 | SimEX: Simulation-Integrated Robotics AutoResearchA coding agent builds a robot toolbox through probe-and-optimize iterations in simulation, then adapts both from a few physical trials that correct the simulator; real towel folding, barcode scanning and plate tasks with 10 minutes of robot interaction. · code | Hu et al. | Sim-to-real | read once |
| 2026-09-30 | Scale and Selection: What Makes Automatic Harness Evolution Work for Visual-Interface Robot AgentsAn optimizer coding agent revises a robot agent's harness from rollout outcomes; more rollouts per round and Champion-Challenger selection make it work, raising held-out success from 51% to 67% over 30 rounds, while unconditional acceptance drifts downward. | Wei et al. | Black-box search | read once |
| 2026-09-30 | Trust the Critic MoreAC2 assigns credit to 10k-token action chunks scored by a learned critic, used only where it is locally accurate, so trajectories need not reach a terminal reward; Qwen3-4B exceeds GRPO's 18.5% peak on IMO-ProofBench with 2.5x fewer decoding FLOPs. | Wen et al. | General RL | read once |
| 2026-09-30 | Cue the Flow: Steering Flow-Matching Policies for Open-World Delivery ManipulationA lightweight cue-conditioned adapter predicts a diagonal affine transform over a pretrained flow-matching VLA's action chunk, steering it toward the grounded target; up to near 2x average task success on tabletop and mobile-base delivery tasks. | Wang et al. | Residual & steering | read once |
| 2026-09-30 | RoboCoach: World Models as Active Coaches for Compositional Robot SkillsRuns skill experts inside an action-conditioned world model, a progress judge finds the first failing subtask, and those failures choose which demonstrations to collect and adapters to update; 150 extra subtask demos: Franka 13.3% to 75.0%, AgileX 40.0% to 83.8%. · code | Liu et al. | World models & eval | read once |
| 2026-09-30 | ChunkTrust: Adapting Execution Horizons for Robot Policies with Action-Expert EvidenceTraining-free selector infers how many actions of a chunk to execute from generation-trace stability and continuity, with an online Beta posterior, around a frozen policy; +6.80 points for pi0.5 over 50 RoboTwin2.0 tasks, real process score 50.4% to 57.5%. · code | Huang et al. | Test-time search | read once |
| 2026-09-30 | PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement LearningOutcome-only RL post-training of VLAs with chunk-level credit: a physical relational graph pools outcomes of rollouts reaching corresponding situations into success potentials, and confidence-gated changes refine trajectory advantages; higher success and faster learning on LIBERO, RoboTwin 2.0 and real robots. | Zou et al. | On-policy PG | read once |
| 2026-09-30 | LocoWM: High-Precision Locomotion through World-Model-Guided Residual AdaptationA residual adapter adds corrections to a frozen locomotion policy, conditioned on a frozen action-conditioned world model's predicted future physical states; better precision and disturbance robustness than end-to-end and reactive residual baselines on terrain leveling, acceleration compensation, push recovery. | Zhao et al. | Residual & steering | read once |
| 2026-09-30 | PrefPI: Preference-Guided Steering into Out-of-Distribution BehaviorsPreference-guided policy iteration: models preferred self-generated trajectories as a conditional distribution and amplifies them with classifier-free guidance, pushing diffusion policies and pi0.5 beyond their initial support; real object transport height 10.7 cm to 19.8 cm with 150 preference-labeled trajectories. | Rho et al. | Advantage-weighted BC | read once |
| 2026-09-30 | Measuring Asset and Scene Reconstruction Effects in Real-to-Sim Robot EvaluationCompares an authored real-to-sim reconstruction (metric geometry, authored physics, scene splat) with a default open-source recipe for evaluating two policies on five bimanual tasks: sim-real correlation r = 0.90 vs 0.51, mean score error 6.97 vs 17.54 points. | Verma et al. | World models & eval | read once |
| 2026-09-30 | EmbodiRSI: Recursive Self-Improvement for Data-Efficient Robot AdaptationBuilds task-specific simulations from deployment scenes and iterates agent-assisted corrections from policy-reached states plus demonstrations targeted at current weaknesses; sim success 50.4% to 83.5% over two rounds; real 83.1% with ten real trajectories per subtask vs 75.0% from 200 demos. | Lang et al. | Sim-to-real | read once |
| 2026-09-30 | Looking Back to Move Forward: Temporal Verification for Generative Robot PoliciesTeV learns a temporal token summarizing recent observation-action history and contrastively trains an energy-based verifier that ranks flow-matching VLA action chunks as trajectory continuations and guides intermediate flow samples toward low energy; simulation and real-world tasks. | Wang et al. | Test-time search | read once |
| 2026-09-30 | DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot AgentsSlow-brain planner plus fast-brain grounding and monitoring around a frozen VLA; failure attribution over execution records drives revisions admitted by paired regression checks. LIBERO-Pro, 800 new initial states: 75.2% vs 17.5% for the frozen policy. · code | Deng et al. | Test-time search | read once |
| 2026-09-30 | When Reasoning Helps Action: Monitoring and Steering Chain-of-Thought in Vision-Language-Action PoliciesTRUST: an offline-trained token-level value model predicts reasoning correctness from partial chain-of-thought and steers generation in frozen VLAs; Alpamayo 1.5 reasoning correctness 75.9% to 90.0% and 30.4% fewer collisions, but little closed-loop change on DeepThinkVLA. | Karnik et al. | Test-time search | read once |
| 2026-09-30 | Online Evolution Strategy for Flow-Matching VLA Policies via Self-Supervised Trajectory Distribution OptimizationEvolution strategy in action-trajectory space: perturbs flow-matching VLA samples, scores executions, and transfers the evolution direction into weights through a self-supervised MSE objective, with failures as negative feedback; matches reinforcement fine-tuning without a value model, in simulation and real. | Yao et al. | Black-box search | read once |
| 2026-09-30 | Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action ModelsA fixed CBF safety module acts as an observe-only teacher whose counterfactual corrections, admitted by outcome, drive guarded LoRA updates of the VLA. VLA-Arena: task success +8.5 and +6.9 points over the base policies, +25.4 and +9.5 over runtime shielding. | Cui et al. | Human corrections | read once |
| 2026-09-29 | Simple Agentic Memory for Generalist Robot PoliciesSimpleARM, a training-free memory layer for frozen generalist policies, keeps compact typed task state with frozen perceptual tools and retrieves it when a subgoal depends on history; RoboMME 16 tasks: 67.17% vs 44.51% for the strongest non-oracle baseline. | Zhang et al. | Test-time search | read once |
| 2026-09-29 | Correcting WHERE, Preserving HOW: Compositional Generalization for Vision-Language-Action Models via Referential GuidanceTraining-free wrapper using object poses to guide the end-effector into demonstration-supported configurations of the instructed referent, where the frozen VLA resumes; success under compositional shifts improves by up to 56.8 points in simulation and 75.0 on a real robot. | Zhang et al. | Residual & steering | read once |
| 2026-09-29 | Cool the Sampler, Not the Learner: Sampling Temperature Moves the Staleness Cliff of Importance-Corrected GRPOSampling stale rollouts at temperature 0.8 while learner and importance weights stay at 1 stabilizes importance-corrected GRPO past its staleness cliff; Qwen2.5-Math-7B at refresh interval 144: cooled runs end at 92-93% vs 68-81% uncooled. | Pan | General RL | read once |
| 2026-09-29 | Diffusion Policy Improvement with Proposal-Conditioned Refinement FlowsPReFlow extracts policies by critic-based selection of behavior proposals plus a conditional refinement flow trained with closed-form adjoint matching under a KL-regularized objective; highest aggregate OGBench score after online fine-tuning among compared methods, 91% after 500K steps. | Ha et al. | General RL | read once |
| 2026-09-29 | Data-Efficient Adaptation of a Driving VLA to Class 8 TrucksAdapt-then-steer: fine-tunes Alpamayo 1.5's action stack on a few hundred truck scenarios, then a flow-time-conditioned residual (Flow Velocity Steering) corrects the frozen model's flow velocity, cutting full-horizon ADE and FDE a further 13.9% and 16.5%. · code | Das et al. | Residual & steering | read once |
| 2026-09-29 | FP2: Equipping Robotic Foundation Models with Force ControlAction-regulation split: a task-adapted robot foundation model generates actions while a plug-in force-control policy predicts force-control parameters from its context plus wrench history. Real arm, four backbones, four contact-rich tasks: every backbone's average score improves, by up to 65 points. | FORTE Lab; Noematrix; Flexiv; SJTU et al. | Residual & steering | ✓ checked |
| 2026-09-29 | Track-and-Complete: Learning Humanoid Skills from a Single Failed Human VideoTRACC imitates the usable motion prefix of a failed human video, then trains with a VLM-proposed task-completion reward to reach the intended goal; evaluated in simulation (Unitree G1, Isaac Gym) on six Oops! failures. · code | Yonsei University | Sim-to-real | ✓ checked |
| 2026-09-29 | Counterfactual Video Generation Enables Scalable Humanoid Loco-ManipulationPRISM expands a few human videos into hundreds of counterfactual interactions, reconstructs physically plausible simulation trajectories, and trains depth-based humanoid policies for joystick-controlled picking, carrying and dropping, transferred zero-shot to real hardware. · code | Amazon FAR; UC Berkeley; CMU; Stanford | Sim-to-real | ✓ checked |
| 2026-09-29 | Encore: Few-Shot Agentic Discovery of Manipulation StrategiesCoding agent reads demo evidence packs (keyframes, gripper events, trajectories), refines a policy program without seeing the success signal, then freezes it; LIBERO-PRO 96.3% (3 demos) vs 89.3% for ASPIRE rerun with the same LLM; real bimanual tasks. · code | MIT; 2077AI Open Source Foundation; Texas A&M; UW–Madison | Black-box search | ✓ checked |
| 2026-09-29 | T²Mem: Learning Test-Time Memory for RoboticsTest-time training compresses observation history into fast weights inside a pretrained VLA; RoboMME 16-task success rises from 17.93% to 56.83%, with 3x+ faster inference than explicit memory. | Stanford; NVIDIA; University of Michigan | Test-time search | ✓ checked |
| 2026-09-29 | Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by TrustLucidWM applies Subjective Logic to categorical latent transitions, giving each a doubt degree and multiplying trust along imagined rollouts without extra parameters; trust-based action selection cuts navigation steps-to-goal from 362 to 190. · code | National University of Singapore; KAUST; A*STAR et al. | World models & eval | ✓ checked |
| 2026-09-29 | RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful RolloutsClosed-loop RL post-training of E2E driving policies inside video world models, keeping only rollouts an inverse-dynamics evaluator deems action-faithful; cuts safety violations 27.6% (DiffusionDrive, nuScenes). · code | CUHK-Shenzhen et al. | General RL | ✓ checked |
| 2026-09-29 | Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-PolicynessTheory of RE(S), REINFORCE with rollouts refreshed every S steps, in softmax bandits: global convergence, Θ(1/T) rate, and moderate off-policyness escaping weak-initialization traps faster. | Tsinghua; Alibaba | General RL | ✓ checked |
| 2026-09-29 | Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied AgentsRoboSkill: multimodal agent loop that explores, executes with vision/tactile feedback, and evolves a reusable code-skill library; LIBERO-10 first-episode success +12.5-25.0 points, real robot +8.3 points. · code | Fudan; Shanghai Innovation Institute; NeoteAI | Black-box search | ✓ checked |
| 2026-09-29 | Real2Gym: Building Gyms from Videos, Bringing Skills to RobotsAgentic Real2Sim2Real: reconstructs simulation gyms from videos; an agent writes stage-level manipulation code and distills successes and failures into reusable skills without weight updates; zero-shot real Franka 9/12 trials vs 5/12 for GPT-6 Astra Direct Mode. · code | Shanghai AI Lab; SJTU; Zhejiang; USTC; CUHK; HKU | Sim-to-real | ✓ checked |
| 2026-09-29 | ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right ContextContextProgress-Bench (24 manipulation tasks, 120 episodes) shows progress reward models get lost even with full history; given the right context their error drops 77-82%, and an agentic VLM loop supplying it cuts a frozen PRM's error 63%. · code | Northwestern; UCSB; UIUC; CMU | Rewards & progress | ✓ checked |
| 2026-09-29 | Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language NavigationSigned credit from action-induced observation changes trains an action reranker around a frozen navigation policy at test time; a separate memory verifies candidate reranker updates and rolls back unsupported changes. | Tianjin University; HKU; Hefei University of Technology | Test-time search | ✓ checked |
| 2026-09-29 | EasyPPO: Stabilizing the Critic Is KeyStabilizes PPO's critic for LLM RL via actor-only truncation filtering, noise-normalized critic regression and smaller critic minibatches; relative gains of 14.89% coding, 2.28% math, 9.47% search over PPO. · code | UC Berkeley; Princeton | General RL | ✓ checked |
| 2026-09-29 | RoboHarn-Evo: Evolving Hierarchical Physical Knowledge for Self-Improving Robotic ManipulationDual-loop harness evolves task- and action-level knowledge from rollouts with the VLM agent frozen; up to +24.2 points on RMBench; held-out success 70.0% to 88.3% (GPT-6) and 48.3% to 75.0% (GPT-5.5) after 80 rollouts. · code | Renmin University of China; XYZ Embodied AI; Tsinghua | Test-time search | ✓ checked |
| 2026-09-29 | TaRL: Learning General and Physical Rewards from Tactile DemonstrationsLearns rewards by regressing task progress from tactile deformation maps of successful and failed demos; lifts nut threading 34% to 56% (sim) and real cube pickup 37% to 97%. · code | National Taiwan University; Delta Electronics | Rewards & progress | ✓ checked |
| 2026-09-29 | Video2STL: Grounding VLM-Generated Temporal Specifications for Robot LearningA VLM turns observation-only videos into parametric Signal Temporal Logic specs, with thresholds grounded from successful robot trajectories, as dense and progress rewards for PPO; four ManiSkill3 sim tasks: 85.8%/67.0% success-once/at-end vs 81.5%/59.5% native dense reward. · code | USC; University of Florida | Rewards & progress | ✓ checked |
| 2026-09-29 | Spotter: Let the Embodied Model Lead, and the VLM Reflect for ItPolicy runs autonomously; a VLM monitors in parallel and intervenes only on detected errors. With GPT: Cosmos Policy +5.6 and pi0.5 +7.5 pts on RoboCasa; pi0.5 47.2% to 57.0% on RoboTwin 2.0 Hard, 53% to 83% real robot. · code | Griffith; Tsinghua; MainCode; Tencent; Fudan; Tongji | Test-time search | ✓ checked |
| 2026-09-29 | Rho: A Foundation for Efficiently Adaptable VLA ModelsOpen-weights dual-arm VLA family (YAM Box, UR AI Trainer, FR3 Duo) with embodiment-specific midtraining; a latent policy adapts online from as few as 15 corrected episodes. | Microsoft Research | Human corrections | ✓ checked |
| 2026-09-29 | WorldLine: Action-Driven Visual Simulation for Robotic ManipulationAction-driven visual simulator learned from 10,000+ h action-free robot video and grounded with 2,000+ h action trajectories; 74% trajectory-success prediction accuracy (strongest baseline 73%); best-of-32 selection with its rollouts beats direct execution by 19.1-21.4 points on RoboTwin, zero-shot. · code | HKUST; Joy Future Academy; Peking University; CUHK | World models & eval | ✓ checked |
| 2026-09-29 | Taming VLAs under Robot Execution Errors: Self-Compensation and Stress TestingDeployment-time adaptation updates a VLA online from the commanded-vs-executed motion residual, without rewards or labels; adds RoboStress sim benchmark (friction, backlash, compliance, gravity-comp error); >30-point real success gains on two Piper arms (one new, one used a year). | POSTECH; MBZUAI | Test-time search | ✓ checked |
| 2026-09-29 | EVO-WAM: Evolving World Action Models through Video-Action VerificationWorld action model adapts to unseen tasks from its own generated video-action rollouts (never executed), training on VLM-selected task-completing prefixes verified by inverse dynamics; Cosmos3 on seven unseen RoboTwin 2.0 tasks 26.9% to 68.0%, three real tasks 20.0% to 76.7%. · code | HIT Shenzhen; SLAI; Tsinghua; JD et al. | World models & eval | ✓ checked |
| 2026-09-29 | Direct Experience World-Model Optimization: Learning the World Beyond Action ImitationDEWO trains a deployed world-action model on successful and failed futures, using progress-gated classifier-free guidance. Across four real dexterous tasks, two rounds improve success 51.0%→71.7%, measured only in grid cells with initial successes. | HIT; Dalian University of Technology; SUSTech | World models & eval | ✓ corrected |
| 2026-09-29 | Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine TuningThree-stage RL fine-tuning for multi-robot VLAs: initialization-aware data collection, credit-filtered offline tuning, online latent-noise RL (DSRL) on the frozen VLA. Average success over the CHORUS SFT baseline: +23.1% RoboTwin, +16.4% RoboFactory, +44% on real dual-Franka tasks. | Beihang; CUHK; PKU-Psibot; Tsinghua et al. | Residual & steering | ✓ checked |
| 2026-09-29 | ★ BlenDAgger: Blended Shared Control for Interactive Imitation LearningKeeps the policy running during corrections, blending human and policy actions per DoF group and giving the human more control when they push hard, disagree, or the policy is uncertain. Final round, real Almond Scooping: 50% vs HG-DAgger 15%. | Carnegie Mellon University | Human corrections | ✓ checked |
| 2026-09-29 | ★ Skill-Space Shooting for Autonomous Robot Policy ImprovementA learned value model halts a pi0.5-class policy when its score drops below zero; Gemini 2.5 Pro picks and checks a repair skill; successful repairs become training data. Real Coffee mean progress: 71.25% vs 46.25% with extra human demos. | Tsinghua University; UC Berkeley; Shanghai Qi Zhi Institute | Human corrections | ✓ checked |
| 2026-09-29 | ★ PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human InterventionsPairs each human takeover window with a frozen pi0.5 chunk from the same state, trains a small Bradley-Terry scorer on the pairs, and uses it to guide sampling. Four real Franka insertion tasks: 90.5% vs frozen 69.0%, intervention BC 84.5%. | Westlake University | Test-time search | ✓ checked |
| 2026-09-29 | ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement LearningGuides the online critic with an IQL value reference learned from successful human data, plus a learned human-action 'fence': no imitation penalty inside it, constrained updates outside. Real plug insertion: 91.7% vs 5% or less for HG-DAgger, HIL-SERL and SiLRI. | Tsinghua University | Off-policy critics | ✓ checked |
| 2026-09-28 | The Low-Rank Structure of VLA Reinforcement LearningRL post-training of flow VLAs (pi0.5, GR00T N1.5/N1.6) yields low-rank updates concentrated in action-expert Timestep Modules; shift-update directions predict task success (ROC-AUC up to 99.6%) and steering along them improves RL-trained policies without more RL. | Oh et al. | On-policy PG | read once |
| 2026-09-28 | DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool LibraryAgentic pipeline with property-specific verifiers and a self-evolving tool library turns a single egocentric human video into sim-generated robot trajectories for training pi0.5; 63.6% average success over 11 real dexterous tasks, about 3.5x SPIDER's. · code | Stanford University; Columbia | Sim-to-real | ✓ checked |
| 2026-09-28 | NavHarness: Towards Lifelong Embodied NavigationTraining-free harness for lifelong navigation that carries maps, task records and house knowledge across sessions, checking them against new observations; on GOAT-Bench with GPT-6 Astra, +18.6 s-SR points over context-only independent sessions and a reported state-of-the-art 83.7 s-SR (SLAM poses). · code | Adelaide University (AIML); CSIRO Data61 | Test-time search | ✓ checked |
| 2026-09-28 | Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization ControlShows policy-gradient stabilization succeeds with zeroth-order (configuration) reward terms alone; adding velocity reward terms can introduce severe sensitivity as their scale grows, while velocity remains necessary in the policy observation (low-dissipation assumption). | UC San Diego, Tsinghua University | Rewards & progress | ✓ checked |
| 2026-09-28 | EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot PlanningAgent runs experiments to learn residual world models (physics engine plus code for missing forces/constraints/hidden state) with Bayesian inference; solves more tasks with fewer interactions than baselines. · code | Cambridge / MIT / Basis / Cornell / Princeton et al. | World models & eval | ✓ checked |
| 2026-09-28 | QAMM: Adjoint MeanFlow Matching for Few-Step Offline Reinforcement LearningTurns the critic's adjoint-matching signal into supervision for MeanFlow average velocity, yielding two-call flow policies competitive with flow-policy baselines on ten HumanoidMaze offline RL tasks. | Fudan; ShanghaiTech; Shanghai Innovation Institute | General RL | ✓ checked |
| 2026-09-28 | Don't Throw Away the Tail: Action Upcycling for Policy AccelerationTraining-free: keeps executing the normally discarded tail of each action chunk until action velocity starts to fluctuate; 1.2-1.7x fewer policy calls with no loss in success rate across three VLAs and one World Action Model, in sim and real. · code | Sungkyunkwan University; KAIST | Test-time search | ✓ checked |
| 2026-09-28 | Zero-Shot Reactive Obstacle Avoidance for Generative Robot PoliciesTraining-free NUDGE injects signed-distance-field gradients, via a differentiable joint-trajectory decoder, into the denoising loop of diffusion/flow policies and VLAs to steer action chunks around obstacles; real UR5 with a moving obstacle: 75.6% contact-free completion (62/82) vs 16.7% (3/18) unguided pi0. | Rice University | Test-time search | ✓ checked |
| 2026-09-28 | Learning from Teacher Continuations at Student StatesOLIVE: DAgger-like online LLM distillation; student generates prefix, teacher continues it, student trains by cross-entropy. +13% over offline SFT on ScienceWorld; async cuts training time 23.8%. | UIUC; MIT; UPenn; University of Washington; IBM | General RL | ✓ checked |
| 2026-09-28 | Behavioral Foundation Models for Quality DiversityBFM-QD runs quality-diversity search in a behavioral foundation model's compact latent space, giving a gradient-free, critic-free improvement operator that beats parameter-space QD on sparse/deceptive continuous control (NeurIPS 2026). | Sorbonne Université (ISIR) | Black-box search | ✓ checked |
| 2026-09-28 | Test-Time Adaptation of Manipulation Policies Under Actuator DegradationTeAR: a lightweight Transformer rectifies a frozen policy's actions from live actuator telemetry (temperature, current, voltage); on a real SO-101 arm under heating, Lift success 85% vs 75% and 70% for the frozen base (20 trials each), no on-robot fine-tuning. · code | Arizona State University | Test-time search | ✓ checked |
| 2026-09-28 | Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome FeedbackSCOUT meta-learns a belief latent shared by policy and forward dynamics model; at test time it updates the belief from dynamics prediction error, enabling fast adaptation and sim-to-real transfer (CoRL 2026). · code | CMU; Tsinghua | Test-time search | ✓ checked |
| 2026-09-28 | Action Chunking Proximal Policy Optimization with Feedback CorrectionPPO with a chunked actor and a standard state-value critic; ACPPO-Corr adds a stepwise feedback corrector inside each chunk. 25 IsaacGym/Bi-DexHands tasks: best aggregate IQM, +30.4% over PPO; per task vs best baseline, 13 wins, 8 ties, 4 losses. · code | Cornell; Seoul National University | On-policy PG | ✓ checked |
| 2026-09-28 | RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action DrivingSimulator-state failure diagnosis retrieves corrections from a human trajectory bank; correction SFT precedes safety-reward GRPO. Driving VLA: NAVSIM v1 PDMS 87.7→91.7; the same checkpoint scores 89.4 EPDMS on original NAVTEST scenes under v2 metrics. | Zhejiang University; Yinwang Intelligent Technology | General RL | ✓ checked |
| 2026-09-28 | WorldGuide: Learning Success-Failure Boundaries in Latent World Models for Vision-Language-Action PoliciesLatent world model pretrained on successful and failed trajectories, then contrastively tuned on matched success-failure pairs, gives a differentiable reward for VLA training and is dropped at deployment; LIBERO average 98.4% vs 97.1% for OpenVLA-OFT; SimplerEnv Google Robot 72.0%. | Dalian University of Technology; Beta Infinity; NTU | World models & eval | ✓ checked |
| 2026-09-28 | WAM-OPD: Sharpening World Action Models via On-Policy DistillationOn-policy distillation of task-specific teachers into a pretrained world action model via prefix-weighted trajectory replay over a fixed rollout pool, improving target tasks without extra environment interaction. | University of Science and Technology of China | On-policy PG | ✓ checked |
| 2026-09-28 | Unified Trajectory Matching Policy Optimization: Diverse T2I Generation and VLA GeneralizationTurns group-standardized rewards into a target distribution and matches it by forward KL over trajectory log-probs instead of maximizing reward; for VLAs, on average +8.0/+4.8 points over the best reward-maximizing baselines on simulated task/scene OOD (MetaWorld/CALVIN). | HUST; CAS; SJTU; Frontis.AI; Tsinghua | General RL | ✓ checked |
| 2026-09-28 | StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action TasksPrerequisite-gated, paced rewards from simulator-verified subtask predicates (not a learned model); RoboCasa365 GR00T-N1.5 49.1% vs 41.5% for the best terminal-reward baseline, and it beats Robometer on LIBERO-Long (96.6 vs 94.2). · code | Penn State, Amazon AWS AI | Rewards & progress | ✓ checked |
| 2026-09-28 | Adjoint Guidance Flow: Amortized Critic Guidance for VLA Policies (AGF)Lightweight guidance net regressed onto the adjoint costate replaces per-step critic backprop for a frozen flow VLA; 3.6x faster per step than QGF. Real YAM arm, pooled (n=75): base 66.7%, QGF 68.0%, AGF 84.0%. | KAIST; SKKU; GIST | Test-time search | ✓ checked |
| 2026-09-28 | FailPatch: Failure Residual Patching for Vision-Language-Action ModelsOffline, supervised hidden-space residual expert bank with a Null route, trained on logged failures of a frozen pi0.5. RoboTwin average 49.50 → 60.50, vs PPO 55.75. · code | Xi'an Jiaotong; Dexmal; Nanjing U | Residual & steering | ✓ checked |
| 2026-09-28 | ARS: Agentic Reward System for Robot LearningAn agentic VLM builds verified event timelines as offline progress labels; LIBERO-Long 57.9 vs 44.2 vanilla BC; exposes reward models that credit the wrong object. | AIRC, Midea Group | Rewards & progress | ✓ checked |
| 2026-09-28 | ★ F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-ImprovementA VLM diagnoses real pi0.5 failures, the scene is rebuilt in Isaac Sim, and the policy is co-trained on failure-conditioned sim data, then PPO-refined and redeployed. Real OOD success: 26.25% → 90.0%, vs 71.25% for budget-matched Targeted BC. | Nanyang Technological University; Xi'an Jiaotong University; Dexmal | Sim-to-real | ✓ checked |
| 2026-09-27 | Train Together or Merge Later? Unifying VLA Experts via a Shared Action InterfacePolicyWeave: shared-action-interface LoRA post-training plus context-guided sparse merging of VLA task experts; 18 RoboCasa365 tasks: 74.1% merging RL experts vs 60.7% joint RL. | University of Queensland (UQMM Lab) | On-policy PG | ✓ checked |
| 2026-09-27 | Dynamic Manipulation with World-Action Models via Counterfactual PlanningDPP: training-free; the WAM's rollout estimates interaction timing, tracked target motion gives the future position, and a counterfactual observation lets the WAM reuse static skills; real robot, grasping a moving object: 76.7% vs 0% for reactive replanning (30 trials). · code | KAIST | Test-time search | ✓ checked |
| 2026-09-27 | Beyond Conservatism: Recoverability-Conditioned Exploration for Model-Based Imitation LearningRECON: model-based IL splitting a conservative main policy from an explorer that seeks epistemically uncertain yet recoverable states (via multi-step imagination); improves interaction efficiency on locomotion, navigation, manipulation. | Nanjing University | World models & eval | ✓ checked |
| 2026-09-27 | DexTaG: Tactile-as-Guidance in Reinforcement Learning for Dexterous ManipulationUses glove tactile signals as RL guidance to steer policy search toward human contact patterns when retargeting mocap to robot hands; learns marker/hammer use, beats distance-based rewards. | UMass Amherst; Genesis AI | Sim-to-real | ✓ checked |
| 2026-09-27 | FoLD: Force-Informed Learning for Dexterous Articulated Object ManipulationCompensatory force fields from human demos plus robot contact state condition a PPO residual policy correcting retargeted dexterous-hand motions on articulated objects. Vs DexMachina over four simulated hands, overall completion ratio 0.606 → 0.772; real runs are open-loop replays. | USTC; CAS Institute of AI for Industries; Shenzhen University | Residual & steering | ✓ checked |
| 2026-09-27 | SwingRL: Adaptive Observation Reinforcement Learning with World-Model Prediction for Cable-Suspended Hoisting ControlRecurrent residual PPO policy over a classical anti-swing prior, plus an age-aware world model for delayed or lost sensing, for robot-arm cable-suspended payload insertion onto rebar pins. Hardest sim setting: 69.5% strict success; physical rig: 18/20 with no real fine-tuning. | Cambridge; Tsinghua; Cardiff; HKU | Residual & steering | ✓ checked |
| 2026-09-27 | Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement LearningOff-policy actor-critic diffusion policy with a two-branch score target gated by an HJ reachability critic: Q-score matching when feasible, recovery denoising otherwise; tested on quadrotor, fixed-wing, Safety-Gymnasium. · code | UC San Diego | General RL | ✓ checked |
| 2026-09-27 | Steer2Grasp: Inference-Time Embodiment-Aware Steering for Diverse Physically Feasible Grasp DiffusionTraining-free Feynman-Kac particle reweighting steers a frozen grasp diffusion model with deployment-specific rewards, shifting samples from infeasible to feasible grasp modes for single- and dual-arm setups. | IIIT Hyderabad; Johns Hopkins | Test-time search | ✓ checked |
| 2026-09-27 | Is Online Interaction Necessary for Recovery? A Minimalist Approach to Robust Planning via PerturbationPARS perturbs proprioceptive state and anchors only the later predicted trajectory to the demo, teaching recovery without extra interaction or expert queries; improves robustness on 51 RLBench tasks. | KAIST | Human corrections | ✓ checked |
| 2026-09-27 | Evolving Dexterous Robots from ScratchEvolution strategies search a contrastively learned genome of freeform manipulator bodies, with RL training each design; printed prototypes run zero-shot, and the real didactyl rotator turns unseen objects at 1.39 rad/s vs 0.74 for Fay et al.'s best design. | Northwestern University | Black-box search | ✓ checked |
| 2026-09-27 | Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-SimSimify reconstructs sim-ready 3D assets from RGB-D and optimizes object placements in massively parallel physics simulation at test time for unseen-object rearrangement; IROS 2026. | Robotics and AI Institute; Imperial College London | Test-time search | ✓ checked |
| 2026-09-27 | ZeroBot: Learning from Scratch in Minutes with Generative Real2SimSingle-view image-to-3D meshes feed massively parallel sim RL with a contact-state-sampling action space; no demos or pretraining, 87% real-world success after ~119 s training. · code | Robotics and AI Institute; Imperial College London | Sim-to-real | ✓ checked |
| 2026-09-27 | Robot-GST: geometry-aware spatial-temporal robot policy representation and evaluationBuilds 3D Gaussian-splat + SAM3D scenes from RGB-D for simulation-before-execution, filtering infeasible action sequences; validated on rigid, soft and deformable object manipulation. · code | KTH; EPFL; Idiap; Beihang; HKUST Guangzhou et al. | Test-time search | ✓ checked |
| 2026-09-27 | Recursive Harness Distillation across Agents for Robot ManipulationA strong agent distills its VLA-intervention experience into a playbook for a light agent, refined recursively from the light agent's execution feedback; real world (3 tasks): 64.0% vs 37.3% for pi0.5 alone; SimplerEnv Bridge: 66.7% vs 41.7% GR00T-only. | Seoul National University; KAIST | Test-time search | ✓ checked |
| 2026-09-27 | Schrödinger–Föllmer Actor–Critic: Diffusion Policy Improvement with Finite-Sample AnalysisSFAC: offline-to-online diffusion policy improvement via Doob h-transform drift correction estimated with paired self-normalized importance sampling and a minimax Bellman critic; finite-sample bounds, six continuous-control tasks. | Wuhan University | Off-policy critics | ✓ checked |
| 2026-09-27 | ActionGround: Training-Free Runtime Refinement of Frozen VLA PoliciesTraining-free neuro-symbolic layer wrapping frozen VLAs: a phase FSM applies rule-based corrections and an inertia-weighted Euler-Lagrange term adds dynamics; on 10 LIBERO-Spatial sim tasks, up to +6 pts success and +19.3 pts stability; <1 ms overhead. | IIT Madras; Indian AI Research Organisation; NTU | Test-time search | ✓ checked |
| 2026-09-27 | Achieve What You Imagined: Learning to Align Actions with Visual PlansFrozen action-conditioned world model scores whether a WAM's actions match its visual plan and goal; Flow Policy Optimization tunes the action head without online robot interaction or task-specific reward models; four real UR5 tasks: 43.4% to 75.1% mean (pi0.5: 61.4%). · code | KTH; Beihang; HKUST Guangzhou; Peking University; Great Bay University | World models & eval | ✓ checked |
| 2026-09-27 | ★ TimelyDAgger: Timing-Aware Expert Querying for VLA Policy ImprovementRobot-gated DAgger: asks for takeover when a PCA residual on VLA features exceeds a threshold that the expert's first post-takeover actions raise or lower. pi0.5 StackCube-Green (ManiSkill sim, oracle expert): 81 vs HG-DAgger 29, offline BC 49. | Tsinghua University; SEEN·E Robotic | Human corrections | ✓ checked |
| 2026-09-27 | ★ Principal Steering Subspaces for Online Adaptation of Frozen Generative Robot Policies (PSS)Finds noise directions that most move a frozen generative policy's actions using finite differences at 64 states; SAC steers those directions. Sim RoboMimic Diffusion-Square, final-five-evaluation mean, one seed: 0.878 vs 0.716 for full-latent steering. | XPENG Robotics; University of Michigan | Residual & steering | ✓ checked |
| 2026-09-26 | SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action ModelsReplay-free continual VLA fine-tuning as constrained optimization: method-of-multipliers dual penalty plus selective anchoring of parameters critical to earlier tasks; after three sequential LIBERO tasks, replay-free baselines score 0% on the first task, SAMBAR 84%. | UC Berkeley | General RL | ✓ checked |
| 2026-09-26 | RecastVLA: From Past Interaction to Future Control with Adaptive Policy StatesFlow-matching VLA carries a shared fast-weight policy state, updated at deployment from its own action-generation features without expert labels; +10.68 pts mean success over a matched policy without test-time training on RoboTwin Clean-to-Clean; also tested on 12 real-robot tasks. | South China University of Technology; Yuanwu Technology | Test-time search | ✓ checked |
| 2026-09-26 | Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RLStudies timestep weighting in ELBO-based flow-matching RL (CIFAR fine-tuning, from-scratch simulated robot control): the best weighting is task- and stage-dependent, motivating a policy-reward-gap hypothesis; dynamic schedules beat the best tested fixed weighting, by small margins on some robot tasks. | Tsinghua; Princeton; EPFL | On-policy PG | ✓ checked |
| 2026-09-26 | Residual Denoising Enables Sample-Efficient Multi-Agent Coordination on DemandCoordination head predicts a residual denoiser over frozen single-agent diffusion policies, trained on few collaborative demos plus self-distilled data. Sim: higher coordination success than baselines; real two-xArm7 task: ties full fine-tuning on coordination while retaining single-arm skills better. | UC Berkeley | Residual & steering | ✓ checked |
| 2026-09-26 | From Static Policies to Adaptive Priors in Offline Reinforcement LearningNeurIPS 2026 position paper: offline RL should produce adaptive policy priors that keep improving online via memory, exploration and self-correction (AORL); Bayesian offline RL as one route. | Mila; Université de Montréal; McGill | General RL | ✓ checked |
| 2026-09-26 | Rank Collapse Is Recoverable, Growing |Q| Is Not: Out-of-Sample Early Warning for Value Divergence in High-UTD Soft Actor-CriticIn high-UTD SAC, rank collapse is survivable, but how far log|Q| has climbed by step 15k ranks runs by how soon the training guard flags them (out-of-sample AUC 0.78 Walker2d, 0.98 Ant, at UTD 4), enabling early aborts. | Rutgers; Nanjing Tech; Nanjing University of Posts and Telecommunications; NYU | General RL | ✓ checked |
| 2026-09-26 | Constrained Flow Policy Updates: A Generalized Schrödinger Bridge ViewRAFALE: off-policy safe-RL actor-critic that differentiates an augmented objective through a flow policy's generation process, framed as a constrained Schrödinger bridge; cost within budget on seven Safety-Gymnasium tasks. | UC San Diego | General RL | ✓ checked |
| 2026-09-26 | MA-FPPO: Multi-Agent Flow-Pretrained Policy OptimizationOnline fine-tunes flow-matching-pretrained cooperative multi-agent policies via explicit action likelihoods and shared team advantages; average relative gains of 52.8% over the strongest offline baselines (30 settings) and 29.8% over purely online learning (38 matched-budget comparisons). · code | Sun Yat-sen University; NUS | General RL | ✓ checked |
| 2026-09-26 | Beyond Scripted Search: Sample-Efficient Reward Discovery via Agentic Black-box OptimizationARBO: an LLM agent builds its reward-search strategy at run time from a persistent evaluation history; with 128 reward samples, ten simulated Bi-DexHands tasks average 0.918 success vs 0.792 for the best baseline (Eureka). | NTU; Yunnan University | Rewards & progress | ✓ checked |
| 2026-09-26 | RE-0: Verified Recursive Improvement of Embodied Code-as-Policy Agents through Local On-Policy DistillationTeacher gives local corrections on the student's own failure histories; only environment-verified corrections become on-policy distillation supervision (RE-OPD), with a per-round gain lower bound. | Jilin University; Dalian University of Technology | Advantage-weighted BC | ✓ checked |
| 2026-09-26 | RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied AgentsTreats the embodied agent's supporting system (context/memory management, hierarchical skills) as a policy evolved from execution traces; lifts GPT-5.5 on EmbodiedBench from 56.9% to 72.7% (27.8% relative) and beats CaP-Agent0 under all LIBERO-PRO perturbations. · code | NTU; Beihang; NUS; Cloud Butterfly; SJTU | Test-time search | ✓ checked |
| 2026-09-26 | SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy AdaptationSplits long-horizon tasks into atomic skills, picks the most-failing ones, restores simulator states captured just before them, and RL-updates shared per-family LoRA adapters with LLM-written success checks. LIBERO average (GR00T N1.5) 53.1% → 65.4% after three rounds, no extra demos. | MBZUAI; University of Sydney; Oxford; AI2 Robotics; Waterloo; Melbourne | On-policy PG | ✓ checked |
| 2026-09-25 | Causeway: Restoring Task Accessibility for Instruction Switching in VLA PoliciesTraining-free inference-time steering: backpropagates through frozen VLA decoding to apply state-directed action-stream edits that recover from 'task islands'; instruction-switch success rises from 3-26% to 47-65% on LIBERO-Goal, plus xArm validation. | University of Maryland; USC | Test-time search | ✓ checked |
| 2026-09-25 | MA-WAM: Multi-Agent World-Action Model for Test-Time PlanningA multi-agent world-action model predicts team return for candidate joint actions so a frozen multi-agent flow policy can rank them at test time; +22.0% mean relative gain over direct execution across 30 offline MARL settings; scoring takes 12.1 ms (A100). · code | Sun Yat-sen University | Test-time search | ✓ checked |
| 2026-09-25 | G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow PoliciesRefines frozen offline multi-agent flow-policy actions at test time with one globally normalized, projected critic gradient; improves 20 of 24 MPE/SMAC settings, with mean relative gains of 9.2% (MPE) and 8.9% (SMAC) at ~6% extra inference latency. · code | Sun Yat-sen University | General RL | ✓ checked |
| 2026-09-25 | Learning to Leverage Compliance: A Policy-Admittance Learning Framework for Robotic InsertionLeCo: multirate contact feedback becomes conflict and high-force reward costs so the policy cooperates with a fixed admittance controller; 94% aggregate success on four real connector tasks, ~30% lower successful-trial force peaks than the comparison baseline. | Northwestern Polytechnical University | Rewards & progress | ✓ checked |
| 2026-09-25 | Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb ExoskeletonMO-HILBO: multi-objective human-in-the-loop Bayesian optimization inferring personalized Pareto-optimal exoskeleton controllers over metabolic cost and comfort; Pareto orderings consistent with validation trials; open-source mohilo package. · code | Georgia Tech | Black-box search | ✓ checked |
| 2026-09-25 | Generate, Track, Improve: Perceptive Multi-Skill Humanoid Locomotion with RL-Fine-Tuned Motion GeneratorsOff-policy RL (structured search plus advantage-weighted regression) fine-tunes a depth-conditioned flow-matching motion generator feeding a tracking policy; in simulation vs the pre-fine-tuned generator, up to +25 points terrain traversal and +80 skill selection; G1 hardware shown qualitatively. | Caltech | Advantage-weighted BC | ✓ checked |
| 2026-09-25 | Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator ControlOnline MBRL with a probabilistic dynamics ensemble and sampling-based MPC learned from scratch on an 11.5-ton excavator; sub-centimeter mean path error after 40 minutes of real interaction. | ETH Zürich; Universidad de Chile | World models & eval | ✓ checked |
| 2026-09-25 | HapticWorld: an Interactive World Simulator with Real-time Torque FeedbackLearned interactive world simulator that predicts joint torques and renders haptic feedback to teleoperators; 1.6x data-collection throughput, policies from generated demos reach 54/60 real vs 19/60 vision-only. · code | Ant Group; UIUC; NVIDIA; Stanford | World models & eval | ✓ checked |
| 2026-09-25 | Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy GradientSTL-SVPG: population-based Stein variational policy gradient using smooth STL robustness as a trajectory-level objective; best success on 5 of 6 quadcopter/manipulator benchmarks, with sim-to-real transfer. | UPenn; Lehigh University | Rewards & progress | ✓ checked |
| 2026-09-25 | Towards VLA-Dreamer: Refining VLA Behavior Using World ModelsConcept paper (no experiments yet) proposing a JEPA-style world model trained in a VLA's vision-encoder embedding space, to test whether those embeddings predict action outcomes and to plan short horizons by sampling VLA actions toward goal images. | University of Hamburg | World models & eval | ✓ checked |
| 2026-09-25 | RoboMonitor: Label-Efficient Runtime Monitoring of Robot Task Execution via Predictive Representation LearningRuntime execution monitor pretrained with action-conditioned future-feature prediction, inverse dynamics and masked prediction; 93.1% phase accuracy from 52 labeled episodes, 35/40 real trials with zero false recoveries. | Siemens; Worcester Polytechnic Institute | Test-time search | ✓ checked |
| 2026-09-25 | PHIRL: Aligning Learned Rewards with Task Progress for Inverse Reinforcement LearningIRL framework calibrating learned rewards with task-progress annotations; higher return and task success with only 20% of demonstrations annotated, and robust to reward hacking. | Yu et al. | Rewards & progress | ✓ checked |
| 2026-09-25 | Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional RecoverabilityRestores simulator states along failed rollouts to estimate recoverability and generate targeted recovery data; SmolVLA recovery success 34.6-38.4%, 5.8-6.7 points over uniform sampling. | Skoltech | Human corrections | ✓ checked |
| 2026-09-25 | VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RLSim-trained TD3 residual that reads the VLA's own latent, aligned sim-to-real with an OT mapper. Real Flower: button 67.5% → 100%, block push 22.5% → 50%. | MSRA Tokyo; KAIST; U Tokyo | Residual & steering | ✓ corrected |
| 2026-09-25 | Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models (FIND)Real RL without per-episode scene resets: a VLM picks feasible, weakest tasks and judges success for a critic-gated residual on a frozen pi0.5. 8-task human-assessed success 55% → 71.9%; a 456-episode run needed 30 scene-recovery interventions. | TU Darmstadt; Hessian.AI; U Augsburg; Tongji University; Shanghai Research Institute for Intelligent Autonomous Systems | Residual & steering | ✓ checked |
| 2026-09-24 | Training-free Behavior CloningBehavior Predictive Control: action-aware retrieval over a demo bank, a Hankel-based action-continuation prior and a closed-form residual correction replace policy training; competitive with pi0.5 (better in some cases), fits in seconds, runs 75+ Hz on Jetson Orin Nano. | Stanford | Test-time search | ✓ checked |
| 2026-09-24 | World Action Agent: Harnessing VLMs for Robot Manipulation via World Action RehearsalVLM agent harness with contact views, imagined action rehearsal and in-view correction plus demo-derived skills; 75.6% on LIBERO-Pro; its traces lift a small model's OOD success from 1.7% to 43.3%. | HKUST Guangzhou; CUHK; Knowin AI | Test-time search | ✓ checked |
| 2026-09-24 | Aim Short to Reach Far: Your Frozen World Model Can Plan Better Than You ThinkAnchored Planning: training-free fix for frozen latent world-model planners that retrieves offline trajectory segments and plans toward intermediate anchor observations, beating final-goal search on long-range goals across four tasks. · code | Tsinghua; Manifold AI | Test-time search | ✓ checked |
| 2026-09-24 | RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot LearningTwo-phase offline evolution (capability curriculum, then self-evolution) improves typed Policy APIs, a ReAct harness and long-term experience memory; at deployment ReAct calls the frozen APIs; 46.0% on LIBERO-Long vs at most 4.0% for Code-as-Policies baselines. | CUHK; HKUST Guangzhou; Knowin AI | Black-box search | ✓ checked |
| 2026-09-24 | HarnessPAI: An Evolving Harness for Physical AIEvolves a program-level harness across rollouts from execution feedback without retraining the wrapped action model; LIBERO-PRO 96.5% vs 34.9% for pi0.5 alone (+61.6 points); fine-tuning pi0.5 on harness-collected rollouts lifts it to 73.7%. | Darwin Agent Team | Black-box search | ✓ checked |
| 2026-09-24 | AdaHVLA: Adaptive Harnesses for Long-Horizon Vision-Language-Action ExecutionMulti-agent loop (analyze, revise, assess) refines the code harness around frozen VLAs from rollout experience via a stateful revision graph; in simulation, NaVILA-LH test success 22.5% to 31.7-57.5% across six adaptation LLMs, manipulation up to +30.8 points. · code | UNC; USTC; HKUST; CUHK | Black-box search | ✓ checked |
| 2026-09-24 | Online Sim-to-Real Adaptation via Closed-Loop System ModelingOSRAM: meta-trains a closed-loop dynamics model in randomized sim, fine-tunes on limited real data, then optimizes reference commands online for a frozen controller; reduces bipedal tracking error on hardware. · code | Duke University | Sim-to-real | ✓ checked |
| 2026-09-24 | Privacy-Preserving Prompted Policy Search for Robotic ControlLLM-as-optimizer policy search that sends only encoded policy parameters and scaled rewards to cloud LLMs; beats the baseline prompted search on 7 of 10 control tasks. | Oakland University; Wayne State; OORT; Columbia | Black-box search | ✓ checked |
| 2026-09-24 | AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive ControlJoint-embedding latent world model with residual dynamics plus action-recovery regularization (heads dropped at test time) keeps candidate actions distinguishable for MPC; OGBench-Cube hard-start success 52.0% vs 3.7% for a matched LeWM baseline; zero-shot Franka pick-and-place 42.2% to 71.1%. · code | Nanjing University | World models & eval | ✓ checked |
| 2026-09-24 | Policy-Calibrated DAgger: Offline Calibrated Noise Injection for Imitation LearningCalibrates DAgger-style noise injection offline from a diffusion policy's own action spread along expert trajectories; beats un-noised DAgger and matches hindsight-optimal noise without sweeps. | Carnegie Mellon University | Human corrections | ✓ checked |
| 2026-09-24 | Self-Adaptive VLA for Robust Robot DeploymentPost-trains a VLA with a plug-in context encoder that turns its own failed rollouts into an AdaLN context token; summing tokens across retries lifts average success under actuation bias from 7.5% to 72.5% within 6 trials (nominal 88.8%), four tasks. | UMass Amherst; Genesis AI | Test-time search | ✓ checked |
| 2026-09-24 | Learning from Mixed-Quality Deployment Experience for Robot ManipulationIQL chunk critic with future latent supervision conditions a diffusion actor on high-value chunks, then selects best-of-N; real Franka StackCup improves from 18/25 to 25/25 trials. | NTU; Changan; Cloudbutterfly | Advantage-weighted BC | ✓ checked |
| 2026-09-24 | ★ Robot Demos Are Easy. Reliability Is Hard (Jason Ma, Dyna Robotics talk)A learned video progress model monitors deployment: progress dips flag errors and steer targeted recovery-data collection and retraining. Dyna-1 napkin folding: 99.4% success over a 24-hour run; Dyna says conventional pipelines stall near 80% (talk, no paper). | Dyna Robotics | Rewards & progress | ✓ checked |
| 2026-09-24 | Res-HIL: Human-Guided Residual Reinforcement Learning for Sample-Efficient Dexterous ManipulationZero-initialized TD3 residual on a 20-demo ACT. Human takeovers become regression targets plus penalties on the steps just before them. After 10 min: 100/64/50/66/92% vs HIL-SERL 90/30/10/0/0 and ResFiT 0/0/0/0/0. | Siemens; TU Munich | Residual & steering | ✓ checked |
| 2026-09-23 | Context-Continuous Preference Learning for Exoskeleton PersonalizationCCPL: a Gaussian-process preference model shares feedback across nearby exoskeleton operating contexts; retrospective analyses of ankle/elbow data (nine adults) estimate ~37%/17% fewer exposures than independent learning for equal reconstruction accuracy; online benefit not yet shown. | Korea University; WIRobotics | Black-box search | ✓ checked |
| 2026-09-23 | RegenHarness: A Robot Agent Harness with Evidence-Gated Recursive Self-ImprovementRobot agent harness with isolated planning/supervision/verification/recovery contexts and a commit gate; evidence-gated self-improvement revises harness configuration from execution records without weight updates; quadruped warehouse demo. | Country Garden Services; HUST; Omni AI | Black-box search | ✓ checked |
| 2026-09-23 | BrickCraft-Duo: Efficient Dual-Arm Skill Learning and Refinement for Compositional Long-Horizon AssemblyLearns reusable dual-arm brick-assembly skills from demos, composes them with stability-aware reasoning, and refines them with human-in-the-loop correction; at least 95% step-level and at least 60% long-horizon success on five real tasks (up to nine steps). | Tsinghua; CMU | Human corrections | ✓ checked |
| 2026-09-23 | Banana Kick: Response-Informed Skill Evolution for Humanoid SoccerRISE ranks bounded objective changes by response sensitivity from cached rollouts and accepts only verified progress, evolving a G1 humanoid's ordinary kick into a curved one. Sim joint target attainment 50.9% vs 15.2% for a learning-progress curriculum; 30 hardware trials. | CMU; General Motors; UT Arlington | On-policy PG | ✓ checked |
| 2026-09-23 | An Analysis of Streaming Deep Reinforcement Learning for Adaptive Continual Learning in RoboticsAfter pretraining, streaming deep RL adapts quadruped policies online to unforeseen changes, improving task success up to 90% over the pretrained policy and beating batch on-policy; manipulation only partially succeeds. | Yale University | On-policy PG | ✓ checked |
| 2026-09-23 | TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuningTAMP with VLM-added predicates asks for teleoperation only on steps it cannot plan; on one task it collects 2.9x the demos of full teleop in equal human time; 20 demos/task lift pi0.5-DROID's average success 0% to 60% (five real tasks). · code | Stanford; Princeton | Human corrections | ✓ checked |
| 2026-09-23 | What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill DiscoveryAgent writes/integrates new manipulation skills over 123 improvement rounds; gains fail to compound due to relational perception limits, skill chains stuck on early steps, and flawed evaluators. | National University of Singapore | Black-box search | ✓ checked |
| 2026-09-23 | Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor PolicyContact-aware morphometric retargeting of human hand motion, RL refinement for dynamic feasibility, then visuomotor distillation; 89.3% zero-shot success over 300 real trials on 30 objects. · code | UC Berkeley | Sim-to-real | ✓ checked |
| 2026-09-23 | Generalizable Robotic Insertion with World ModelsTD-MPC2-based world model from wrist-camera depth and proprioception, trained on up to 90 insertion tasks; in simulation, 56% zero-shot success on unseen part geometries vs 7% for a model-free baseline; finetuning on held-out parts is more data-efficient than from scratch. | NVIDIA; UC San Diego; USC | World models & eval | ✓ checked |
| 2026-09-23 | CereVLA: Cerebellum-Inspired Consequence-Aware Residual Governance for Efficient Vision-Language-Action ExecutionFlow-based residual refinement on a frozen action-chunked VLA, gated by a recurrent consequence predictor that suppresses unfavorable corrections. Real SO-101 pick-and-place, 40 trials: success 57.5% → 90.0% vs frozen SmolVLA, 19.6% fewer control steps on successful trials. | HKUST Guangzhou | Residual & steering | ✓ checked |
| 2026-09-23 | ★ Dissecting Advantage-Guided Post-Training for Vision-Language-Action PoliciesAblates separately how critic-derived advantages are built, calibrated and used; n-step TD, per-value-bin normalization and soft exponential weights on the pi0.5 loss worked best. Real bimanual, 4-task mean success: SFT 0.11, source-weighted DAgger 0.45, continuous weighting 0.74. | Shanghai Jiao Tong University; Xiaomi Robotics | Advantage-weighted BC | ✓ checked |
| 2026-09-23 | ★ Physical Self-PlayPreviews RL post-training via self-play: in simulated robot soccer the policy's one stated objective is to score, playing recent versions of itself, then is transferred to a real humanoid. No metrics; 140 simulated years of play, real match on video. | Skild AI | Industry loops | ✓ checked |
| 2026-09-23 | BEE: Intervention-Adaptive Real-World Reinforcement Learning with Vision-Language-Action ModelsResidual actor on a frozen pi0.5, constrained toward human corrections by a learned per-dimension Gaussian correction model (Mahalanobis constraint). Matched online-data budget, 3 real + 1 sim task: average success 91.2% vs RLT 57.5%, DSRL 42.1%, SFT base 38.8%. | AgiBot; SCUT | Residual & steering | ✓ corrected |
| 2026-09-23 | Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action PolicyNovelty/competence-gated exploration-noise controller for PPO fine-tuning of SmolVLA on LIBERO-10. No task collapses (pooled definition) in its 3 seeds, vs 2 of 3 seeds for fixed noise and every learned-noise seed run to iteration 200; no arm beats BC. | KIT, Cukurova University | On-policy PG | ✓ checked |
| 2026-09-22 | Hill Sampling for Test-Time Scaling: A Simple and Better Alternative to Repeated Sampling, Evolution, and TrainingFrozen-LLM hill climbing conditions new candidate programs on the best verified solution. In non-robotics tests, it outperforms repeated sampling and test-time ES; the paper reports circle-packing records relative to published methods. | Oracle | General RL | ✓ corrected |
| 2026-09-22 | Sample, Simulate, Select: Physics-in-the-Loop Text-to-Motion for Humanoids Without TrainingTraining-free best-of-N: sample 8 text-to-motion candidates, simulate each under the pretrained SONIC tracker on Unitree G1, keep the best-executed; upright execution rises from 80.5% (first sample) to 89.5% on 4,184 HumanML3D test prompts. | University of Bonn | Test-time search | ✓ checked |
| 2026-09-22 | DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End DrivingClosed-loop driving simulator built on a simulator-grounded autoregressive video model; under its new FDpi fidelity metric it beats the strongest prior closed-loop simulator 1.6x on nuScenes and 4.7x on NAVSIM; its Navhard-CL benchmark exposes missing recovery behaviors. · code | UCLA; Toyota Research Institute | General RL | ✓ checked |
| 2026-09-22 | Know Your Body: A Harness for Direct and Self-Improving Robot Control with VLMsKnowBody: frozen-VLM control harness with an explicit, revisable body model seeded from one trajectory and refined by interaction; 75% vs 25% completion for native harness on four real-robot tasks. · code | Nanjing University; Ant Group; Zhejiang University | Black-box search | ✓ checked |
| 2026-09-22 | HABILIS Brain 0: Geometry-Change Supervision for Vision-Language-Action and Residual Flow RecoveryGC-VLA predicts multiview geometry-change tokens, then adds Geometry-Conditioned Residual Flow learned from closed-loop feedback; LIBERO success rises from 95.20% to 99.55%. | Tommoro Robotics | Residual & steering | ✓ checked |
| 2026-09-22 | Imperfection for Precision: Upcycling Imperfect Data for High-Precision Robotic Manipulationε4P co-trains a flow-matching VLA on imperfect data by noise level (low-precision target-task data at high noise, high-precision mismatched-task data at low noise); real robot: up to +31.7 points; replacing equal high-quality data costs 4.2 points on average. | Samsung Robotics eXperience; NUS; UIUC | Advantage-weighted BC | ✓ checked |
| 2026-09-22 | DynaForge: Planning-Guided Residual Learning for Dynamic Manipulation Demonstration GenerationPlanner plus object-centric IK with a residual RL policy and implicit curriculum lifts sim demo-generation success 41.30% → 78.37%; DP3 policies trained on its data reach 30-60% on three real tasks vs 0-10% with DOMINO data. | LandSpace; SJTU | Residual & steering | ✓ checked |
| 2026-09-22 | SafeLoop: Risk-Aware Rollback for Vision-Language-Action ManipulationNon-invasive wrapper predicts collision/object-failure risk from vision and proprioception, then checkpoints or rolls back and re-queries the frozen VLA; ~70% fewer hazards on 24 LIBERO and 3 real tasks. · code | Nanjing University; HKUST Guangzhou; Beijing University of Technology | Test-time search | ✓ checked |
| 2026-09-22 | ★ PAKT: Physically-Aligned Kinesthetic Teaching for Reinforcement LearningOperators correct HIL-SERL-style real-world RL by hand-guiding the arm via admittance control sharing the policy's action space, limits and impedance tracker. Vs HIL-SERL, four real insertion tasks: 62%-86% fewer interventions; 23%-48% lower cycle time, mostly from the new controller stack. | NVIDIA | Human corrections | ✓ checked |
| 2026-09-22 | HiRE: Hindsight Reward Editing for Policy FinetuningTraining-free reward: a vMF-kernel density log-ratio of foundation-model features (DINOv2, SigLIP) from success vs failure buffers, added via potential-based shaping; real bottle pick-and-place rises 20% to 67% (30 trials), while sparse and Robometer baselines nearly fail early in training. · code | UC Berkeley, CUHK | Rewards & progress | ✓ checked |
| 2026-09-22 | RouteRLT: Learning When and Which RL Specialist Should Control a Vision-Language-Action PolicyA learned router decides when a per-phase RL specialist (RL-Token-style readout) takes over from a frozen SmolVLA. Full-task success on three LIBERO-Object tasks 85.00% → 92.22%; real cable insertion 6.7% → 35.0% (operator-aligned handoff). | University of Toronto | Residual & steering | ✓ checked |
| 2026-09-21 | Learning Beyond What Humans Can DemonstrateGLIDE infers task failure modes as executable guardrails, refined from trajectory feedback, that filter teleop and policy commands; data-collection success 0-10% to 70-90%, guarded policies 60-70%. · code | UC Irvine | Human corrections | ✓ checked |
| 2026-09-21 | Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AIJoint-trajectory-conditioned autoregressive diffusion world-model simulator with streaming rollouts, 24 FPS after optimization, and multi-embodiment/multi-camera support; code and weights released. · code | D-Robotics AI Lab | World models & eval | ✓ checked |
| 2026-09-21 | Visuomotor Robotic Pruning in Planar Orchards Using Hybrid Reinforcement LearningHybrid PPO mixing offline motion-planner demos with online sim rollouts trains an optical-flow visuomotor pruning policy; 49.9%/46.0% sim success (apples/cherries), zero-shot transfer in 38 physical trials, and 7/15 vs 4/15 for point-cloud RRT-Connect in the lab. | Oregon State University | Sim-to-real | ✓ checked |
| 2026-09-21 | GraspTune: Tactile-Driven Execution Refinement for Robust GraspingDiffusion-pretrained residual policy, aligned with PPO, applies bounded tactile-driven TCP corrections during grasp execution. Sim stable-grasp success +9.55 to +20.70 pts across four proposal generators; real UR5e GraspNet 71.0% → 84.3% (600 trials each). | HKUST Guangzhou; KTH | Residual & steering | ✓ checked |
| 2026-09-21 | Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy LearningResidual RL repairs human mocap in sim (MANO hand); contact-anchored retargeting plus a second residual policy adapt it to robot hands. Sim replay success 7.3% → 59.3% (single-hand), 16.0% → 62.4% (dual-hand); four hardware tasks use no real-robot training data. · code | DexGEM Lab; SJTU; Tongji; DexRobot | Residual & steering | ✓ checked |
| 2026-09-21 | InsertAnything: Generalizable Contact-Rich Precision Insertion from Simulation to RealitySim-only RL insertion policy using 3D fingertip force feedback and a decoupled gated reward; real insertion down to 0.02 mm nominal clearance, 20/20 on ManipulationNet peg-in-hole (Human-in-the-Loop protocol, autonomous insertion motions), and 95.0% (152/160) zero-shot on eight unseen real tasks. · code | UCAS; Institute of Automation CAS; PaXini AI | Sim-to-real | ✓ checked |
| 2026-09-21 | Performance-Preserving Online Adaptation in Social Navigation via Diffusion SteeringApplies DSRL to social navigation: trains only the latent-noise policy over a frozen diffusion policy distilled from multi-seed RL policies, adapting to new social conventions; hardware-in-the-loop validated. | Kyushu University | Residual & steering | ✓ checked |
| 2026-09-21 | D-JEPA: A Decision-Aligned Latent World ModelLatent world model learning ordinal relations among candidate futures from executed outcomes via a permutation-equivariant operator; lifts VLA success on four RoboTwin tasks from 61.72% to 76.76% and a V-JEPA 2-AC planner's on two real tasks from 64.0% to 81.0%. · code | HKUST Guangzhou; Boston University; SJTU | Test-time search | ✓ checked |
| 2026-09-21 | Beyond Visual Quality: A Study of Test-Time Planning with World Action ModelsStudies picking among sampled actions by their imagined futures in world action models: an oracle pick raises same-state success from 68.9% (random pick) to 79.2%, but tested selectors leave much of that gain unrecovered; opportunity concentrates in few decisions. | Oxford; Purdue; UWE Bristol | Test-time search | ✓ checked |
| 2026-09-21 | Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement LearningDSRL over a frozen pi0; its critic imagines 10-step visual/torque futures with confidence weighting. Four real tasks, 50 trials each, 100 RL trajectories including 50 warm-up: success improves 23.6% relative over DSRL and 60% relative over pi0. | KAIST; USTB; SJTU; ICT CAS | Residual & steering | ✓ checked |
| 2026-09-20 | ReVeal: A Reconstruction-Aware Real-to-Sim Framework for VLA Policy EvaluationScores phone-video reconstructions with view and planar-geometry fidelity metrics for VLA evaluation; the authors' depth-supervised PGSR-D vs 2DGS raises real-sim Pearson r 0.773 to 0.960 and cuts MAE 0.246 to 0.050 (GR00T, SmolVLA, pi0.5; 8 humanoid tasks). | Purdue; Samsung SDS Research America | World models & eval | ✓ checked |
| 2026-09-19 | Anatomy of a Closed-Loop Collapse: A Causal Case Study of a Compressed VLA PolicyAn 8-layer Octo-Base distillation passes offline checks but scores 0/72 on simulated WidowX pick-and-place (teacher 40/72); replacing half the training stream with deployment-distribution teacher rollouts restores teacher parity (18/36 vs 17/36). · code | The Ohio State University | Human corrections | ✓ checked |
| 2026-09-19 | Verti-WM: A Physics-Aided Exteroceptive World Model for Off-Road Reinforcement LearningFuses a frozen Transformer (rigid terrain) with a neuro-symbolic terramechanics model (deformable) to replace simulator rollouts for off-road RL; PPO trains in 1.36 h vs 33.48 h in Chrono at similar success; real robot 4/5 vs 2/5 for direct sim-to-real. | George Mason University | World models & eval | ✓ checked |
| 2026-09-19 | Expert-Play Contouring Control: Faster-than-Demonstration Planning from Slow Expert and Fast PlayEPCC: extracts a time-independent progress contour from slow expert demos, trains a world model on fast play data, and plans to maximize contour progress; 2.0x the Diffusion Policy baseline's throughput, averaged over three simulated manipulation tasks. | Seoul National University; Georgia Tech | World models & eval | ✓ checked |
| 2026-09-19 | Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy OptimizationStarts world-model branches at states where the flow policy's 3-step and 5-step samples disagree. RoboTwin average 58.8 vs 54.3 for VLA-MBPO. | Nanjing University; Cirquar; Mila; Université de Montréal | World models & eval | ✓ checked |
| 2026-09-19 | ForceRFT: Refining VLA Actions through Force-Guided Residual Reinforcement LearningForce-aware TD3 residual bounded to 1 mm and 0.5 deg on a frozen force-conditioned SmolVLA, plus human corrections. Plug insertion: 12/30 prior → 25/30 within a 30-min budget. | EIT Ningbo; Jiangnan U; SJTU; HKUST-GZ; NUS | Residual & steering | ✓ checked |
| 2026-09-19 | ★ Stable and Efficient Real-World Online VLA Post-Training via Asynchronous Replay-Anchored Policy Improvement (RAPolicy)A rollout copy of pi0.5, re-synced after each small update group, collects data while the learner trains an IQL-style chunk critic and one-step flow actor only on stored actions. Real, 4 tasks, 20 trials each: 86.3% average vs EXPO-FT 50%. · code | Fudan University; Shanghai Key Laboratory of Multimodal Embodied AI; Singapore Management University | Off-policy critics | ✓ checked |
| 2026-09-18 | FAN: Foresight Action Normalization for Continual Adaptation of Vision-Language-Action ModelsCompares five action-normalization strategies for continual VLA adaptation; FAN freezes stats from a small task-independent calibration set, avoiding coordinate drift, best across four real single-arm/bimanual task streams. | HKU; SUSTech; EIT Ningbo; HUST; INFIFORCE | General RL | ✓ checked |
| 2026-09-18 | Simultaneous Forward and Inverse Human-in-the-Loop OptimizationBootstraps a forward outcome model, uses it for active preference queries that learn a user-specific reward, then optimizes the assistive policy without extra trials; in simulation, an estimated 413 vs 563 queries (mutual information) to reach 10% preference error. | Stanford | Human corrections | ✓ checked |
| 2026-09-18 | ProTracer: Proprioception-Guided Failure Diagnosis in Robot ManipulationTraining-free VLM framework that turns proprioceptive robot-state signals into text to detect, categorize, explain and localize onset of manipulation failures; introduces FailTime benchmark. | Adelaide University | Rewards & progress | ✓ checked |
| 2026-09-18 | Stability-aware Residual Reinforcement Learning Framework for Robotic Manipulator Disturbance CompensationResidual RL over an NMPC + disturbance-observer loop, with a latent disturbance estimator and an ISS-derived action bound. Real 6-DOF arm, zero-shot, vs NMPC+DOB: tracking error −27.8% (circle), −16.7% (figure-eight); path RMSE −38.0% under unseen base vibration. | Hanyang Univ.; Kyonggi Univ.; Kookmin Univ. | Residual & steering | ✓ checked |
| 2026-09-18 | Fewer Steps, Better Actions: Rethinking Flow-Matching Inference for VLA PoliciesCoda: frozen flow-matching VLA runs few-step sampling, then a lightweight Transformer predicts a demo-supervised endpoint residual; RoboTwin 50 tasks 71.64%→74.68% (5 steps); SmolVLA 2-step 60.8%→69.4%. | ICT CAS; UCAS; Zhongke Haichuan Intelligent | Residual & steering | ✓ checked |
| 2026-09-18 | Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World ModelsFreezes the latent world model and learns small residuals around the predictor from its own prediction error, no rewards; over 21 AdaJEPA-benchmark conditions, 1.3x frozen-model success, 95% of the best AdaJEPA variant, 97-99% fewer adapted parameters than AdaJEPA. | Montanuniversität Leoben | World models & eval | ✓ checked |
| 2026-09-18 | FRAMES: Failure Recovery And Monitoring of Embodied Skills for Humanoid Loco-ManipulationPlanner, VLM Monitor and Recovery agents supervise humanoid mid-level skills; monitor catches 48/50 failures and accepts 46/50 successes (94% accuracy) over 100 MuJoCo trials. | Duke University | Test-time search | ✓ checked |
| 2026-09-18 | CommitFlow: Semantic Commitment Verification and Local Correction for Long-Horizon Robot Manipulation VLA ExecutionClosed-loop wrapper for a frozen VLA: a semantic commitment monitor holds back dependent actions until stage conditions hold, BoundaryFlow adds gain-calibrated local corrections; 75.9% mean success on ten RoboTwin 2.0 tasks, 22.7 points above pi0.5 (53.2%). | National University of Defense Technology | Test-time search | ✓ checked |
| 2026-09-18 | CARF: Contrastive Attraction-Repulsion of Failure-Guided Flow MatchingProgress-based scorer trained on successes labels imperfect-data segments; flow-matching policy is attracted to progressive segments and repelled from failure-critical ones, beating baselines in sim and real. | UC Berkeley | Advantage-weighted BC | ✓ checked |
| 2026-09-18 | CounterPlay: Counterfactual Post-Training for Self-Play Driving PoliciesPost-trains self-play driving policies by backtracking from failures via value estimates, retrying under reward-conditioned styles, keeping verified retries; BehaviorBench SOTA with 1B transitions (1% of anchor's 100B). | University of Freiburg; CARIAD; KIT | General RL | ✓ checked |
| 2026-09-18 | A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation PoliciesReplays simulated expert trajectories open-loop on a real Franka FR3 to record real observations, unifying data collection, closed-loop evaluation and sim-to-real gap measurement for chunk-based VLAs. · code | ISIR; CNRS; Sorbonne University | Sim-to-real | ✓ checked |
| 2026-09-18 | Latent Policy Steering: An Efficient and Flexible Framework for Cross-Embodiment TransferEmbodiment-agnostic world model pretrained with optical flow, finetuned on target actions, steers a frozen base policy via latent-space search; with 50 target demos, relative gains of 16%/62% (Robomimic/real) for Diffusion Policy and 8%/14% for Pi0.5. | Carnegie Mellon University | Test-time search | ✓ checked |
| 2026-09-18 | Scaling Vision-Language Reward Learning for Robot Manipulation in Parallel SimulationRAPID: GPU-parallel IsaacLab rollouts plus single-request VLM preference labeling for reward learning; vs an RL-VLM-F baseline on five simulated Franka tasks, 8.0x end-to-end speedup, 95.5% fewer API calls, success 86.3% to 98.7%. · code | KU Leuven; Flanders Make | Rewards & progress | ✓ checked |
| 2026-09-18 | VLA-Scope: Shift-Aware Failure Prediction for Vision-Language-Action ModelsTwo-stage runtime monitor for OpenVLA: detects and classifies input shift, then logistic regression on shift category, action-prefix and progress features predicts failure; LIBERO-Spatial: 0.945 OOD ROC-AUC, 0.850 failure ROC-AUC after 60 actions (0.791 without progress). | Texas Tech; Purdue | Test-time search | ✓ checked |
| 2026-09-18 | VT-Bridge: Bridging Pretrained Foundation VLAs to VTLAs via Lightweight Residual Adaptation0.98M-parameter residual adapter fed estimated force/torque signals, on a LoRA-fine-tuned then frozen pi0, pi0.5 or SmolVLA; trained on up to 50 teleoperated demos per task (supervised, not RL). Four real contact-rich tasks: completion 11.7% (fine-tuned VLA) → 62.9%. | TU Munich; MBZUAI; Shanghai University | Residual & steering | ✓ checked |
| 2026-09-18 | SynthDemo-RL: Breaking the Zero-Reward Barrier in VLA Adaptation with LLM-Guided Synthetic DemonstrationsPrivileged-state scripted teacher with LLM-revised waypoints synthesizes demos; the VLA is SFT'd, then PPO-refined with binary reward. LIBERO-PRO, no new human demos: rescues all 27 zero-success tasks (direct PPO: 10); 97.8%/97.1% Position/Task-axis averages. | Fujitsu; Institute of Statistical Mathematics | On-policy PG | ✓ checked |
| 2026-09-18 | ★ SeeQ: Training Generalist Value Functions for Long-Horizon Robotic ManipulationTrains a critic that only values finishing the active subtask (reward at subtask end, no bootstrapping across boundaries) and reranks 8 samples from a frozen pi0.5 with it. Real bimanual, 24 trials per task: 35.4% → 66.7% average. · code | Carnegie Mellon University | Test-time search | ✓ checked |
| 2026-09-18 | From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention (PARTS: Policy Adaptation with RL on Targeted Subtasks)Masked, bounded TD3+BC residuals on human-identified bottleneck subtasks of a frozen pi0.5 SFT policy, gated by agent-written selectors and verifiers. Full task 50% → 95% (Franka) and 32% → 61% (YAM), with tens of minutes of real-world RL per task. | UT Austin; Autel; UC Berkeley | Residual & steering | ✓ checked |
| 2026-09-17 | Learning and Transferring Closed-Loop Robot SoftwareCoding agent iteratively refines closed-loop robot policy code with simulation feedback (RoboCasa 28.3% to 64.2%); archived optimized code transfers to new tasks, +15.6 points over unoptimized references. | Sakana AI | Black-box search | ✓ checked |
| 2026-09-17 | Accelerating Visual Policy Learning with Sampling-Based Model Predictive ControlSGPS: BC-initialized depth policies alternate sampling-based MPC action-target refinement with short-horizon first-order policy-gradient updates in differentiable simulation; learns simulated Go2/G1 skills, and Go2 specialists are distilled into one depth policy that transfers zero-shot to a real Go2. | Yale University; University of Sydney | Sim-to-real | ✓ checked |
| 2026-09-17 | StageGuard: Learning Stage Transitions for Long-Horizon Robot Tasks via Agentic DistillationA teacher VLM (Qwen3.5-397B) plus demos generates subtask-completion rationales, distilled by SFT into a 0.8B student VLM that decides stage transitions. Transition completion 96.23% (LIBERO), 90.18% (BEHAVIOR-1K); real arms 18/20 and 16/20 closed-loop trials, no real-robot baseline. | Huawei; UBC; University of Toronto; McGill | Rewards & progress | ✓ checked |
| 2026-09-17 | WM-VS: Progress-Aligned World Models for Closed-Loop Visual ServoingWorld model aligned to DINOv2 target-centric servo error; reactive policy trained with imitation, consequence supervision and imagined rollouts favoring error contraction; real 7-DoF arm reaches 10% error region in 30/30 trials, retains it in 25/30 (26.67% without future-error alignment). | SJTU; China University of Mining and Technology; Shanghai Innovation Institute | World models & eval | ✓ checked |
| 2026-09-17 | REFINEPPO: Learning Continuous Control Policies by Iterative Action RefinementPPO whose actor builds each action by iterative residual refinement: a shared network repeatedly corrects an initial proposal given state and current action. Matches or exceeds PPO on 14 Gymnasium control tasks, converging faster on several. | Northeastern University | General RL | ✓ checked |
| 2026-09-17 | Visual Sim-to-Real Learning for Robotic Insertion under Geometric Variations: Application to Rebar InstallationRebarSim: privileged state-based RL teacher on procedural rebar geometries, DAgger-distilled into a multi-view RGB student; zero-shot 91.3% (137/150) real seating at 1.4 mm clearance; adapting to a new design takes 4-6x fewer distillation samples than training from scratch. · code | University of Washington; McGill; Princeton | Sim-to-real | ✓ checked |
| 2026-09-17 | Hybrid Residual Reinforcement Learning for Contact-Rich Robotic Book InsertionResidual PPO adds bounded local corrections and release timing to a nominal task-space controller; 98.5% sim success over 512 conditions, real xArm7 26.7% to 63.3%. | Deakin University | Residual & steering | ✓ checked |
| 2026-09-17 | Catch Me If You Can: Real-Time Feedback Denoising for Responsive VLAsVLA-Feedback: low-frequency diffusion planning with the final denoising step kept as a high-frequency visual feedback interface; vs GR00T (same planner), dynamic sim tasks 27.5%→85.0%, real robot (3 tasks) 51%→73% (CoRL 2026). · code | UT Austin; University of Central Florida | Test-time search | ✓ checked |
| 2026-09-17 | Diverse and Adaptable Arm Coordination for Octopus-Crawling via Diffusion-Based Uncertainty-Aware OptimizationDiffusion-based optimization discovers and preserves diverse crawling coordination strategies for a simulated muscle-actuated soft multi-arm robot without demonstrations, adapting to new constraints without full retraining. | University of Illinois Urbana-Champaign | Black-box search | ✓ checked |
| 2026-09-17 | WorldContact: A Contact-Centric World Model for Scalable Robot LearningContact-centric world model for deformable bag manipulation using larger time steps than its source simulator (10x faster state rollouts, excluding rendering); real bag lifting: 13/20 for a VLA fine-tuned on source-sim data alone vs 19/20 with WorldContact-expanded data. | UBC; Westlake; Style3D Research | World models & eval | ✓ checked |
| 2026-09-17 | DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot ManipulationAction-conditioned visuo-tactile world model trained on 5h robot plus up to 100h human touch data; used as surrogate for policy evaluation and synthetic trajectory generation. | HKUST Guangzhou; Xspark AI; Peking University; Tsinghua; HKU | World models & eval | ✓ checked |
| 2026-09-17 | MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-ExecutionTurns failed long-horizon manipulation rollouts into recovery supervision: a coach proposes corrections, kept only if re-execution from that point improves outcomes; beats distillation and trajectory repair (CoRL 2026). · code | Sileane; LAAS-CNRS | Human corrections | ✓ checked |
| 2026-09-17 | EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied IntelligenceRL post-training of an embodied VLM (Qwen3-VL-8B) that outputs text plans and answers, not motor actions: rejection-sampling SFT, difficulty-queued IR-GRPO, then Trie-GRPO with prefix-tree step-level advantages. 70.02% average over 18 multimodal, spatial and planning benchmarks. | ZTE | General RL | ✓ checked |
| 2026-09-17 | OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free TeacherRL-trained privileged teacher on vectorized inputs supervises closed-loop fine-tuning of camera driving policies in AlpaSim (3DGS); scene scores rise 1.6x (TransFuser) and 9.5x (VaVAM); in a separate vectorized WOMD study it matches RL fine-tuning with ~1000x fewer episodes. | KE:SAI; ETH Zürich; NVIDIA; Stanford; ELLIS Tübingen | General RL | ✓ checked |
| 2026-09-17 | GR2PO: Group Relative Return Policy Optimization for Continuous Robot ControlCritic-free GRPO-style RL for continuous control: discounted returns from parallel trajectories, group-normalized per time index, clipped updates. MuJoCo Ant/Walker2d/Humanoid: beats immediate-reward GRPO, near PPO/SAC (trails PPO on Humanoid); Jetson TX2 test is inference-only. | HKUST; HKUST Guangzhou | General RL | ✓ checked |
| 2026-09-17 | AnyviewMeter: Adapting Robotic Reward Models with Camera Geometry and Multi-View AttentionLoRA-adapts a pretrained Robometer progress reward model with Plucker-ray camera conditioning and synchronized multi-view block attention; in sim, 41-69% lower progress error than averaging single-view RGB predictions; ~21% lower MAE on real tasks. | NTU | Rewards & progress | ✓ checked |
| 2026-09-17 | TraceFlow: Guiding Frozen Flow-Matching Robot Policies with Success and Failure TracesKDE scores from retrieved success and failure traces guide a frozen pi0.5, with no critic. Real ordered fruits TSR 21/50 → 39/50, then 47/50 after one more round. · code | HKU; SUSTech | Test-time search | ✓ checked |
| 2026-09-17 | DEXTERA: From a Single Image to Deployable Dexterous Manipulation via Real-to-Sim-to-RealSingle-image real-to-sim-to-real pipeline for dexterous hands (Gaussian background, VLM-inferred physics, VR teleop, trajectory synthesis); on six task pairs, 1:1 sim-real co-training lifts mean real success of ACT, DP and pi0.5 from 29.2% (sim-only) to 61.9%. | UT Austin, University of Florida, BrainCo | Sim-to-real | ✓ checked |
| 2026-09-17 | Towards High-DoF Dexterous Manipulation through VLA Post-TrainingSFT, then human-gated DAgger, then a latent residual TD3 on a frozen pi0.5 for two 20-DoF hands. All 5 tasks reach 20/20. Transfer goes 10% → 65% → 70% → 100% with RL, using 3,000 online transitions per task. | Wuji Technology; ShanghaiTech | Residual & steering | ✓ checked |
| 2026-09-17 | HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation InterfaceQueries the current pi0.5 policy in shadow mode during handheld UMI demos; human-policy discrepancy triggers collection, then advantage-conditioned BC. Real Franka Stack Cube task-progress score (0-100, 10 trials) rises 84 to 100 over 3 rounds. | Xi'an Jiaotong; Peking University; PrimeBot; JD Technology | Human corrections | ✓ checked |
| 2026-09-16 | Rethinking Critic Learning in PPO: Understanding and Mitigating Value FlatteningIdentifies Value Flattening in PPO critics (flat predictions vs sharply varying Monte Carlo values); SP3O supervises only three well-separated states per response, improving LLM RL on Qwen3-Base. | SJTU; Shanghai AI Lab; Westlake; Nanjing; Tsinghua; CUHK; NTU | General RL | ✓ checked |
| 2026-09-16 | ForceDelta-VLA: Distilling Force-Conditioned Action Corrections for Contact-Rich ManipulationDistills force-conditioned corrections to a base VLA action; 82.2% versus 54.4% ForceVLA success over nine contact-rich tasks, with approximately 26% lower mean peak force over successful trials. | University of Hamburg; TU Munich; USTC | Residual & steering | ✓ checked |
| 2026-09-16 | Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action ModelsShift diagnostics choose variable-rank LoRA adapters for VLA components; real xArm-7 instruction-shift trials yield 21/30 successes, compared with 22/30 after full fine-tuning, while updating only 0.04% as many parameters. | Carnegie Mellon University | General RL | ✓ corrected |
| 2026-09-16 | Function-Preserving Data Generation for Zero-Shot Real-to-Sim-to-Real ManipulationGenerates synthetic demos from reconstructed assets without teleoperated source trajectories, using constraint-guided mesh deformation that preserves contact geometry; zero-shot deployment on contact-rich long-horizon tasks, no real fine-tuning. · code | HKUST Guangzhou; HKU | Sim-to-real | ✓ checked |
| 2026-09-16 | Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement LearningStudies when continual model-based RL should drop pre-shift replay: forgetting helps after large permanent dynamics shifts, hurts when dynamics recur; estimates staleness from interaction data across locomotion morphologies. | Brown University | World models & eval | ✓ checked |
| 2026-09-16 | RAFAIL: Relationship-Aware Failure Detection for Robotic ManipulationRuntime failure detection via point-cloud anomaly detectors on task-relevant entity relationships, labeled offline by a VLM from successful demos only; 73.4% balanced accuracy on three real tasks. | KIT | Test-time search | ✓ checked |
| 2026-09-16 | Causal-History Test-Time Scaling for Failure Recovery in Autoregressive World-Action ModelsTraining-free failure recovery for autoregressive world-action models: progress-aware trigger detects stalls, rolls causal KV history back to a reliable prefix, then verifies three recovery hypotheses before committing. | HKUST; ACCESS; Zhejiang University | Test-time search | ✓ checked |
| 2026-09-16 | FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure FeedbackA progress plus failure-risk evaluator shapes SAC for a distilled specialist; real mean 92.0% vs 83.0% with Robometer-guided RL after 300 rollouts (single seed). · code | NTU, Guangzhou Cloudbutterfly | Rewards & progress | ✓ checked |
| 2026-09-16 | DistAL: Distance-based Advantage Learning for VLA Fine-TuningDense reward inside RECAP: negative kNN distance to the base VLA's training embeddings. Hardware mean 44.0 base / 60.0 binary reward / 69.5% DistAL. | University of Oxford | Advantage-weighted BC | ✓ checked |
| 2026-09-16 | ★ Reinforcement Learning for Real-Time Vision-Language-Action Policies (Real-Time EXPO-FT)Slow pi0.5 proposes 32 chunks from a stale observation; a fast policy adds bounded edits from the fresh one; a Q-ensemble picks one. Real, 4 dynamic tasks, 10 min online data: 12.5/30 → 29/30 (EXPO-FT w/ RTC 25/30). · code | Stanford University | Residual & steering | ✓ checked |
| 2026-09-15 | The Robot Data FactoryInfrastructure for a continuous Deploy-Measure-Learn-Repeat robot-experience loop, with a mission-to-capability hierarchy, fleet/sensor/compute scaling laws, and three physical training grounds. | Haddadin, Laptev, Reid et al. | Industry loops | ✓ checked |
| 2026-09-15 | XPACE: Joint World and Action Modeling from Heterogeneous ExperienceXPACE shares a video backbone between a world-action policy and simulator. Heterogeneous human/robot training improves IRON robustness; fine-tuning on simulator-generated recovery trajectories further improves real-world task completion. | XPENG | World models & eval | ✓ checked |
| 2026-09-15 | Residual Fault Adaptation for Dexterous In-Hand Manipulation Under Runtime Joint FaultsFrozen healthy teacher plus recurrent residual policy, trained with fault-injection randomization and adaptive fault sampling, compensates hidden single-joint command faults without fault labels; beats the healthy policy in sim and deploys zero-shot on a real LEAP Hand with software-injected faults. | HKUST; Huazhong University of Science and Technology | Residual & steering | ✓ checked |
| 2026-09-15 | Intrinsic Robot Rewarding: Reusing VLA Representations for Autonomous Evaluation and Policy ImprovementPosition paper: reuse the VLA's frozen visual encoder plus a reference bank of successful demonstration endpoints as a built-in reward/evaluator, avoiding a separate reward model; COMAU Racer 3 demonstrator at TRL 4, no policy-improvement results yet. | Deggendorf Institute of Technology | Rewards & progress | ✓ checked |
| 2026-09-14 | Learning Multimodal One-step Flow Policy via Value-weighted Optimal TransportOffline RL: state-wise entropic optimal transport couples a value-aware reference flow policy to a one-step policy, prioritizing high-value distillation targets without direct critic maximization; best state-based OGBench average (48.3 vs 45.6 for GFP). · code | Yonsei University | General RL | ✓ checked |
| 2026-09-14 | Refinement-Based Flow Policy OptimizationOnline RL for flow policies: refines the policy's own samples toward a Q-induced energy-based distribution, then flow-matches to them. Matches or beats SAC on 5 of 6 MuJoCo tasks (not Hopper); fits six synthetic 2D multimodal targets without mode collapse. | KAIST | Off-policy critics | ✓ checked |
| 2026-09-14 | Steering Generative Robot Policies with Lexicographic PreferencesFrozen-policy dynamic-barrier guidance protects higher-priority costs to first order, followed by lexicographic sample selection. Sim navigation, 60 episodes/repeat: mean successes 44.3 → 56.0, versus selection-only 48.0 and weighted-sum 49.7-51.0 (three repeats). | MIT AeroAstro | Test-time search | ✓ corrected |
| 2026-09-14 | Real-World Reinforcement Learning with MPC Scaffolding for Dexterous ManipulationBC/Soft-SARSA pretraining and MPC-guided SAC use 20 MPC trajectories (12 hardware minutes). Real Allegro rotation: first 5/5 after 7.4 online minutes; separate evaluation 20/20 at 55.3 deg/s versus MPC 20/20 at 9.9 deg/s. | Honda Research Institute USA | Off-policy critics | ✓ corrected |
| 2026-09-13 | VGFM: Expressive Robot Policies via Dense Value Guidance in Flow MatchingOffline RL with dense critic guidance at randomly sampled flow timesteps, without backpropagation through the flow trajectory. Evaluates 40 simulated OGBench tasks and changes inference integration steps without retraining (IROS 2026). · code | Cornell University | Off-policy critics | ✓ corrected |
| 2026-09-13 | REVOLVE: An Automated Closed-Loop Framework for Evolving Robot Manipulation with Minimal Human InterventionVLM failure monitoring, deterministic recovery and automatic resets reduce human intervention during iterative learning. On four real ViperX tasks, average success rises from 40.0% to 58.5% over five iterations; human help remains available. | Jiangnan University | Human corrections | ✓ checked |
| 2026-09-12 | ReWeight: Leveraging Human Data for VLA Post-Training via Demonstration Retrieval and Sample WeightingOptimal-transport retrieval of egocentric human demos plus sample weighting for pi0.5 post-training; RoboTwin simulation success 39% robot-only to 57%, real-world 40.0% to 68.8%. Weighted BC, without RL. | HKU; TeleAI, China Telecom; Northwestern Polytechnical University | Advantage-weighted BC | ✓ checked |
| 2026-09-10 | FARM: Reading Failure Signals from the Internal Predictive States of a Frozen Robotic World ModelA 33,985-parameter readout detects failures from frozen VLA-JEPA states: 85.68/88.59 pooled AUROC/AUPRC across seven tasks. Real-robot transfer varies and is tested with adaptation; 0.23ms timing covers only the readout. | Institute of Automation CAS; HIT | Test-time search | ✓ checked |
| 2026-09-10 | Smooth Exponentials for RoboticsEssay arguing robotics has entered smooth exponential progress via many players and standardized hardware-agnostic pretraining data; predicts the first 2-3 general-purpose home robots on the market in October 2027. No RL fine-tuning content. | Eric Jang | Industry loops | ✓ checked |
| 2026-09-10 | Pelican-Sim 1.0: A General World Model Simulator for Embodied IntelligenceA world-model simulator uses a unified 28-dimensional action space. In RoboTwin simulation, adding 500 generated trajectories to 50 demos/task raises policy success from 70% to 93%; evaluation correlation is 0.994 across five checkpoints. · code | Beijing Innovation Center of Humanoid Robotics (X-Humanoid) | World models & eval | ✓ checked |
| 2026-09-09 | When Validation Stops Learning: Auditing Update Admission for Continual Embodied AgentsAudits policy-update admission: paired-binomial checks admit 31.6% of updates at 2,000 episodes/stage versus zero for a range-based gate in synthetic pushing. Unconditional replay learns better closed-loop; evidence remains analytical and simulated. | Rice University; UC Berkeley | General RL | ✓ checked |
| 2026-09-09 | ★ RoboDrop: Curating VLA Post-Training Data via Local Gradient CompatibilityIn one warm-up epoch, compares each sample's compressed gradient with one from the most similar clean validation frames, and drops conflicting episodes before re-post-training pi0.5. Real, 4 tasks, 20 rollouts each: 35.0% → 67.5% with automatic filtering. | Tsinghua University; Striding AI | Advantage-weighted BC | ✓ checked |
| 2026-09-09 | HaWMPO: Hallucination-Aware World Model-based Policy Optimization for Generalist Robot PolicyAction-conditioned model scores hallucination per imagined chunk (AUROC 0.9375 on 50 human-labeled LIBERO chunks); scores down-weight chunk rewards in GRPO. LIBERO Spatial/Goal/Object average 63.7 vs 60.9 for reproduced WoVR* (no PACE). | JD Joy Future Academy; Jilin University | World models & eval | ✓ checked |
| 2026-09-08 | WorldAgen: Unified State-Action Prediction with Test-Time World Model TrainingShared transformer with world-model and action heads (mixed unidirectional attention mask); at deployment samples exploratory actions and test-time-trains the world model on collected transitions; CALVIN, LIBERO gains. | Wan et al. | Test-time search | ✓ checked |
| 2026-09-08 | ★ A Decade of Bayesian Optimization for Controller Tuning and Robot Learning: Tutorial, Review, and Future ProspectsSurvey framing controller tuning as episodic policy search where a GP surrogate picks each trial; reviews 110 hardware papers and releases TuneControl. 57 of 110 use vanilla BO; sim 4-parameter cart-pole: LogEI 4310±48 vs Sobol 4647±190 (median±IQR cost, lower better). · code | RWTH Aachen, University of Technology Nuremberg, TU Munich | Black-box search | ✓ checked |
| 2026-09-08 | No Free Checker: A Survey of Verifiers for Robot PoliciesA survey of roughly 150 robot verifiers grouped by who supplies the judgment; finds credibility falls as availability rises, and argues robotics has yet to search a learned verifier for inputs it wrongly accepts. · code | Zhejiang University, CityU Hong Kong | Rewards & progress | ✓ checked |
| 2026-09-08 | Proxy Policy Steering (PPS)Nudges the base velocity field by gamma*(v_task - v_ref) from two small proxy models, with no RL. Eight real tasks: pi0.5 22% → 79%, versus LoRA 59%, residual 19%, DSRL 25% (chart reading). · code | Cornell | Residual & steering | ✓ corrected |
| 2026-09-07 | Phase-and-First-Arrival VLM Feedback for Sparse-Reward Reinforcement Learning in Surgical ManipulationVLM feedback returns the furthest visually verified task phase and its first-arrival time so sparse-reward RL reuses partial progress from failures; 75.2% vs 52.1% for CLIP rewards in simulation. | Hong Kong Polytechnic University; CUHK; MBZUAI; Great Bay University | Rewards & progress | ✓ corrected |
| 2026-09-07 | Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement LearningDefines a Q-function over diffusion latents and a noisy-space policy gradient that avoids backprop through denoising; improves offline RL on state-based D4RL and vision-based OGBench (ICML 2026). · code | TRATON CV AB; KTH Royal Institute of Technology | Off-policy critics | ✓ checked |
| 2026-09-06 | VLA-Corrector: Stage-Aware Observable State Understanding for Prompt-Based Closed-Loop Recovery of Vision-Language-Action PoliciesLearned verifier estimates stage-wise progress and risk from observations, proprioception and actions, triggering stage-conditioned recovery prompts for a frozen VLA; improves LIBERO/LIBERO-Plus robustness without parameter updates. | Song et al. | Test-time search | ✓ checked |
| 2026-09-05 | ★ Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Models (DLS)Finds pi0/pi0.5 failures in parallel digital twins, turns hand-defined task phases into a dense potential, and applies a critic-free contrastive velocity loss plus a real-demo SFT anchor. pi0.5, 3 real tasks, ID/Unseen: 84.4/72.2 vs PPO 81.1/67.8, 100-demo SFT 78.9/54.4. | Show Lab, National University of Singapore | Sim-to-real | ✓ checked |
| 2026-09-05 | How to Learn from What a Human Would Avoid? Intervention-Aware World Models with Real-World RL for Dexterous ManipulationReal-world HIL residual SAC over a frozen BC prior; an intervention-aware latent world model predicts operator takeovers and shapes the actor away from them. Pick & Place: BC 43.3%, model-free HIL residual 60.0%, WHIRL 90.0% (30 trials). | HKUST (Guangzhou); Italian Institute of Technology; Zhejiang University | World models & eval | ✓ checked |
| 2026-09-04 | TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich ManipulationWorld-action model caches predicted contact; a tactile expert corrects chunk actions during execution, 20.7x cheaper than regenerating; real success 22% to 64% across five tasks. · code | Fudan University; Shanghai Innovation Institute; NeoteAI | Residual & steering | ✓ checked |
| 2026-09-03 | Continual Field-Adaptive Models (CFAMs) for Post-Deployment Physical AIFrozen slow model plus on-device Capsule Field memory adds verified near-OOD successes without gradients. Matches an in-house full-data policy using 40% of its data; memory growth raises mean action success from 74.0% to 87.9%. | Skylark Labs; Carnegie Mellon; UC Berkeley | Test-time search | ✓ checked |
| 2026-09-03 | FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action ModelsTimestamp-level VLA failure detection learns from weak labels on anomalous action chunks, then actively selects uncertain trajectories for annotation. Evaluated across VLAs with a fixed 15% timestamp-annotation budget. | Ma et al. | Test-time search | ✓ checked |
| 2026-09-03 | R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language ModelsCalibrated Isaac twin plus pairwise VLM preferences aggregated with Bradley-Terry. Mean absolute real-sim success-rate gap: 2.13 percentage points, with identical aggregate ranking across six VLAs. | Sharpa; Nanjing U; Tongji; Tsinghua | World models & eval | ✓ corrected |
| 2026-09-03 | WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action ModelsSchedules bounded world-model rollouts at interaction-relevant states to refine candidate actions. Three simulated tasks: 2.61 versus 11.45 GPU-hours for full imagination (77% less). Four real tasks, standard setting: pi0 success 60.0% → 77.5%. | Tsinghua; BAAI | World models & eval | ✓ corrected |
| 2026-09-03 | ★ VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action ModelsPost-trains pi0.5's action-expert LoRA with flow-matching BC on demos, successes and human corrections, plus ensemble-critic advantages relative to a frozen reference instead of absolute-Q maximization. 9 real chemistry-lab tasks: 98.3% vs demo-only pi0.5 67.8%, ConRFT (Octo) 7.8%. · code | University of Science and Technology of China; Beihang University; Zhongguancun Academy; Hefei SpinX Technology | Off-policy critics | ✓ checked |
| 2026-09-02 | Harnessing human expertise for high-precision robotic assembly in industrialized construction: A sample-efficient installer-in-the-loop interactive reinforcement learning frameworkInstaller-in-the-loop RL (Q-chunking with Flow Q-Learning, teleop demos plus binary takeovers) for window-unit seating, in MuJoCo sim: at 2 mm per-side clearance, 100% autonomous seating after 3.0 h online training with 12-15 min supervision. | McGill University | Human corrections | ✓ checked |
| 2026-09-02 | Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RLDCRL: recursively splits trajectory segments into a balanced binary tree, cutting worst-case bootstrap depth from linear to logarithmic; on the five hardest long-horizon OGBench tasks, raises the best prior average score from 55 to 64. | Yonsei University; Seoul National University | General RL | ✓ checked |
| 2026-09-02 | Do Better Imagined Rollouts Mean Better Robot Control? A Controlled Study of World-Model Evaluation Under FeedbackSix state estimators on simulated differential-drive path tracking, 24 sensing conditions: replay position RMSE predicts closed-loop cross-track error (Spearman 0.923) better than open-loop rollout error (0.774). · code | Georgia Tech; Emory University | World models & eval | ✓ corrected |
| 2026-09-02 | World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action ModelsSamples futures from a frozen World Action Model, ranks video surprisal and action-path effort, and learns an execution-mismatch predictor. RoboTwin 2.0 Hard: 55.80% base WAM to 60.90% with limited randomized-scene supervision. | Institute of Science Tokyo; DENSO IT Lab; National Institute of Informatics | Test-time search | ✓ checked |
| 2026-09-01 | Knowing When to Stop: Adaptive Action Chunking via Internal Cross-Attention Dynamics in VLAsTraining-free adaptive execution horizon for VLAs: detects high-entropy plateaus in action-to-observation cross-attention to decide when to replan; improves success on RoboTwin 2.0 and LIBERO. | Tsinghua University | Test-time search | ✓ checked |
| 2026-09-01 | Facet-0: A Robotic Foundation Model for Contact-Rich Precise ManipulationA wrench-predicting VLA uses contact-selective credit tags and bounded TD3+BC adaptation. Five real precision-assembly tasks: 16% alignment-only, 38% value-guided refinement, 82% full system; authors’ matched pi0.5+RECAP-style implementation scores 15%. · code | NTU PINE Lab | Advantage-weighted BC | ✓ checked |
| 2026-09 | Towards Universal Post-Training for Robotics (blog)Position essay for an EXPO-FT-style standard robot post-training recipe. It cites EXPO-FT reaching 30/30 with an average of about 19 min of online interaction. The baseline averages cover only the 4 tasks every baseline ran. · code | Stanford | Residual & steering | ✓ checked |
| 2026-08-31 | PAVE: Predictive Alignment and Value-Guided Evolution for World-Action PoliciesCombines multi-horizon future-latent prediction with value-derived quality conditions for a flow actor. Matched-data comparisons improve success by 5.0 points on LIBERO-Plus and 4.5 points on RoboTwin Random (simulation). | Zhao et al. | Advantage-weighted BC | ✓ checked |
| 2026-08-31 | Motus2: A Self-Evolving General World Model for Dexterous ManipulationSingle shared-weight model serves as policy, simulator and evaluator in a closed loop; curated expert demos train actions while failed and suboptimal interactions train dynamics and value; tactile sensing on a dexterous bimanual platform. | Bi et al. | World models & eval | ✓ checked |
| 2026-08-30 | SmoothRL: Online Reinforcement Learning During Asynchronous ExecutionBounded residual TD3 actor on a frozen pi0.5 trained under the real asynchronous execution loop; tossing 39% → 94%, pen capping 8% → 83% over ~250 episodes. | Astribot | Off-policy critics | ✓ checked |
| 2026-08-29 | DREAM: Deployment-Time Demonstration Generation via Real-to-Sim for Scalable Policy AdaptationReconstructs the deployment workspace in sim, translates instructions to symbolic goals via LLM, and uses TAMP to generate verified, randomized demos for VLA fine-tuning without human teleoperation. | Sato et al. | Sim-to-real | ✓ checked |
| 2026-08-28 | In-Context Learning Results Hint at a "GPT Moment" for RoboticsChris Paxton reviews video-prompted in-context learning (Skild S1, GEN-1.5, Rhoda, Lingbot VA-2) as an alternative to retraining; notes wholly new skills at high success still need more data or on-robot RL. | Chris Paxton | Industry loops | ✓ checked |
| 2026-08-27 | FLARE: A Failure-Aware Framework for Autonomous Correction and Recovery in Visual-Language Robotic ManipulationRetry via perturbation and bridging-segment augmented demos, plus Reset skills learned from offline MLLM failure-video analysis and gated by an online MLLM monitor, improves VLA robustness on contact-rich tasks. | Zhao et al. | Test-time search | ✓ checked |
| 2026-08-27 | Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPOAnalyzes ES post-training for LLM reasoning: unlike GRPO (entropy collapse), ES improves Pass@1 and reaches higher Pass@K than GRPO; gains concentrate in a sparse subset of larger-magnitude parameter updates; tests sequential GRPO/ES combinations (adjacent, LLMs). | Ba et al. | General RL | ✓ checked |
| 2026-08-27 | CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical SimulatorsCross-embodiment action-conditioned video world model: curriculum from latent actions on unlabeled video to end-effector/language actions; approaches or surpasses single-embodiment models on DROID, spans Bridge, YAM, G1. · code | Princeton University | World models & eval | ✓ checked |
| 2026-08-27 | GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot ManipulationConRFT-style online VLA RL with training-only visual anchors. Across four real lab tasks, final ten training episodes per task yield 33/40 successes (82.5%), versus proposal-union supervision 23/40 (57.5%); these are online training endpoints. | USTC | Off-policy critics | ✓ corrected |
| 2026-08-26 | R^3: Training Robots to Reason in Natural Language via Reinforcement LearningPost-trains a VLM as a robotic reasoner: mid-training on expert reasoning traces, then rubric-based RL on offline action data; language reasoning steers low-level policies at test time. | Wu et al. | Test-time search | ✓ checked |
| 2026-08-26 | RA-VLA: Retrieval-Augmented VLA for Test-Time AdaptationRA-VLA trains behavior-aligned demonstration retrieval and context-following execution on training tasks. It then adapts to held-out tasks through retrieved demonstrations without further weight updates; evaluated on LIBERO and a real UR5e. | POSTECH | Test-time search | ✓ checked |
| 2026-08-26 | Beyond Pairwise Feedback: Listwise Vision-Language Supervision for Preference-Based Reward LearningA VLM ranks K=3-5 replay-buffer frames at once; a Plackett-Luce listwise loss trains the reward model. Competitive with pairwise Bradley-Terry and RL-VLM-F on three Meta-World tasks (Drawer Open: PL 86% vs RL-VLM-F 92%). | Katkuri et al. | Rewards & progress | ✓ checked |
| 2026-08-25 | RoG-DAgger: Rollout-Guided Post-Training for End-to-End DrivingDAgger uses short-horizon kinematic rollouts to generate driving corrections and time takeovers. In CARLA, improves SimLingo by 5.3 driving-score points on Bench2Drive and raises Fail2Drive OOD success from 55% to 66%. | Zhong et al. | General RL | ✓ checked |
| 2026-08-24 | Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency (ARLI: Asynchronous RL with Intermediate Information)DSRL under asynchronous VLA inference. It feeds the noise policy the committed actions plus a fresher mid-inference frame. Real success goes from around 40% to near 100% in 100-125 episodes. | Siemens; UC Berkeley; Microsoft; ETH | Residual & steering | ✓ checked |
| 2026-08-23 | WAM-OPD: On-Policy Distillation for World Action ModelsA frozen teacher relabels an accelerated WAM student’s own rollout histories. Preliminary RoboTwin 2.0 simulation, 12 paired evaluations/task: Flash-WAM improves Handover Mic from 0/12 to 7/12 and Put Object Cabinet from 2/12 to 4/12. | University College London | On-policy PG | ✓ checked |
| 2026-08-22 | CounterAlign: Counterfactual Supervision for Vision-Language-Action ModelsPairs expert actions with mismatched instructions and trains an adversarial discriminator as a reward for offline-RL VLA post-training; improves LIBERO-PRO robustness and real-robot (TX-G2) results. | Kondoh et al. | Off-policy critics | ✓ checked |
| 2026-08-22 | CIDER: Continual Interactive Distillation for Embodied Reinforcement LearningRLPD plus reverse-KL distillation preserves a frozen teacher’s skills. After six sequential real tasks, CIDER reports 100/100 successful trials per task (Wilson 95%: 96.3–100%); plain fine-tuning averages 16.7% over those tasks. | AgiBot; SJTU | Off-policy critics | ✓ checked |
| 2026-08-21 | Decoupling Policy Extraction for Offline Reinforcement LearningTrains the actor purely as a behavior model and does policy improvement at inference by critic reranking of behavior-supported candidates; reported to beat BC and jointly trained offline RL. | Lin et al. | Off-policy critics | ✓ checked |
| 2026-08-21 | ★ Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-PlanningKeeps the policy frozen as a proposal, scores 32-64 candidate chunks with a separate ~1B Q-function, runs their softmax-Q-weighted average, and retrains only the critic. Real robot, 5 iterations: stack-cups 40% → 90% vs 55% for SFT on successful rollouts. · code | Georgia Institute of Technology | Test-time search | ✓ checked |
| 2026-08-20 | EXIMO: VLM Guided Exploration of VLA PoliciesVLM-guided subgoals collect exploration trajectories, successful episodes fine-tune the VLA, and residual off-policy RL further optimizes it; evaluated on 22 simulated ALOHA manipulation tasks. | Google DeepMind | Advantage-weighted BC | ✓ checked |
| 2026-08-19 | SCAPE: Scenario-Conditioned Simulation-Augmented Policy EvaluationPredicts scenario-level real-world policy performance from few paired sim/real samples plus many sim rollouts, with sim-bias correction and conformal calibration. Sim-to-sim: error down 4.9%/34.7% (driving), 14.5%/27.7% (quadruped) vs neural/aggregate baselines; also run on a real Go2. | Zhu et al. | Sim-to-real | ✓ checked |
| 2026-08-19 | Dream2Reward: Transition-Alignment Reward Models from Positive Demonstrations for Robotic ManipulationLearns reward from success-only demos by predicting expected transitions and scoring observed displacement on direction and magnitude agreement; penalizes wrong-direction and overshoot, reducing reward hacking in online/offline RL. | Zhang et al. | Rewards & progress | ✓ checked |
| 2026-08-19 | Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task ManipulationReplays frozen base-VLA rollouts with new expert demonstrations to prevent forgetting. Real ALOHA gear insertion: 30% with expert-only fine-tuning to 90% with self-demonstration replay. | UIUC | Advantage-weighted BC | ✓ checked |
| 2026-08-19 | ★ ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement LearningPost-trains a reposing-pretrained PPO arm-hand policy: BC-distill the actor into new observations, warm up a fresh critic with the actor frozen, then PPO at actor LR 1e-5. Sim peg insertion: 5/5 seeds reach ADR 50 vs 0/5 at LR 1e-3. | NVIDIA; University of Michigan | On-policy PG | ✓ checked |
| 2026-08-18 | Hydra-0: Action Flow for Generalist World Modeling and ControlGeneralist world model conditioned on pixel action flow across embodiments; 90.4% lower robot-motion error than action-conditioned baseline, r=0.96 replayed-vs-reference success correlation on RoboLab. | Li et al. | World models & eval | ✓ corrected |
| 2026-08-18 | Reuse Before You Retrieve: Diagnosing Headroom and Complementarity for Test-Time Augmentation of Embodied Multimodal PoliciesDiagnoses frozen VLAs via recoverable headroom (resampling/retry selection) versus retrieval complementarity (external demos); gains up to 21 points on LIBERO track measured headroom. | Jeong et al. | Test-time search | ✓ checked |
| 2026-08-18 | ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human FeedbackSupervised residual head that turns coarse human feedback ('too left') into joint corrections for a frozen ACT, with no RL. Sim induced-failure Cube Transfer 60.0% → 92.3%. | AIST Japan | Residual & steering | ✓ checked |
| 2026-08-18 | Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome GroupsAdds 0.2-weighted execution quality (contacts, jerk or VLM scores) to success rewards, using RLOO advantages. With non-tied quality scores, binary GRPO theoretically needs ~1.76x more rollouts per informative group at G=8, p=0.1/0.9. | Purdue, AWS AI | On-policy PG | ✓ corrected |
| 2026-08-17 | Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical IntelligenceHarness evolves runtime critics and recovery skills around frozen VLAs. Across 40 LIBERO-Pro Goal/Long task-settings, pi0.5 success rises from 32.0% to 71.1%; on 18 RoboCasa tasks, GR00T rises from 73.6% to 93.6%. | Ding et al. | Test-time search | ✓ checked |
| 2026-08-17 | Le Critique: Privileged Value Functions for LLM Reinforcement LearningPrivileged value functions give critics information hidden from the policy; TETHER mixes value and group-relative baselines. Both improve over standard value baselines and are competitive with or outperform mean-baseline GRPO on tested reasoning tasks. | Venkatraman et al. | General RL | ✓ checked |
| 2026-08-17 | HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RLInverts demos into flow noise, keeps K=8 DCT modes, BC-initializes, then runs SAC on a frozen humanoid VLA. Toy Storage OOD with pi0.5: 2/10 → 8/10, while DSRL training failed. | Peking University; X-Humanoid; Nankai; Xi'an Jiaotong | Residual & steering | ✓ checked |
| 2026-08-17 | τ0-VLA (tau0-VLA): a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time ComputationConfidence-gated beam search proposes subtasks, ranked with world-model and value predictions. Real Book Organization: 6/10 with the same hierarchy’s Plan Once baseline versus 9/10 with test-time search. · code | Shanghai Innovation Institute; Agibot; CUHK | Test-time search | ✓ checked |
| 2026-08-17 | Q-Learning With World Models (QWM)World-model tree search selects actions during online RL and evaluation atop EXPO/RLPD. Policy and Q-functions train only on collected environment transitions; simulated Robomimic and LIBERO experiments report improved sample efficiency from predictive search. | Stanford; Peking University | Off-policy critics | ✓ corrected |
| 2026-08-16 | ViTaR: Visuo-Tactile Residual Adaptation for Foundation VLA ManipulationEffect-Guided Modeling uses tactile feedback to select bounded residual corrections atop frozen OpenVLA-OFT. Seven simulated UniVTAC contact tasks: 61.3% mean success, 30.6 percentage points above the base. | Wang et al. | Residual & steering | ✓ checked |
| 2026-08-16 | ★ GAINS: Leveraging Inconsistent Human Intervention Signals in Reinforcement LearningLearns the return distribution with a truncated-quantile critic and acts on its lower tail, because takeover timing is inconsistent (over half of repeated marks differ by 10+ frames). Four sim and two real tasks: 22% higher success than RLIF. · code | Beijing Institute of Technology; Beijing Innovation Center of Humanoid Robotics; City University of Hong Kong; Nankai University | Human corrections | ✓ checked |
| 2026-08-16 | Robo-Dopamine 2.0: History-Conditioned and OOD-Aware Process Reward Modeling for Robotic ManipulationAdds rollout-history panels and OOD failure edits to the pairwise hop model; real repeated insertion 71/80 vs 48/80 for event-sparse reward. | Peking University, EvoPhys AI and others | Rewards & progress | ✓ checked |
| 2026-08-15 | Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning (PDF subtitle: "with Monte Carlo Q-Chunk Critics")A Monte Carlo chunk critic selects imitation targets from buffer or actor actions by Q. Real USB insertion: ACT QChunk-MCBC reaches 99% in 30 minutes; HIL-SERL peaks at 100% after 5 hours. | Wang et al. | Advantage-weighted BC | ✓ checked |
| 2026-08-15 | PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied ManipulationA phase-aware distributional critic gives cleaner RECAP labels. LIBERO-Long 73.8 (ReCAP) → 83.3 (PACE). | Dalian UT; Jilin University | Advantage-weighted BC | ✓ checked |
| 2026-08-15 | ★ StructRL: Structured Action-Space Exploration for Flow-Based VLAsKeeps the flow decoder deterministic and adds exploration noise only to the output action, AR(1)-correlated within a chunk with learned, group-bounded scales for position, rotation and gripper. ManiSkill pi0.5 OOD average: SFT 25.9, piRL 49.3, action-space Gaussian 65.9, StructRL 70.3. | Fudan University; Singapore Management University; Shanghai Artificial Intelligence Laboratory | On-policy PG | ✓ checked |
| 2026-08-14 | Ontology-Grounded World Models for Failure Diagnosis and Closed-Loop Repair in Physical AI SystemsOnto-EV-WM layers a task ontology and deterministic rules over the EV-WM world model to record unmet predicates and route verification-gated corrections. LIBERO-Goal replay windows: 86.95% to 94.05% after direct simulator-state corrections; PointMaze ties EV-WM at 94%. | Wang et al. | World models & eval | ✓ checked |
| 2026-08-14 | PRM-as-a-Judge 1.5: A Toolkit for Robot Process AssessmentEvaluation toolkit that turns rollout videos into dense progress curves with three metrics (failure-side progress, recovery, success-side execution quality); adds RoboPulse++ to validate process reward models. · code | Liu et al. | Rewards & progress | ✓ checked |
| 2026-08-14 | ★ Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models (CoRe)Flags out-of-distribution states via a Mahalanobis test on the action-expert latent, rewinds to a recovery anchor, imagines the frozen policy's moves, and plans minimal object restorations; training-free. Sim LangSwitch (ManiSkill) switch average: pi0 18.3 → 87.5, pi0.5 35.5 → 92.7. | Case Western Reserve University | Test-time search | ✓ corrected |
| 2026-08-13 | ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World ModelsAction-conditioned latent world model trained on unlabeled trajectories predicts multi-view embeddings under planned actions; a lightweight failure probe aborts failure-prone actions before contact, without modifying the policy. | Zheng et al. | World models & eval | ✓ checked |
| 2026-08-13 | FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous DrivingLow-reward, low-diversity groups trigger privileged-teacher self-distillation. Open-loop nuScenes, four-sample evaluation: mean L2 1.500m versus GRPO 1.848m (SFT 1.004m); single-sample improvement is inconclusive. | Dou | General RL | ✓ checked |
| 2026-08-13 | ★ Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement LearningA frozen VLM proposes checkable stages compiled into predicates; group advantages are computed per stage over rollouts that reached it, so good early actions escape blame for later failures. Sim RoboTwin 2.0, OpenVLA-OFT, macro-average: 75.8 vs SimpleVLA-RL 68.8 (SFT 38.3). | Institute of Software, CAS; UCAS | On-policy PG | ✓ checked |
| 2026-08-11 | Self-Evolving Embodied Agents via Skill-Harness EvolutionSHAPER: training-free self-improvement that keeps model weights frozen and evolves reusable skills plus a context-code harness; compared to fine-tuning and test-time scaling on VLABench and ESI-Bench. (adjacent) | Wang et al. | Test-time search | ✓ checked |
| 2026-08-11 | Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-TuningPretrains only a BC actor offline, then starts online fine-tuning with a fresh critic after a short warm-up on offline data; plugs into existing offline-to-online algorithms; gains on simulated OGBench manipulation, roughly on par on Robomimic. | Li et al. | Off-policy critics | ✓ checked |
| 2026-08-11 | ★ Adaptation of Generalist Robot Policies with Minimal Data (MiDAS: Minimal-Data Adaptation Strategy)LoRA-tunes pi0.5 on one demo, then freezes it and trains a sparse-reward off-policy edit actor-critic that outputs the full executed chunk, so it can leave the base's action support. LIBERO-Long, 1 demo: 91.2 vs DICE-RL 89.3, DSRL 33.5. | Carnegie Mellon University | Residual & steering | ✓ checked |
| 2026-08-10 | verdi: retrieval is not transfer for continual world model optimizationTreats optimization strategies retrieved for world models (Ctrl-World, Cosmos family, RoboCoin adaptation) as hypotheses validated on-target by a frozen verifier: 68% lower search cost vs cold start, negative transfer 0.34 (unconditional reuse) to 0.06. | Wu et al. | World models & eval | ✓ corrected |
| 2026-08-10 | RefineFly: Failure-Aware Post-Training for Aerial Vision-Language NavigationToken-level PPO post-training of a UAV VLA with dynamic failure memory, a two-stage scene curriculum and stage-specific reference regularization. TravelUAV (AirSim sim): +3.12 to 8.37 pp success over AerialVLA, with total rollouts about 30% of training-set size. | Wang et al. | On-policy PG | ✓ checked |
| 2026-08-10 | VANE: Reliable Test-Time Training for Vision-Language-Action Models via Future Visual Representation PredictionTest-time prompt adaptation for VLAs: candidate updates are isolated, checked against future visual observations, and committed only if supported; +3.2 points over baseline TTT on SimplerEnv WidowX. | Ji et al. | Test-time search | ✓ checked |
| 2026-08-10 | FACT: Failure-Aware Causal Training for World-Action ModelsWorld-action model that predicts future video and task progress conditioned on executed actions, so failure rollouts supervise action consequences. Adding failure data lifts real bimanual seen-task success 82% to 89%; RoboTwin sim 87.5% vs Motus 87.8%. · code | Peng et al. | World models & eval | ✓ checked |
| 2026-08-10 | Efficient Real-World Online Reinforcement Learning for Robot Manipulation via Centralized Training and Critic Decomposition (HARC: Hybrid Actors with Reward-decomposed Critic)Separate arm and gripper actors with one reward-decomposed central critic; real 3-task average 40% → 75% vs HIL-SERL. | IIT Genoa and others | Off-policy critics | ✓ checked |
| 2026-08-10 | SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement LearningRainbow DQN picks the playback speed of a frozen ACT's chunks, so it hill-climbs cycle time rather than success. Sim Cube Transfer runs 3.81x faster at 97.9% (ACT 100%). The arXiv date is 2026-08, but the venue is ICRA 2025. · code | Stanford | Residual & steering | ✓ checked |
| 2026-08-10 | ★ RynnValue: Scaling Robotic Value Foundation Models with Temporal DistancePredicts seconds until task completion from an instruction and a few frames, trained on over 7,000 hours of video with no preference labels. As reward for real online DSRL on frozen pi0.5: 72.5% vs Robometer 52.5%, sparse 48.8%. · code | DAMO Academy, Hupan Lab | Rewards & progress | ✓ checked |
| 2026-08-07 | AutoIntervene: Calibrated Intervention for Action-Chunking Imitation Learning PoliciesScores proposed action chunks by visual similarity and consistency against a memory of successful demos, with separate calibrated thresholds for policy-operator handoffs; corrections from successful rollouts adapt the policy. | Tang et al. | Human corrections | ✓ corrected |
| 2026-08-07 | TEMPO: Semantic-Action Decoupled RL Post-Training for Vision-Language-Action ModelsTwo TD3 loops at different speeds for a VLA's semantic projection and action expert; CALVIN SR5 81.7 vs FLOWER 77.8. · code | Zhejiang Gongshang University; KTH | Off-policy critics | ✓ checked |
| 2026-08-06 | Beyond Flat Policies: Hierarchical Post-Training for Embodied Agents in Robotic Manipulation (HiRoC)Hierarchical post-training: a Qwen2.5-VL-3B planner emits subgoals; an OpenVLA-OFT executor is first aligned to planner subgoals, then improved with online GRPO. LIBERO average 93.5% vs 91.0% for the strongest listed baseline. | Kong et al. | On-policy PG | ✓ checked |
| 2026-08-04 | EvoHIL: Self-Evolving Reward and Flow-Matched Policy Optimization for Robust Human-in-the-Loop Reinforcement LearningHIL-SERL-style RL with a self-evolving success classifier, flow-matching action-chunk actor, and offline fine-tuning on relit replay; real FR3 USB insertion at the 60% point of a composite lighting-shift index: 1.00 vs HIL-SERL 0.50. | Chongqing University | Off-policy critics | ✓ checked |
| 2026-08-03 | Towards the Harness of Embodied AgentsThea brings the coding-agent harness to robots: a persistent scene graph serves as context, and evaluations act as exit codes that detect completion, judge success and diagnose failures on long-horizon tasks. | Qi Wang, Wentao Zhu et al. | Test-time search | ✓ checked |
| 2026-08-03 | Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?Experiments attribute action-chunking gains to non-Markovian expressivity, reduced compounding error and implicit ensembling over delays. An explicit delayed-policy ensemble outperforms chunking in many tested settings. | Lazzati et al. | General RL | ✓ checked |
| 2026-08-03 | ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action PoliciesA 3.5M-parameter causal critic on DINOv3 for AWR reweighting and mistake detection; sandwich completion 70% to 85% (n=20, Fisher p=0.45, not significant). | Chef Robotics | Rewards & progress | ✓ checked |
| 2026-08-02 | You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy ImprovementLLM controller coordinates recoverable train-evaluate-improve runs. Bootstrap tests on one measured lineage give hardened gates 0.001 false promotions/run versus 0.005–0.021 for shipped gates; checkpoint selection loses to taking the last checkpoint. | Yu | Black-box search | ✓ corrected |
| 2026-08-02 | RL Bootstrapping of OpenVLA-OFT for a Novel Robot EmbodimentDemo-free RL in simulation aligns OpenVLA-OFT to a cable-driven parallel robot: PPO on directional primitives, then GRPO with expanded instructions. Held-out directional success 34.25% (after PPO) to 53.50% (after GRPO); move-to-object only 9.75%. | Nurtdinov et al. | On-policy PG | ✓ checked |
| 2026-08 | ★ Not Just a Model, But a Product (Dyna Robotics blog on scaling customer deployments)Fleet workflow: an auto-labeler splits each deployed episode into SOP steps with outcome and failure mode, humans review uncertain cases, and engineers query the fleet to retrain. Napkin folding: 35/hour, 75% passing (Dyna-1) → 95/hour, 93% (Dyna-2). | Dyna Robotics | Industry loops | ✓ checked |
| 2026-07-31 | BWM: A Low-Cost High-Fidelity World Simulator for Robot LearningAction-conditioned world model with initial-environment guidance and visual history supports imitation-data generation and closed-loop policy evaluation. Reports the highest combined score across WorldArena Track 1 and both Track 2 applications. · code | BWM Team | World models & eval | ✓ checked |
| 2026-07-31 | ★ CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-TuningTunes closed-weight Gemini Robotics On-Device via an SFT API only, tagging the top 30% of chunks by dense-reward advantage over similar-start chunks as 'advantage positive'. Real Unitree G1, 2 cycles of 100 rollouts: Plate Handover 53% → 96%. | UC Berkeley, Google DeepMind, NVIDIA Research | Advantage-weighted BC | ✓ checked |
| 2026-07-31 | WCM: A World Critic Model for Vision-Language-Action Reinforcement LearningA history-aware critic trained with latent next-state prediction; beats a single-frame VLM critic on all 7 real RECAP/AWR tasks (best 44/50) and reaches 99.0% ManiSkill IND with OpenVLA-OFT (PPO critic: 97.7%). · code | Tongji, Shanghai Innovation Institute, Fudan | Rewards & progress | ✓ checked |
| 2026-07-30 | Gemini Robotics 2 brings whole body intelligence to robotsWhole-body VLA plus ER 2 reasoning model and On-Device 2, which adapts to new bi-arm embodiments typically with under 200 examples; charted success ranges 32-92%, multi-finger dexterity still the hardest. No RL post-training described. | Google DeepMind | Industry loops | ✓ checked |
| 2026-07-30 | ★ RedFlow: Redirect Failure into Action-level Corrections for Flow-Matching VLA PoliciesClusters action chunks by estimated progress and joint state; within a cluster, positive-scored chunks (progress gain plus episode outcome) form correction targets for negative chunks, which are also suppressed. Real pi0 clothes folding: 36 → 67/100 (AWR 48). · code | HKUST | Advantage-weighted BC | ✓ checked |
| 2026-07-29 | RLMM-Flow: A Flow-based Mobile Manipulation Framework with Latent-Space Reinforcement LearningFrozen pretrained flow policy steered by an offline-RL latent-noise actor, with action-critic warm-up and coarse-to-fine latent-to-residual expansion; beats imitation-only and RL post-training baselines on success and collisions in simulated mobile manipulation, transferred zero-shot to real. | Wang et al. | Residual & steering | ✓ checked |
| 2026-07-29 | RL2-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action ModelsOffline-RL head on VLA latents is blended into the velocity to diversify candidates for verifier best-of-N, gated by a SAFE failure detector. PolaRiS pi0.5 OOD average 14.3 → 42.7 (rephrase 31.8). · code | NUS; University of Toronto | Test-time search | ✓ checked |
| 2026-07-29 | ★ Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning? (IPE: Initialization via Policy Ensemble)Shows demo-pretrained critics barely beat random ones, then rolls out an ensemble of diverse policies trained on the same data so the critic sees contrasting actions. Six Robomimic/OGBench sim tasks: reported 1.26x average fine-tuning improvement over naive Q pretraining. | Stanford University | Off-policy critics | ✓ checked |
| 2026-07-24 | ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot ManipulationAction-conditioned visuo-tactile world model trained on real plus simulated tactile data; predicts aligned vision and touch for policy evaluation and rollout-based data augmentation of tactile policies. | Huang et al. | World models & eval | ✓ corrected |
| 2026-07-22 | GRACE: Gradient-Free Robot Action Generation via Combined Diffusion-MPPI Posterior Mean EstimationSteers pretrained diffusion policies using MPPI forward cost evaluations, including non-differentiable constraints. Real 7-DoF arm: no collisions and 9/10 task successes; the unguided prior collides with the deployment-time obstacle in all 10 trials. | Park et al. | Test-time search | ✓ corrected |
| 2026-07-22 | ★ Closing the Lab-to-Store Gap: A Data-Efficient Post-Training and Experience-Driven Learning VLA Framework for Retail Humanoids (DEED)Data-efficient GR00T N1.6 post-training plus a RECAP-adapted refinement (text advantage prefix, VLM value function) for chip restocking on a Unitree G1, ~108 min of data. 50 real episodes each: SFT 32%, iteration 1 42% (CIs overlap), iteration 2 22%. | HIVE Robots; Technical University of Denmark | Advantage-weighted BC | ✓ checked |
| 2026-07-20 | RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon PlanningOrchestrates heterogeneous policies (VLA, RL, TAMP) as agentic skills, using execution memory and a Memory Bridge to route and hand off between them; evaluated on 500 tasks and 135 real-robot trials. | Huawei Noah's Ark Lab; UBC; University of Toronto; McGill | Test-time search | ✓ checked |
| 2026-07-20 | Test-Time Scaling for World Action Models via Zero-Shot Geometric EvaluationTraining-free best-of-N ranks world-action-model futures by cross-view depth-reprojection consistency from a frozen geometry model. At N=8, RoboCasa simulation group averages rise from 66.3% to 68.4% for Cosmos Policy and 80.8% to 82.5% for X-WAM. | Zhao et al. | Test-time search | ✓ checked |
| 2026-07-17 | Sunday Robotics ACT-2: What 778 of 785 Laundry Folds Actually ProveThird-party evidence audit of ACT-2's 778/785 (99.1%) laundry-fold result: one fixed checkpoint across unseen homes, no per-home adaptation, but first-party, task-specific, not independently reproduced. | Physical AI Guide | Industry loops | ✓ checked |
| 2026-07-17 | Data and Learning Where it Matters for Contact-Rich ManipulationMotion planning for free-space motion; autonomous dense data collection plus offline SAC-based RL only for the contact segment; real 4-task average 96% vs 55% for the strongest baseline. | TU Munich; Siemens | Off-policy critics | ✓ checked |
| 2026-07-17 | ★ ACT-2 Preview: Generalizing Reliability (Sunday Robotics blog)Pretrains ACT-2 on sensorized human data from Sunday's own capture hardware, then post-trains on repeated in-house Memo runs and recoveries (update rule undisclosed). Laundry: 99.1% over 785 autonomous attempts in unseen homes, one fixed checkpoint, zero per-home adaptation. | Sunday Robotics | Industry loops | ✓ checked |
| 2026-07-17 | ★ Foresight Residual RL for Long-Horizon Robot Manipulation with Vision-Language-Action ModelsTrains a PPO residual on a frozen per-phase pi0 for each subtask, multiplying each phase's reward by a learned predictor of next-phase success. Sim wrench-on-nut full task: 85.6±3.9% vs 54.5±3.6% with plain per-phase residuals. | Rutgers University | Residual & steering | ✓ checked |
| 2026-07-16 | RoboTTT: Context Scaling for Robot PoliciesFast-weight test-time training compresses history into weights, scaling visuomotor context to 8K timesteps. Real-robot average task-completion score: 79% versus single-step GR00T N1.7 at 42% and strongest baseline GDN at 56%. | Jiang et al. | Test-time search | ✓ checked |
| 2026-07-16 | DriftWorld: Fast World Modeling through DriftingAction-conditioned drifting world model generates future frames in one pass: 30+ fps, averaging 17x faster than diffusion baselines. Robomimic Lift policy evaluation: predicted versus simulator success rates have reported Pearson r²=0.9916. · code | Lu et al. | World models & eval | ✓ corrected |
| 2026-07-15 | Zero2Skill: Bootstrapping Robot Skills through Autonomous Data Collection, Training, and DeploymentLanguage corrections persist in a corrective memory for autonomous data collection. Matches teleoperation episode-collection success at 16% of human working time; separately, two corrections raise real single-attempt collection success from 12.5% to 47.5%. · code | GigaAI and collaborators | Human corrections | ✓ checked |
| 2026-07-15 | Never Too Late for Force: Accelerating VLA Post-Training with Reactive Force Injection (LIFT)Grafts force sensing onto pi0.5 during online DAgger. Real towel-folding task score peaks at 0.825 using 2.8K human-intervention frames, versus vision-only 0.725 at 3.1K frames. · code | Shanghai Jiao Tong University; Shanghai Innovation Institute; SUSTech; Noematrix | Human corrections | ✓ corrected |
| 2026-07-14 | Deconstructing Actor-Critic: A Large-scale Empirical Study of Design Components for Practitioners33,000+ experiments on a simulated PID-gain-tuning task built from real water-treatment-plant data dissect actor-critic design choices; Gaussian policies with pathwise gradients are among the least reliable, bounded distributions with adaptive update schedules stay robust. · code | Shah et al. | General RL | ✓ checked |
| 2026-07-14 | Deployable Human Preference Alignment in Robotics: Learning Representative Rewards from Diverse Human PreferencesPREC clusters users into preference-coherent groups from binary preference labels and learns one reward model per cluster, beating a single shared policy in simulated locomotion. | Kim et al. | Rewards & progress | ✓ checked |
| 2026-07-14 | DenseReward: Dense Reward Learning via Failure Synthesis for Robotic ManipulationFailures synthesized in sim train a Qwen3-VL-4B progress model (MAE 0.081 vs 0.230 for RoboReward-8B); real DSRL stack cups 60% sparse to 80% (10 trials). | UNC Chapel Hill, CMU, SJTU, Amazon | Rewards & progress | ✓ checked |
| 2026-07-14 | UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress ProxiesTraining-free correction of t/T labels via cross-episode nearest neighbours; advantage-conditioned cloth folding 72.8% (131/180) to 78.9% (142/180), vs a no-advantage baseline. | The University of Hong Kong | Rewards & progress | ✓ checked |
| 2026-07-11 | VINE: Taming Generative Control Policies for Reinforcement LearningReplaces the Euler sampler so direct Q-gradients through a flow policy stay stable; OGBench 40 tasks 60 vs QAM 55. | AgiBot; HKUST; PKU | Off-policy critics | ✓ checked |
| 2026-07-10 | Learning More from Less: Reinforcement Learning from Hindsight (LfH)A VLM relabels low-reward GRPO groups (mean reward <0.8) with a hindsight instruction and reward. LIBERO-PRO OOD: matches GRPO's final score in ~5 vs ~30 steps. Real Franka FR3: 56% vs GRPO's 22% at 160 rollouts. | MIT, MIT-IBM, Stanford, UCSD | On-policy PG | ✓ checked |
| 2026-07-10 | Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement LearningA history-conditioned VLM value guides quality-conditioned pretraining and filters rollouts for residual adaptation. Real chip insertion improves from 46% to 86% over three rounds of 500 rollouts. · code | Renmin University; Beijing Innovation Center of Humanoid Robotics | Advantage-weighted BC | ✓ checked |
| 2026-07-10 | ★ PAC-ACT: Post-training Actor-Critic for Action Chunking TransformersChunk-level PPO on a pretrained ACT (CVAE removed, learnable Gaussian noise), with a KL term between consecutive updates and a reward penalty for leaving the frozen BC policy. Sim Contour and Square Assembly: ACT 60.0%/51.2% → 100.0%/98.2%. | SUSTech | On-policy PG | ✓ checked |
| 2026-07-09 | Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided AgentsLLM coding agents compose a frozen VLA with analytic primitives, learning operating ranges from execution traces. Best planner backend versus frozen VLA alone: LIBERO-Pro 82.4% versus 50.0%; RoboCasa365 57.1% versus 30.0% (simulation). · code | Zhang et al. | Test-time search | ✓ checked |
| 2026-07-09 | ★ Prompt-Driven Exploration: Language as an Exploration Space for VLA Reinforcement Learning (PDE)A VLM rewrites task prompts to discover successful rollouts; PPO trains with a mixture including the deployment prompt. Sim LIBERO-PRO, pi0.5 zero-start Regime 3 task, step 120: 81.8% vs action-noise PPO 0%. | MIT Improbable AI Lab; MIT-IBM Computing Research Lab | On-policy PG | ✓ checked |
| 2026-07-09 | ★ FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent SpaceInverts the flow sampler to find the noise that would have produced the operator's correction and trains a small noise policy on it, VLA frozen. MetaWorld, pi0.5: 0.78 vs SFT 0.71; held-out skills 0.88 (base 0.96) vs SFT 0.02. · code | Microsoft Research; Microsoft; ETH Zurich; University of Washington | Human corrections | ✓ checked |
| 2026-07-08 | WAM-TTT: Steering World-Action Models by Watching Human Play at Test TimeTest-time training updates a lightweight adaptive memory inside a frozen world-action model via self-supervised prediction on unlabeled human videos; beats in-context human-video conditioning on manipulation tasks. | Feng et al. | Test-time search | ✓ checked |
| 2026-07-08 | Reward-Aware Population Scaling of Evolutionary Strategies in LLM Fine-TuningIn tested capable 0.5B–7B LLMs, z-score normalization can make binary-reward ES with N=2 fail; removing it improves GSM8K/TREC results. Derives zero-advantage probability and cautions that N=2 is not universally sufficient. | Cho et al. | General RL | ✓ checked |
| 2026-07-08 | PriGo: Test-Time Primitive Guidance to Diffusion and Flow Policies for Adaptive Robotic ManipulationLightweight PANet predicts motion primitives and applies differentiable guidance to pretrained diffusion/flow policies at inference, no retraining; gains on LIBERO, CALVIN, SIMPLER and real-robot tasks. | Li et al. | Test-time search | ✓ checked |
| 2026-07-08 | ★ HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout SegmentationA progress critic (VLAC-CUT) cuts autonomous pi0.5 rollouts into progress, recovery, idle and failure-inducing segments; only progress and recovery segments join HITL recovery data for retraining. Real round 2, Microplate: 42 tasks/hr at 90% vs HITL-only 25 at 60%. · code | Shanghai AI Laboratory | Human corrections | ✓ checked |
| 2026-07-07 | RoboSnap: One-Shot Real-to-Sim Scene Generation for Generalizable Robot Learning and EvaluationSingle RGB image to sim-ready scene (collision-aware foreground assets, 3DGS background); introduces DROID-Sim from 564 real scenes for replay, synthetic data generation and sim-real correlated policy evaluation. · code | Zhang et al. | Sim-to-real | ✓ corrected |
| 2026-07-07 | LAMP: Latent Motion Prior-Guided Real-World Learning for Dexterous Hand ManipulationResidual SAC/RLPD uses a learned 2-D hand-motion latent on a 6-DoF hand. Real four-task average: 56.25% → 98.75%; raw-action BC with the same residual RL averages 3.75%. · code | Fudan; PsiBot; Tsinghua; PKU | Residual & steering | ✓ corrected |
| 2026-07-07 | Optimal Transport Q-Learning for Flow Policy Steering and Acceleration (OTQL)Advantage-weighted optimal-transport pairing in flow matching; OGBench 0.59 at 2 function evaluations vs QAM 0.55, real single-task 36% → 86%. | University of Edinburgh; Honda Research Institute Europe | Off-policy critics | ✓ checked |
| 2026-07-06 | Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action ModelsPPO plus an annealed KL to a frozen SFT policy. At 1M steps it matches plain PPO at 2M (IND 0.93, OOD 0.77) and beats SFT warm-start and a BC loss. · code | MIRAI | On-policy PG | ✓ checked |
| 2026-07-06 | ★ SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable RoutingTrains PPO from scratch in sim on a capsule-chain cable for single-clip routing, then deploys through a 10 Hz twin whose sim joint angles become real targets. Real 3-clip routing, zero demos: 18/24 vs 12/24 copied from the hierarchical-IL paper. | NVIDIA; UC San Diego | Sim-to-real | ✓ checked |
| 2026-07-05 | VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA ModelsGRPO trains a Qwen3.5-9B instruction rewriter around frozen pi0/OpenVLA, once per episode. VL-Think success: pi0 12.6% to 38.7%; OpenVLA with the grounder 63.9% versus GEPA’s 55.2%. | MIRAI, MISIS | Black-box search | ✓ corrected |
| 2026-07-05 | HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action ModelsSmall TD3 adapter on a frozen world-action model reads its latents and proposed chunk; real average 26.4% → 87.1%. · code | CASIA; GigaAI; Tsinghua | Off-policy critics | ✓ checked |
| 2026-07-04 | OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual PoliciesGelSight tactile SAC residual with a scheduled scale (0.05 → 0.15) on a frozen visual policy. Real base 40/10/5/5% → 100/100/90/85% after 40-80 min, including a 12-minute warm-start. | U Maryland; Georgia Tech | Residual & steering | ✓ checked |
| 2026-07-04 | Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models (SVA: Search, Value, and Act)Distills simulator MCTS returns into a Q-model selecting candidates from a frozen VLA. Across five backbones, EmbodiedBench simulation success improves by 15.4 points on EB-Habitat and 13.2 on EB-Navigation over the base policies. | Nanjing University; ANU; CASIA | Test-time search | ✓ checked |
| 2026-07-03 | TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-TrainingVisuo-tactile world model imagines corrections at stalled states for policy post-training. Six real tasks, after two iterations: average task score 0.375 → 0.825, versus DreamGen 0.550. | Peking University | World models & eval | ✓ corrected |
| 2026-07-03 | DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any FinetuningDINOv2-space world model plus progress critic picks among pi0 samples and fixed motion primitives. Real OOD objects 23.75% → 66.25% (42.50% with policy samples only). | Meta FAIR; University of Minnesota | Test-time search | ✓ checked |
| 2026-07-02 | VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action HorizonTraining-free plug-in for chunked VLAs: a latent-space vision monitor detects drift mid-chunk, truncates execution, and replans with online gradient guidance, giving an event-triggered adaptive action horizon. | Pan et al. | Test-time search | ✓ checked |
| 2026-07-02 | Rank-Then-Act: Reward-Free Control from Frame-Order ProgressA GRPO-trained VLM ranks shuffled frames and the reward is rank agreement; games and sim only (Catrap L2 1.00 vs GVL 0.47). · code | T-Tech | Rewards & progress | ✓ checked |
| 2026-07-02 | GigaWorld-1: A Roadmap to Build World Models for Robot Policy EvaluationBenchmark (WMBench) and roadmap for world-model policy evaluators: 324k world-model rollout segments chained into human-scored episodes. Long-horizon action faithfulness and memory matter; background and photometric consistency correlate negatively with human scores. · code | GigaAI; Tsinghua | World models & eval | ✓ checked |
| 2026-07-02 | ★ WorldSample: Closed-loop Real-robot RL with World ModellingAdds a Dyna-like stream to real-robot SAC/RLPD: jittered actions are rendered by a post-trained Cosmos-Predict2.5 model, reward-labeled, and gated into the critic loss. Real robot, 5 tasks: 82% in 64 min vs HIL-SERL 56% in 83 min. | Nanyang Technological University; Tsinghua University; Central South University; Beijing University of Posts and Telecommunications | World models & eval | ✓ checked |
| 2026-07-02 | ★ One Demonstration Is Enough for Real-World Robotic Reinforcement Learning (AutoSERL)Replaces HIL-SERL's human interventions with one-demo rules: a planner corrects drift beyond 2 cm; stalls trigger partial demo replay. Real, no start randomization: 50/50 on all 6 tasks in 8-45 min; SERL (20 demos, same time): 0-20/50. · code | Institute of Automation, CAS; PKU-PsiBot Joint Lab; Peking University | Off-policy critics | ✓ checked |
| 2026-07-02 | Guided Action Flow: Q-Guided Inference for Flow-Matching Vision-Language-Action PoliciesIQL critic trained on 100 SmolVLA rollouts guides the frozen flow sampler. Real bottle-to-box: 19/40 → 34/40; baseline episodes also enter critic-development data, while guided episodes are subsequent. No other method baselines. | UCL | Test-time search | ✓ corrected |
| 2026-07-01 | ★ RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy EvaluationConverts Wan2.1-T2V-1.3B into a causal, KV-cached, 4-step autoregressive video model on DROID, trained with Step Forcing to limit drift, and scores rollouts with GPT-4o. Across 8 VLAs (4,186 rollouts): Pearson r = 0.989 with the real RoboArena leaderboard. | KAIST; Config | World models & eval | ✓ checked |
| 2026-07-01 | FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy ImprovementFailure-Aware Retry uses a value function to locate problematic action chunks, applies preference-based policy updates between attempts, and adds lightweight action perturbations during retries to avoid repeating failures. | CMU | Test-time search | ✓ checked |
| 2026-06-30 | Stage-Transition Dense Reward Modeling for Reinforcement LearningSTDR learns visual dense rewards from expert videos: stage-transition plus within-stage progress signals, with OOD detection and grasp regulation; beats LIV/VIP on all 14 sim manipulation tasks and matches or beats handcrafted dense rewards on several. | Yang et al. | Rewards & progress | ✓ checked |
| 2026-06-30 | ELASTIC: Efficiently Learning to Adaptively Scale Test-Time Compute for Generative Control PoliciesSAC allocates denoising steps and samples per state. Real pi0.5 matches Best-of-10 with 34% lower latency; a fixed schedule derived from its learned allocation also matches, while adaptation wins at a smaller budget. | CMU | Test-time search | ✓ checked |
| 2026-06-30 | Z-1: Efficient Reinforcement Learning for Vision-Language-Action ModelsGRPO on pi0.5 where each group branches from one cloned approach-phase state, so credit lands on the manipulation phase. RoboCasa 24 tasks 67.4 → 80.6. TurnOnSinkFaucet 29.2 → 98.4. | Zioneer Robot Team | On-policy PG | ✓ checked |
| 2026-06-29 | Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models (T^2VLA)GRPO uses DTW similarity to the policy’s most-confident rollouts as reward, without success labels. Simulated LIBERO: OpenVLA-OFT 91.0% to 97.2%; oracle-reward SimpleVLA-RL reference 99.1%. · code | Fudan, Shanghai Innovation Institute, MoShen | On-policy PG | ✓ checked |
| 2026-06-29 | ★ STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot LearningAn ensemble of small VLMs predicts time offsets between expert frame pairs (forward and reversed); the ensemble minimum gives per-frame advantages for CFGRL training of pi0. Real towel folding: 92.3% vs one-iteration RECAP-style baseline 55.6%, BC 33.3% (trials unstated). | Institute of Automation, CAS; Tsinghua University; UCAS | Advantage-weighted BC | ✓ checked |
| 2026-06-28 | Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering (MoRE)A temporary mode classifier guides policy-weight edits with a retain loss. Simulated Push-T desired-mode success averages 50.0% before versus 98.3% after MoRE; the original policy already completes the task in either mode. · code | Texas A&M University and others | Advantage-weighted BC | ✓ checked |
| 2026-06-27 | ReGuide: From Test-Time Guidance to Self-Improving Diffusion PoliciesPhase-gated latent-dynamics guidance at test time, with the successes distilled back through filtered BC. Tool Hang 0.0312 → 0.2404 after two iterations (sim). · code | Texas A&M | Advantage-weighted BC | ✓ checked |
| 2026-06-26 | SimFoundry: Modular and Automated Scene Generation for Policy Learning and EvaluationReconstructs video-to-sim digital twins and affordance-preserving object, scene and task variants for policy training. Across seven manipulation tasks and five policies, simulated evaluations correlate with real performance (mean Pearson 0.911). · code | Ranawaka et al. | Sim-to-real | ✓ checked |
| 2026-06-26 | WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data CurationSelf-supervised relative-progress reward model trained on time-warped (speed-varied, reversed) demos; used to filter/reweight BC action chunks, giving up to ~18x successful T-shirt-folding throughput over vanilla BC on a real bimanual robot. · code | Yu et al. | Rewards & progress | ✓ checked |
| 2026-06-25 | World Action Models Enable Continual Imitation Learning with Recurrent Generative ReplaysREGEN: a world action model generates synthetic replay trajectories from instructions and observations, cutting catastrophic forgetting up to 50% vs sequential fine-tuning without storing demos. | Govind, Reilly, Patel, Le, Das | World models & eval | ✓ checked |
| 2026-06-25 | KinetIQ Ascend: Towards 100% Manipulation Reliability and Superhuman SpeedReal-world PPO on a flow-matching VLA (ODE-to-SDE exploration noise, sparse success reward, speed curriculum), with gains measured against a concurrent frozen-policy A/B baseline. Tote handling 77.6%→98.9%, bottle picking 80%→98%, throughput +42% to +129% in 3-5 robot-days. | Humanoid | Industry loops | ✓ checked |
| 2026-06-25 | ★ Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience (SCORE: Support-Constrained Off-Domain REinforcement)Freezes a real-demo flow policy; in a scanned sim twin, PPO learns to steer its input noise plus a small residual, staying near the base's action support. Real Franka/LEAP, 8 tasks, no real-world RL: 37.8% → 89.9% vs Residual-RL 59.5%. | University of Washington; Amazon FAR | Residual & steering | ✓ checked |
| 2026-06-25 | E-TTS: A New Embodied Test-Time Scaling Framework for Robotic ManipulationJoint sampling of reasoning traces and actions scored by reasoning and action verifiers, with critique-and-resample. Real MolmoAct 22.13% → 48.75%; sim E-CoT 6.67 → 39.81 vs naive best-of-N 23.61. | CASIA | Test-time search | ✓ checked |
| 2026-06-25 | Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)AWR sampling, RECAP conditioning and CFG improve a simulated bimanual SO-ARM101 policy that also predicts value: 79.63%, first of 62 teams. The second-place real final instead uses plain BC. · code | Independent researcher | Advantage-weighted BC | ✓ checked |
| 2026-06-24 | Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?Two-stage RL for multi-fingered hands: task-agnostic play pretraining on diverse objects, then sparse-reward assembly fine-tuning; 33x more sample-efficient than RL from scratch (even with dense rewards); zero-shot real 60% (6/10) on 0.5 mm-clearance insertions. · code | Lum et al. | Sim-to-real | ✓ checked |
| 2026-06-24 | RMTL: Reinforced Micro-task Learning for Long-Horizon Manipulation with VLM RewardsSplits long-horizon tasks into language-described micro-tasks, computing multi-view VLM rewards per active micro-task with hierarchical RL and reverse curriculum; faster learning than single-prompt VLM rewards on Fetch. | Ateş et al. | Rewards & progress | ✓ checked |
| 2026-06-24 | ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action ModelsReplaces PPO's policy loss with distillation to a closed-form, token-level advantage-tilted teacher. OOD 69±4 → 73±4 vs PPO, but not better on every ID cell, and 40.8 h vs 25.4 h compute. | UNSW, Deakin, AFRL | On-policy PG | ✓ checked |
| 2026-06-24 | FORCE: Efficient VLA Reinforcement Fine-Tuning via Value-Calibrated Warm-up and Self-DistillationConRFT-style Cal-QL + consistency-policy recipe without human interventions, adding an on-policy value-calibration warm-up and value-filtered self-distillation; ManiSkill 6-task sim average FORCE (Octo) 82.3% vs ConRFT without human intervention 71.1%. | BAAI; CASIA | Off-policy critics | ✓ checked |
| 2026-06-23 | Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot ImitationReTVL treats retry events in imperfect demos as supervision, combining global progress calibration with local pairwise preferences to learn a non-monotonic value model that reweights demonstration segments for BC. | Qin et al. | Rewards & progress | ✓ checked |
| 2026-06-23 | World Value Models for Robotic ManipulationWorld-model-based value estimation for data curation and policy learning. Reports the highest Value-Order Correlation among tested baselines and introduces Suboptimal-Value-Bench with 800 annotated suboptimal trajectories; improves simulated and real manipulation. | Wang et al. | Rewards & progress | ✓ corrected |
| 2026-06-23 | Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop RefinementReal-to-sim cloth mesh estimate plus MPPI in a GPU cloth simulator around the base plan. Real single-arm fold 3/10 → 9/10, dual-arm 1/10 → 8/10. | NUS; SJTU | Test-time search | ✓ checked |
| 2026-06-22 | dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action ModelsRL fine-tuning for discrete-diffusion VLAs: treats denoising as an MDP and optimizes the sampled path's joint probability; 99.7% on LIBERO, and 61.4% → 92.0% over its SFT backbone on eight RoboTwin 2.0 tasks. | Wu et al. | On-policy PG | ✓ checked |
| 2026-06-22 | ★ Learning Process Rewards via Success Visitation Matching for Efficient RLTrains a classifier on the policy's own successful vs failed rollouts and adds its clipped log-odds to the sparse success reward. Real WidowX with DSRL, Pick and Place: about 0.93 vs about 0.2 for outcome-only RL (plot reading). | UC Berkeley | Rewards & progress | ✓ checked |
| 2026-06-22 | HiL-ResRL: A Model-Agnostic Finetuning Adapter via Human-in-the-loop Residual Reinforcement LearningSAC residual plus SpaceMouse interventions on frozen Diffusion Policy or pi0.5. Bases at 50-80% rise to over 90% within 40-90 min of real training. | Huawei Cloud; SCUT | Residual & steering | ✓ checked |
| 2026-06-21 | PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action ModelsPolicyTrim uses RL to improve action-chunk execution efficiency. For pi0.5 on LIBERO-Object, physical steps fall to 51.4% of baseline (48.6% fewer); the paper reports up to 5.83x end-to-end speedup. · code | Wang et al. | On-policy PG | ✓ checked |
| 2026-06-21 | ★ FlowDPG: Deterministic Policy Gradient on Flow Matching Policies for Real-World ManipulationPredicts the clean action from a noisy flow point, nudges it along the critic's gradient, and regresses velocity toward it alongside BC, without ODE backprop. Real dual-arm AirPods assembly: 88% offline vs 80% for RLT/QAM (BC 64%). | Carnegie Mellon University, University of Pennsylvania, Skild AI | Off-policy critics | ✓ checked |
| 2026-06-20 | ★ RARM: Confidence-Gated Progress Reward Modeling for RL in ManipulationA comparator trained on non-robot videos (DAVIS, MOSE) matches rollout clips to one successful demo, rewarding only confident forward progress. Real pi0 with DSRL, cloth folding after 90 rollouts: 25/30 vs binary reward 7/30, Robometer 9/30. | National University of Singapore, Booking.com, University of Cambridge, Nanjing University | Rewards & progress | ✓ checked |
| 2026-06-19 | Robot Self-Improvement via Human-Video Dynamics ModelsHuman-video dynamics plus value models relabel failed states with better actions. Stretch average 85.3% vs RISE 76.0%; pi0.5 62.7 → 88.0. | ETH Zurich; TU Munich; Microsoft; MCML | World models & eval | ✓ checked |
| 2026-06-19 | Robot Critics that Sweat the Small StuffFine-tunes a Qwen2.5-VL pairwise progress critic from policy success/failure rollouts (binary labels) to pick among video-model-predicted outcomes of sampled actions. Real diffusion policy: Stacking 23/50 → 32/50, Push-Bowl mIoU 0.140 → 0.321. · code | Columbia; TRI | Test-time search | ✓ corrected |
| 2026-06-18 | Temporal Self-Imitation LearningTSIL mines fast successful RL trajectories into adaptive temporal targets and efficiency-weighted self-imitation, improving success, speed and robustness on 30 long-horizon manipulation and navigation tasks. | Duke | On-policy PG | ✓ checked |
| 2026-06-18 | Reward as An Agent for Embodied World ModelsAgentic Qwen3.5 reward that checks planning, visual quality, instruction following, physics and task completion, plus DynDiff-GRPO rollout diversification, for RL post-training of robot video world models; PAI-Bench robot Domain score +0.80 (Cosmos-Predict2.5-2B), +3.08 (Kairos-3.0-Robot). · code | Li et al. | World models & eval | ✓ checked |
| 2026-06-18 | ★ ENPIRE: Agentic Robot Policy Self-Improvement in the Real WorldCoding agents edit training recipes, train, run real rollouts scored automatically, and keep what raises success. Pin insertion: 8 agent/robot pairs reached near-perfect success in about 40 min vs over 1.5 h with one (success counts up to 8 retries). · code | NVIDIA, Carnegie Mellon University, UC Berkeley | Black-box search | ✓ checked |
| 2026-06-17 | SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video GenerationVideo-model policy evaluator enforcing forward-inverse dynamics, cross-view and test-time consistency; 0.929 closed-loop Pearson and 0.119 MMRV across seven real-world VLA policies. | Tseng et al. | World models & eval | ✓ checked |
| 2026-06-17 | Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation4D wrist-view surfel-indexed memory for action-conditioned world models; Pearson correlation with real performance rises from 0.85 (Ctrl-World) to 0.97, and synthetic-data training lifts average success on three long-horizon real tasks from 58% to 72%. | Zheng et al. | World models & eval | ✓ corrected |
| 2026-06-17 | DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient FinetuningExplores diffusion-policy candidates with critic min-value plus disagreement: three policy samples and one base sample, with ten critics (DSRL normally uses two). Reports sample-efficiency gains on simulated RoboMimic and OpenAI Gym tasks. · code | Stanford; Brown | Off-policy critics | ✓ checked |
| 2026-06-17 | Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA EnhancementTD3 residual on 6-DoF object poses, proprioception and the base action, trained only in a simple MuJoCo scene (measured primitives) against a paired 'sim VLA'. Real GR00T-N1.5 five-task average goes from 42% to 76% with zero real-world RL. | KAIST; MSRA Tokyo; U Tokyo | Residual & steering | ✓ checked |
| 2026-06-16 | Reversal Q-LearningOffline RL for flow policies: treats each flow refinement step as an MDP action, using flow reversal for synthetic on-policy trajectories; no backprop through time; tops flow baselines on 50 simulated robot tasks. · code | Oberai et al. | Off-policy critics | ✓ checked |
| 2026-06-16 | Recover, Discover, Plan: Learning Skills and Concepts from Robot FailuresReSYNC learns RL recovery skills and discovers relational predicates from them to refine an abstract planner; outperforms evaluated baselines on unseen long-horizon tasks across four simulation domains, with a real-robot sim-to-real demonstration. | Li et al. | On-policy PG | ✓ corrected |
| 2026-06-16 | WAM-RL: World-Action Model Reinforcement Learning with Reconstruction Rewards and Online Video SFTOnline RL for world-action models that jointly optimizes world and action branches via environment interaction; actor-only tuning helps short-horizon tasks, joint optimization is needed for long-horizon gains. | Qian et al. | World models & eval | ✓ checked |
| 2026-06-16 | Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement (VERITAS)Gradient-free verifier scores sampled π0 action chunks by distance to a Gemini-drawn 2D end-effector trace; verified successes are distilled back by BC. Reported average steering gains: 12.6% sim, 35% real; BC fine-tuning +9.7% over π0-Bridge in sim. · code | Princeton | Test-time search | ✓ checked |
| 2026-06-15 | Steering Generative Reinforcement Learning into Stable Robotic Controller (SteerGenPO)Replaces stochastic latent sampling of a trained generative RL policy with a learned state-dependent latent actor for deterministic deployment; beats baselines on six Isaac Lab tasks plus Unitree G1 locomotion. | Wang et al. | Residual & steering | ✓ checked |
| 2026-06-15 | Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes (HABC: Hierarchical Advantage-Weighted Behavior Cloning)Viability and efficiency critic heads turn episode success bits into per-transition flow-matching weights. On three real bimanual tasks, initial online fine-tuning raises SFT success 36/44/12% to 60/78/22%; compared online methods share the same interaction budget. | ACE Robotics; Tsinghua SIGS; CUHK | Advantage-weighted BC | ✓ corrected |
| 2026-06-15 | SafeDojo: Safe Reinforcement Learning for VLA via Interactive World ModelLagrangian-constrained GRPO inside a Wan 2.2-based video world model, with a learned per-step safety-cost head. Real Franka (5 tasks, 10 trials each): safe-success 70% vs 36% for the best baseline (SafeVLA), 22% for WoVR. | Peking University; Beijing Innovation Center of Humanoid Robotics; NTU; HKUST; UESTC | World models & eval | ✓ checked |
| 2026-06-15 | ★ ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement LearningSplits each human intervention into autonomous rollout, human adaptation and human recovery, placing the failure penalty after adaptation; an optimistic expectile critic drives RECAP-style conditioning. Iteration 1, same robot data, Erase/Bread: 60.0%/73.3% vs RECAP 50.0%/66.7%. | XPENG Robotics; Fudan; CUHK; SJTU | Advantage-weighted BC | ✓ checked |
| 2026-06-12 | Inference-time Policy Steering via Vision and Touch (ViTaL)Vision picks the mode (latent world model plus VLM reward), then tactile-reward-guided diffusion editing refines contact. Real, visual base policy (appendix): wiping 30 → 90%, pipette 30 → 75%, insertion 40 → 80%. | CMU | Test-time search | ✓ corrected |
| 2026-06-11 | EV-WM: Event-Verified World Models for Long-Horizon Robotic ManipulationDecodes imagined futures from a pretrained-feature world model into predicate-level event states and scores task progress, semantic consistency, physical feasibility and uncertainty to guide CEM planning. PointMaze random-target success 0.90 → 0.94 (simulation only). | Wang et al. | World models & eval | ✓ checked |
| 2026-06-11 | WEAVER, Better, Faster, Longer: An Effective World Model for Robotic ManipulationMulti-view flow-matching world model: Spearman 0.870 with real policy success. Across five real tasks, mixed real/synthetic fine-tuning raises π0.5 success 44%→82%; best-of-four steering reaches 58%. Reports 5–10× faster planning than Ctrl-World. · code | Jain, Wu, Farebrother, Swamy, Bajcsy | World models & eval | ✓ corrected |
| 2026-06-11 | Improving Robotic Generalist Policies via Flow Reversal Steering (FRS)Flow-reverses coarse human/VLM hints into noise for frozen pi0.5, then trains noise-space BC/RL. Real DROID ~20%→~87% with zero-shot human steering (chart reading). Towel hanging: 5%→50% with BC→80% after RL. | Stanford; UC Berkeley | Residual & steering | ✓ corrected |
| 2026-06-10 | Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model SteeringSearches for language instructions that raise a frozen VLA's closed-loop success, distills them into a language feedback policy, and uses a conformalized improvement head to refuse harmful steering; tested with pi0.5 on LIBERO-OOD and a real Franka. | Carnegie Mellon University | Residual & steering | ✓ checked |
| 2026-06-10 | ★ TacCoRL: Integrating Tactile Feedback into VLA via SimulationAdds contact-gated tactile tokens to a pi0.5-style VLA, co-trains on real plus sim demos, then runs sim PPO with a real-data BC anchor. Real, 4 bimanual tasks after sim RL: 72.5% visuo-tactile vs 50.0% vision-only. | UCLA, UC San Diego, Peking University, University of Utah, UESTC | Sim-to-real | ✓ checked |
| 2026-06-10 | ★ UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement LearningAdds an intervention agent to HIL-SERL: a VLM-based proxy value model and a temporal value-risk critic detect stagnation, then steer back to a remembered high-value state. 5 real UR7e tasks: HIL-SERL 81% success at 34.3% intervention → 88% at 14.6%. · code | Nanyang Technological University; Beijing University of Posts and Telecommunications | Human corrections | ✓ checked |
| 2026-06-09 | VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models3D-aware 7B verifier over k=8 VLA samples. SimplerEnv OpenVLA 37.0 → 55.0 vs RoboMonkey 40.5, V-GPS 36.0, MG-Select 39.0. | CASIA | Test-time search | ✓ checked |
| 2026-06-09 | ★ SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic ManipulationClassifies the active action primitive to gate a mixture-of-experts value head regressing normalized remaining steps, used as reward for a residual TD3+BC policy on frozen pi0.5. Real, Round 3 Whiteboard: 18/20 vs fine-tuned Robometer 13/20, sparse 11/20. · code | Stanford, UC Berkeley, SJTU, xdof.ai | Rewards & progress | ✓ checked |
| 2026-06-09 | ★ Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning (QGF, Q-Guided Flow)Freezes a BC flow policy and IQL critic; each flow step adds grad Q, evaluated at a one-step estimate of the finished action, to the velocity. OGBench sim, charted 20-task mean: beats tested test-time methods, slightly edges EDP. · code | UC Berkeley; Physical Intelligence | Test-time search | ✓ checked |
| 2026-06-08 | ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action PoliciesFrozen VLA plus residual recovery policies trained in sim with VLM-selected structured reward masks; sim success 36.7% (fine-tuned pi0.5) to 66.7%, 61.7% zero-shot sim-to-real. | Hu et al. | Residual & steering | ✓ checked |
| 2026-06-08 | ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action ModelsTraining-free runtime loop: feature probe for 3D object positions, kinematic failure state machine, and CBF action filter; lifts OpenVLA-OFT from 69.6% to 74.1% on LIBERO-plus. | Zhang et al. | Test-time search | ✓ checked |
| 2026-06-08 | DexPIE: Stable Dexterous Policy Improvement from Real-World ExperienceOne RECAP round with a continuous sigmoid optimality indicator on a dexterous diffusion policy. Average 50.0 BC / 75.3 HG-DAgger / 80.0 RECAP / 87.3% (50 trials per task). · code | Hunan University | Advantage-weighted BC | ✓ checked |
| 2026-06-08 | TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich ManipulationFrozen wrench-aware pi0.5 plus small per-contact-phase actor-critics; full-task 12/30 (pi0.5) and 21/30 (reference) → 28/30. | Meituan; CASIA and others | Off-policy critics | ✓ corrected |
| 2026-06-07 | ActProbe: Action-Space Probe for Early Failure Detection of Generative Robot PoliciesAction-chunk consistency/magnitude failure detector. Reports +12.7% detection-frontier hypervolume and +9.0 points early ROC-AUC on unseen tasks; combining automatic success resets with early-failure resets reaches default PPO’s plateau with 2.9× fewer interactions on one LIBERO task. · code | Huang et al. | Test-time search | ✓ corrected |
| 2026-06-07 | Reinforcement Learning for Flow-Matching Policies with Density TransportRLDT fine-tunes flow policies toward an SVGD transport field from a maximum-entropy RL objective without backpropagation through time. Evaluated in simulation on Gym, state-based Furniture-Bench and vision-based Robomimic. | Lei et al. | Off-policy critics | ✓ checked |
| 2026-06-07 | ★ Video2Sim2Real: Full-Stack Autonomous Dexterous Skill Acquisition from a Single Human VideoBuilds a sim twin from one RGB-D human video, re-optimizes robot poses at object-motion keyframes, distills them into a point-cloud policy, adds a PPO finger residual. Real, 7 tasks, no robot demos: 67/70 (95.7%) vs 11/70 with FoundationPose poses. · code | Georgia Institute of Technology, University of Pennsylvania, Toyota Research Institute, Carnegie Mellon University | Sim-to-real | ✓ checked |
| 2026-06-06 | Q-VGM: Q-Guided Value-Gradient Matching for Offline-to-Online RL of Flow-Matching VLA PoliciesValue-gradient matching at late pi0.5 denoising steps avoids BPTT; LIBERO few-shot-SFT mean 84.6% becomes 90.7% offline and 98.4% after online training. | Shanghai Jiao Tong University; University of Michigan | Off-policy critics | ✓ checked |
| 2026-06-04 | Depth over Fidelity in Fixed-Budget Noisy Evolution StrategiesProbabilistic elite membership (PEM) replaces hard ES rank weights with expected weights under ranking uncertainty. Residual-bootstrap RB-PEM improves COCO bbob-noisy, policy-search and hyperparameter-optimization results in tested noisy, fixed-budget settings. · code | Wang et al. | General RL | ✓ checked |
| 2026-06-03 | Potential-Guided Flow Matching for Vision-Language-Action Policy Improvement (ForesightFlow)The VLA generates a success-potential channel used both as the AWR baseline and for best-of-K selection. Real average 35.4% vs IDQL 32.6% vs filtered BC 24.6%. | BIT; Tsinghua; HKUST (Guangzhou) | Advantage-weighted BC | ✓ corrected |
| 2026-06-03 | ★ FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference OptimizationOperators roll back and teleoperate fixes, giving paired failed/corrected segments for a critic-free DPO-style flow-matching loss with proximal and SFT terms. Real Pack, pi0 base, 3 rounds: 99% vs 94% for a RECAP-style advantage-conditioning baseline, DAgger 88%. | Tencent Robotics X; Futian Laboratory; Tsinghua University | Human corrections | ✓ checked |
| 2026-06-02 | TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action ModelsReward-free test-time latent-prompt tuning uses a state-grounding proxy loss. SimplerEnv WidowX: base pi0.5 51.1%, retrained latent-prompt policy 63.5%, then test-time tuning 67.4% (simulation). | ByteDance Seed | Test-time search | ✓ checked |
| 2026-06-02 | Preference-Calibrated Human-in-the-Loop Reinforcement Learning for Robotic Manipulation (PACT: Preference-calibrated Actor-Critic Training)Treats each human takeover as a preference to deflate credit for pre-takeover actions; average 58.0 → 82.5 vs HIL-SERL on 5 real tasks. | Central South University; NTU | Off-policy critics | ✓ checked |
| 2026-06-01 | Set-Supervised Diffusion Policy: Learning Action-Chunking Diffusion through Corrections (SDP)Uses the robot's rejected chunk to build a 'desired set' around the correction; robust to noisy corrections (noisy Square 0.946 vs DP 0.683). · code | TU Delft; University of Stuttgart | Human corrections | ✓ checked |
| 2026-06-01 | Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards (CSIL++)Dense coherent reward from a small BC policy avoids the fine-tuning dip on a frozen pi0.5; Threading 0.14 → 0.92 in sim, Nut Assembly no gain. | TU Darmstadt, Oxford and others | Off-policy critics | ✓ checked |
| 2026-05-31 | τ0-WM: A Unified Video-Action World Model for Robotic ManipulationUnified video-action predictor plus action-conditioned simulator that scores candidate actions by task progress; trained on ~27,300 hours of teleop, UMI, egocentric human video and rollout/failure data. · code | Zhou et al. | World models & eval | ✓ checked |
| 2026-05-31 | Lagrangian Perturbation Diffusion Steering: Latent Reinforcement Learning for Generative Policies (LP-DS)Learns residual noise perturbations for a frozen diffusion policy using a Lagrangian trust region to limit prior deviation. Walker2D return: approximately 5000 versus 4000 for the strongest compared baseline. · code | Bilkent University | Residual & steering | ✓ checked |
| 2026-05-29 | FLAG: Flow Policy MaxEnt-RL by Latent Augmented GuidanceMaxEnt RL for flow policies: augments the state with a flow latent to localize importance sampling, mitigating weight collapse via a provably consistent proxy objective; scales to high-dimensional control (DMC, MyoSuite, MuJoCo). · code | Kim et al. | Off-policy critics | ✓ checked |
| 2026-05-29 | Feat2Go: Visual Feature-Grounded Value Estimation for Embodied Reinforcement LearningValue model trained on progress targets from world-model patch similarity to subgoals reshapes terminal rewards for PPO/GRPO VLA fine-tuning; ManiSkill3 (sim) average OOD success 82.9% vs 76.8% for OpenVLA-OFT with plain PPO and 17.5% after SFT. | Shu et al. | Rewards & progress | ✓ checked |
| 2026-05-29 | HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended LoadsNoise-space RL adapts GR00T N1.6 for suspended-load pushing. Sim placement error: 22.44cm to 2.54cm after 20 rollouts, 6.25cm after 30. Real error: 9.28cm to 6.38cm; yaw worsens from 14.5° to 28.9°. | University of Florida | Residual & steering | ✓ checked |
| 2026-05-29 | StressDream: Steering Video World Models for Robust Policy Evaluation and ImprovementOptimizes diffusion-world-model noise toward plausible VLM-specified failures. Failure recall rises 54% to 94%; using steered failures to downweight demonstrations raises real pi0.5 success from 39% with uniform-weight fine-tuning to 71%. · code | CMU; NVIDIA; UW; Stanford | World models & eval | ✓ checked |
| 2026-05-28 | VE2VF: Vision-Enabled to Vision-Free Distillation via Real-world Reinforcement Learning for Robust Contact-Rich ManipulationHIL-SERL-style vision teacher distilled into a vision-free student via real RL; 95.0% overall vs residual RL 85.7% in about 50 min. | TU Wien | Off-policy critics | ✓ checked |
| 2026-05-28 | Sample-Efficient Diffusion-based Reinforcement Learning with Critic Guidance (CGPO: Critic-Guided diffusion Policy Optimization)Critic-guided diffusion sampling distilled back with advantage-weighted denoising; highest reported mean return among compared methods on all five MuJoCo tasks. · code | ShanghaiTech University | Off-policy critics | ✓ corrected |
| 2026-05-28 | BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA ModelsIQL-style offline chunk critic in a consistency-policy dexterous VLA, followed by frozen-base residual adaptation. Current v3: six real standard-object tasks improve 60.8%→82.5% with 20 online trajectories per task; held-out objects improve 52%→70%. | SJTU, CASIA, Shanghai AI Lab, USTC | Off-policy critics | ✓ corrected |
| 2026-05-27 | SPAR: Support-Preserving Action RectificationOffline improvement through local residual rectification of a frozen BC policy. Latent Self-Imitation uses weighted latent regression to avoid manifold-normal value-gradient drift under the paper’s assumptions; reports strong results against evaluated D4RL baselines. | Zhejiang University | Residual & steering | ✓ corrected |
| 2026-05-27 | ProgVLA: Progress-Aware Robot Manipulation Skill LearningCompact 0.1B VLA with two-stage Perceiver resampling and jointly trained progress heads (offline-RL critics over normalized remaining-horizon targets) that weight flow-matching imitation; beats imported larger-baseline scores on LIBERO and Meta-World; progress weighting adds 2.3 points on LIBERO. | Kim et al. | Advantage-weighted BC | ✓ checked |
| 2026-05-27 | How VLAs Fail Differently: Black-Box Action Monitoring Reveals Architecture-Specific Failure Signatures450 PushT/ALOHA episodes across VQ-BeT, Diffusion Policy, ACT: action direction reversals predict failure (AUROC 0.79-0.93), velocity violations barely do; training-free SafeContract monitor with conformal calibration. · code | Gupta | Test-time search | ✓ checked |
| 2026-05-26 | VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy CorrectionMC-dropout uncertainty picks 2 s failure clips from Isaac Lab rollouts for VR correction; success comparable to unguided HITL review at about 40% less collection time per sample, up to +23 pp over BC (sim). | ETH Zurich | Human corrections | ✓ checked |
| 2026-05-25 | Scaling World-Model Reinforcement Learning Through Diffusion Policy OptimizationReplaces MPPI search with diffusion policy optimization anchored by an implicit energy/KL term. Competitive or better online across 121 simulated tasks; offline scaling generally helps, but both methods decline at 340M parameters on the 30-task dataset. · code | Cheng et al. | World models & eval | ✓ checked |
| 2026-05-25 | ★ A Factory-Floor Deployment Case Study of VLA Pipelines for Industrial Packaging Task: Workflow, Failures, and LessonsRuns a pure-SFT loop on pi0.5 in a real Siemens factory: teleop demos, manual review, re-fine-tuning, and targeted recovery demos, relaxing constraints each round. 2,535 factory episodes; no final success rate given; authors say it fell short of expectations. | Siemens | Industry loops | ✓ checked |
| 2026-05-25 | TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic ManipulationAction-VAE posterior turns a policy's actions into extra latent candidates; a task-progress verifier drops low scorers and score-weights the rest. CALVIN ABC→D avg length DP 2.41 → 2.58, OpenVLA 3.30 → 3.51; real π0 78.3% → 83.3%. · code | Harbin Institute of Technology | Test-time search | ✓ checked |
| 2026-05-25 | ★ EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action ModelsSamples 8 pi0.5 chunks, adds a tanh-bounded learned edit to each, runs the argmax-Q of 16; the VLA keeps training on its supervised loss, with human interventions. Real, 8 tasks: SFT 20.5/30 → 30/30 each, 19.1 min average (HG-DAgger 22.1/30). · code | Stanford University | Off-policy critics | ✓ checked |
| 2026-05-24 | Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged LocomotionPolicy initialization, timeout-aware critic targets and multi-step returns let SAC match PPO’s performance in parallel legged-locomotion simulation, with slower wall-clock training. Real-robot fine-tuning remains future work. · code | ETH Zurich RSL | Off-policy critics | ✓ checked |
| 2026-05-22 | From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward ModelsDemo2Reward optimizes a frozen VLM reward model’s prompt using 3–10 demonstrations to reduce false positives. Evaluated in simulated policy learning and one real Franka Panda Lid-On-Pot task, without extra model training. | Gumbsch et al. | Rewards & progress | ✓ checked |
| 2026-05-19 | PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action ModelsTreats a VLA as planner plus executor: identifies planning actions via action variation and trajectory outcome, weights them by causal sufficiency/necessity, and injects weights into GRPO advantage estimation. | Guo et al. | On-policy PG | ✓ checked |
| 2026-05-19 | Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning (ZPRL, Z-Perturbation RL)SAC perturbs a variational-bottleneck latent in a frozen flow policy. Four real tasks: reported mean gain 33.7 percentage points over imitation; Insert Bills 20%→77.5%. Online collection totals about 28.5 hours. · code | HKU; Shanghai Qizhi; SJTU | Residual & steering | ✓ corrected |
| 2026-05-17 | DyGRO-VLA: Cross-Task Scaling of Vision-Language-Action Models via Dynamic Grouped Residual OptimizationLearns shared multi-task VLA representations, then applies dynamically routed residual RL experts. Evaluated on LIBERO and RoboTwin2; sim-to-real tests on four tasks show mixed gains and limited real-world coverage. | Lin et al. | Residual & steering | ✓ checked |
| 2026-05-16 | Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden StatesTraining-free steering of a frozen policy's action-expert activations with contrastive conceptors fit from about 15 success/failure rollouts per task. Mean success up over 20 points in sim (partly early, under-trained checkpoints) and 40 points on 3 real tasks. | University of Pennsylvania | Test-time search | ✓ checked |
| 2026-05-15 | OHP-RL: Online Human Preference as Guidance in Reinforcement Learning for Robot ManipulationUses human takeovers as pairwise preferences in SAC with a critic-based intervention gate. Real Franka Move Sand Bag success: 68% versus SILRI 66% and HIL-SERL 16%. | Hong Kong University of Science and Technology (Guangzhou) | Human corrections | ✓ corrected |
| 2026-05-14 | Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Intervention (HandITL)Shared hand-arm corrections for a 56-DoF bimanual VLA with two 21-DoF hands; one hour of Copilot-mode correction data raises mean normalized subgoal score from about 0.70 to 0.95 across three real tasks. | Shanghai Jiao Tong University | Human corrections | ✓ checked |
| 2026-05-13 | RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited DataVLM planner and video simulator co-evolve via GRPO exploration and near-miss preference mining. Planner gains 36.4/24.6 points over base Qwen3-VL on EB-ALFRED/EB-Habitat; simulator gains ~48% relative Gemini-judged task success on BridgeData V2. | HKUST (Guangzhou); HKUST | World models & eval | ✓ checked |
| 2026-05-13 | D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action ModelsDistributed async RL system for large VLAs: decouples data and weight traffic, four-thread swimlane pipeline, dual-pool VRAM management; higher throughput than RLinf and RL-VLA3 for pi0.5 and OpenVLA-OFT in ManiSkill (16 GPUs). | Guo et al. | On-policy PG | ✓ checked |
| 2026-05-12 | DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA PoliciesTest-time framework for VLAs: a trigger detects critical phases, a proposer samples alternative action chunks, and an evaluator trained on success/failure/boundary data dreams futures to pick one; four real tasks average 72.5% vs 48.8% for base π0.5. · code | Fan et al. | Test-time search | ✓ checked |
| 2026-05-12 | Think Twice, Act Once: Verifier-Guided Action Selection For Embodied AgentsVeGAS: MLLM embodied agent samples multiple candidate actions and a verifier trained on LLM-synthesized failure cases picks one; up to 36% relative gain over CoT baselines on Habitat and ALFRED. | Singhi et al. | Test-time search | ✓ checked |
| 2026-05-12 | Discrete Flow Matching for Offline-to-Online Reinforcement LearningDRIFT fine-tunes an offline-pretrained CTMC policy with advantage-weighted discrete flow matching, a path-space penalty and candidate-set approximation; achieves the highest average Jericho text-game score among evaluated baselines, without robot experiments. | Khan et al. | Off-policy critics | ✓ corrected |
| 2026-05-12 | ★ Behavioral Mode Discovery for Fine-tuning Multimodal Generative PoliciesA noise-steering policy on a frozen diffusion policy, conditioned on a per-episode code, learns distinct modes via a DIAYN-style classifier reward, kept as an RL fine-tuning bonus. ManiSkill sim Avoid, success ~1.00: DSRL keeps 2/24 modes, DPPO 6.33/24, DSRL[BMD] 10/24. | Naver Labs Europe; Stanford | Residual & steering | ✓ checked |
| 2026-05-12 | Reinforcing VLAs in Task-Agnostic World Models (RAW-Dream)A task-agnostic world model and zero-shot VLM rewards refine a policy initialized with one-shot SFT. LIBERO success: 43.4% to 52.3% without target-task world-model training; real Piper few-shot SFT: 50.0% to 71.7%. | Nanjing University; UIUC; UTS; Tsinghua | World models & eval | ✓ checked |
| 2026-05-12 | TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy FinetuningContext-smoothed pretraining plus a learned noise-level dial broadens exploration beyond the base policy. Real widowx-sausage reaches 1.0 success at about 50 episodes while DSRL stays at 0 (chart reading; RSS 2026). · code | U Washington; Amazon FAR | Residual & steering | ✓ corrected |
| 2026-05-11 | Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models (adjacent)RAM: reward-corrected pretraining target with a consistency loss, no SDE rollouts or backward adjoint sweeps; matches Flow-GRPO peak reward on SD3.5-M in up to 50x fewer steps. Image generation only. · code | Bergmeister et al. | General RL | ✓ checked |
| 2026-05-11 | RankQ: Offline-to-Online Reinforcement Learning via Self-Supervised Action RankingSelf-supervised action-ranking losses replace CQL-style pessimism; antmaze-large-play 91.2% vs Cal-QL 67.7%, plus pi0 VLA experiments in sim. | Horizon Robotics | Off-policy critics | ✓ checked |
| 2026-05-11 | Adaptive Action Chunking via Multi-Chunk Q Value Estimation (ACH)Causal-transformer critic scores every action-chunk prefix to select chunk length per state. On simulated OGBench puzzle-4x4-play, offline→online success rises 13%→90%, versus Q-chunking 11%→38%. | KAIST | Off-policy critics | ✓ corrected |
| 2026-05-11 | Retrieve-then-Steer: Online Success Memory for Test-Time Adaptation of Generative VLAsStores progress-calibrated successful segments during deployment, retrieves and consistency-filters them into an elite action prior, and injects it into a frozen VLA's action sampler with confidence-adaptive strength. LIBERO-10: pi0.5 92.4% → 94.4%, pi0 81.6% → 84.4%; no weight updates. | Xi'an Jiaotong University; One Robotics; Shenzhen University of Advanced Technology | Test-time search | ✓ checked |
| 2026-05-11 | ★ UniSteer: Unified Noise Steering for Efficient Human-Guided VLA AdaptationTrains a SAC-style noise actor for frozen pi0, inverting the flow decoder by fixed-point iteration so human teleop corrections become noise-space targets. Real AgileX Piper, 4 tasks: average success 20% → 90% (~66 min training) vs DSRL 55%, DAgger 60%. | Microsoft Research; Tsinghua University; University of Technology Sydney | Residual & steering | ✓ checked |
| 2026-05-09 | ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic ManipulationProcedure-grounded progress labels train a 2B reward VLM. Real stack-bowls soft success with SFT versus reward fine-tuning: 37.5%/62.5% at 5k steps, 70.8%/83.3% at 10k (12 rollouts per evaluation). · code | Renmin University, Zhipu AI, Tsinghua | Rewards & progress | ✓ checked |
| 2026-05-08 | Is the Future Compatible? Diagnosing Dynamic Consistency in World Action ModelsDiagnoses action-state consistency in World Action Models and uses a value-free consensus over sampled futures for test-time selection; improves RoboCasa and RoboTwin 2.0 without extra training or reward models. | Ruan et al. | Test-time search | ✓ checked |
| 2026-05-08 | Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action ModelsA failure action head sharing the VLA backbone gives adaptive CFG-style negative guidance during sampling. LIBERO average pi0.5 96.9% → 98.4% (sim only). | United Imaging Intelligence; UIUC | Advantage-weighted BC | ✓ checked |
| 2026-05-08 | Sword: Style-Robust World Models as Simulators via Dynamic Latent Bootstrapping for VLA Policy Post-TrainingStyle augmentation plus latent scheduled sampling make the world-model simulator robust (Mixed FVD 111.19 vs 198.84 for WoVR). RL success is measured only inside the world model. | Tianjin University; Tsinghua; others | World models & eval | ✓ checked |
| 2026-05-07 | Entropy-Regularized Adjoint Matching for Offline Reinforcement LearningME-AM adds mirror-descent entropy maximization and a mixture behavior prior to adjoint-matching flow-policy Q-learning, countering popularity bias; competitive or superior on sparse-reward continuous control. | Ghanem et al. | Off-policy critics | ✓ checked |
| 2026-05-07 | Adaptive Q-Chunking for Offline-to-Online Reinforcement Learning (AQC)Trains critics for several chunk lengths and picks each state's chunk size by discount-normalized advantage over a per-horizon baseline; OGBench 25-task offline→online success 62→96% vs QC 52→86%. | Peking University; Galbot; University of Toronto | Off-policy critics | ✓ checked |
| 2026-05-06 | When Life Gives You BC, Make Q-functions: Extracting Q-values from Behavior Cloning for On-Robot Reinforcement Learning (Q2RL: Q-Estimation and Q-Gating from BC for RL)Reads Q-values from BC log-probabilities and Monte Carlo values, then gates BC/SAC actions. Best evaluated real checkpoints (20 trials each): peg insertion 0.70→1.00; pipe assembly 0.20→0.75. · code | RAI Institute; Brown; Northeastern | Off-policy critics | ✓ corrected |
| 2026-05-04 | OGPO: Sample Efficient Full-Finetuning of Generative Control PoliciesOff-policy TD critics provide terminal rewards for group-sampled PPO over denoising chains, without GRPO’s variance normalization. On simulated Robomimic Square/Transport, DPPO takes about ten times longer to reach OGPO+’s final success. · code | CMU, UC Berkeley, UW, TRI | Off-policy critics | ✓ checked |
| 2026-05-02 | ★ VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic ModelSamples two chunks; when their DTW distance exceeds a threshold, samples 16 and picks one by knockout tournament under a relative action critic trained on auto-generated preference pairs. LIBERO-LONG, 50 trials/task: pi0 82.8% → 90.6% gated (92.2% ungated); RoboMonkey 56.5%. | University of Sydney; Central South University; USTC; SenseTime Research; HKUST | Test-time search | ✓ checked |
| 2026-05-01 | ★ Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot PoliciesA 16-robot fleet logs rollouts and takeovers with sparse rewards; off-policy RL bootstraps the critic from an uncertainty-adaptive quantile of a distributional value model, with QAM policy extraction. 8 real tasks, mixed success/stage score: 0.95 vs SFT 0.76, RECAP 0.85. | AGIBOT Finch; Shanghai Innovation Institute; Columbia University | Off-policy critics | ✓ checked |
| 2026-05 | What 10 Months in Production Taught Us About the Robotics "Bubble"York Yang essay: pretraining alone is insufficient; capability matures through iterated post-training on deployment data. It says DYNA-1 shipped in April 2025 running 24h+ at 99%+ success on tasks like napkin folding; longest deployment ~10 months of daily use. | Dyna Robotics | Industry loops | ✓ checked |
| 2026-04-30 | LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent ReasoningLAPO jointly optimizes latent reasoning and actions with adaptive reasoning depth. LaST-R1 reports 99.9% average success on simulated LIBERO after one-shot warm-up and evaluates transfer across four real manipulation tasks. | Chen et al. | On-policy PG | ✓ checked |
| 2026-04-24 | Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learning (RINSE: Ranking and INdexing Smooth Examples)Smoothness scores retain top-ranked demonstrations. On simulated RoboMimic Transport mh, spectral arc length selects 50 of 300 demos for 55% success versus 39% with all 300; trajectory-envelope distance scores 46%. | UCLA | Advantage-weighted BC | ✓ checked |
| 2026-04-24 | dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World ModelDiscrete-token masked-diffusion evaluator with failure data. Real AgileX r 0.918 / MMRV 0.02, with better action controllability than video evaluators. | Current Robotics; U Toronto | World models & eval | ✓ checked |
| 2026-04-24 | ★ RL Token: Bootstrapping Online RL with Vision-Language-Action ModelsCompresses frozen pi0.6 embeddings into one 'RL token'; a tiny TD3-style actor-critic then edits the VLA's own sampled action chunk under an L2 penalty, only in each task's critical phase. Screw critical-phase success: 20% → 65%. | Physical Intelligence | Off-policy critics | ✓ checked |
| 2026-04-23 | ★ Hi-WM: Human-in-the-World-Model for Scalable Robot Post-TrainingA human corrects failure-prone policy rollouts inside a learned action-conditioned world model, with rollback and branching from cached states; the corrections post-train the policy. 3 real tasks, DP and pi0: +37.9 points over base, +19.0 over world-model self-training without humans. | Current Robotics; Tsinghua University; Peking University; University of Toronto | Human corrections | ✓ checked |
| 2026-04-21 | ★ FASTER: Value-Guided Sampling for Fast RLA critic predicts action Q-values from raw noise, so only the best seed gets denoised. pi0.5 on five libero_90 tasks versus EXPO: inference FLOPs 37.55 TF → 4.70 TF, comparable success (chart reading). · code | Stanford University | Test-time search | ✓ checked |
| 2026-04-20 | Fisher Decorator: Refining Flow Policy via a Local Transport MapOffline RL refines a flow policy through a residual local transport map with Fisher-information-weighted anisotropic regularization instead of isotropic L2. Provides a local approximation bound under stated assumptions and reports strong results across evaluated offline benchmarks. · code | Cheng et al. | Residual & steering | ✓ corrected |
| 2026-04-20 | OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RLMix-of-Transformers VLA (reasoning, spatial, action experts) with Flow-GSPO: flow matching recast as an SDE and trained with GSPO (Group Sequence Policy Optimization) on action chunks; evaluated on LIBERO and LIBERO-Plus. | Jie et al. | On-policy PG | ✓ checked |
| 2026-04-17 | Rewind-IL: Online Failure Detection and State Respawning for Imitation LearningTraining-free runtime safeguard for a frozen ACT policy: a chunk-consistency failure detector (TIDE, conformal threshold) triggers a respawn to the latest VLM-verified checkpoint state. Real 6-task average: 66.7 → 80.0% unperturbed, 18.3 → 76.7% under adversarial perturbation. | Vanderbilt, Waterloo, Sydney | Test-time search | ✓ checked |
| 2026-04-16 | Bessemer Predicts: Robotics and Physical AIInvestor prediction piece with founder quotes: going from 80% to 99.9% task success is not a linear problem and needs different approaches; data is expensive and capital is the moat; sim-to-real manipulation remains open. | Bessemer Venture Partners | Industry loops | ✓ checked |
| 2026-04-13 | ScoRe-Flow: Complete Distributional Control via Score-Based Reinforcement Learning for Flow MatchingAdds a closed-form score-function drift (computed from the velocity field, no extra network) plus learned noise variance to SDE-based flow-policy RL. Robomimic average success 92.5 vs ReinFlow-S 85.9 and DPPO 75.9; 2.4x faster convergence than ReinFlow on D4RL. | Zhejiang University; SJTU; Shanghai Innovation Institute; Infinigence-AI | On-policy PG | ✓ corrected |
| 2026-04-13 | WM-DAgger: Enabling Efficient Data Aggregation for Imitation Learning with World ModelsNot RL: a world model synthesizes detour-and-recover data in place of human DAgger. Bag pushing reaches 93.3% with 5 demos vs 26.7% for BC. | Peking University; JD Logistics; others | World models & eval | ✓ corrected |
| 2026-04-11 | MoRI: Mixture of RL and IL Experts for Long-Horizon Manipulation TasksGate selects a BC expert or BC-regularized SAC expert per state. Across four real tasks, average success is 97.5% versus BC 37.5%, with 142–288 minutes of online fine-tuning per task. | Huazhong University of Science and Technology | Off-policy critics | ✓ corrected |
| 2026-04-09 | ★ ViVa: A Video-Generative Value Model for Robot Reinforcement LearningBuilds the RECAP value model from the Wan2.2 video generator, jointly predicting future proprioception and a return separating successes [0,1) from failures [1,2). Real, three tasks, two rounds: 80.0% average success vs 63.3% with a VLM value. · code | GigaAI, Sichuan University, Tsinghua University | Rewards & progress | ✓ checked |
| 2026-04-07 | Going Beyond World Models & VLAsPosition post: goals matter more than VLA/world-model labels; GEN-1 has ~99% of parameters trained from scratch on 500k+ hours, targeting 99%+ success from about one hour of robot data. | Generalist AI | Industry loops | ✓ checked |
| 2026-04-04 | Human-Robot Copilot for Data-Efficient Imitation LearningLow-cost bidirectional leader arm collects HG-DAgger correction clips. Simulated Nut Assembly achieves 56% overall success versus 46% for the base policy at 40 training trajectories, evaluated over 50 rollouts. | UC San Diego | Human corrections | ✓ corrected |
| 2026-04-03 | ARM: Advantage Reward Modeling for Long-Horizon ManipulationTri-state (+1/0/-1) relative-progress labels credit recovery; advantage-weighted BC folds towels at 99.4% vs 78.5% with SARM weights and 62.1% plain BC (trial count not reported; CVPR 2026 Workshops). · code | LimX Dynamics, BUPT, Zhejiang University | Rewards & progress | ✓ checked |
| 2026-04-02 | ★ GEN-1: Scaling Embodied Foundation Models to MasteryPretrained on 500K+ hours of human wearable-device data, adapted per task with about 1 hour of robot data; RL is one of five listed ingredients, not detailed. Three charted tasks: 99% average vs 64% for fine-tuned GEN-0 (whole-system comparison). | Generalist AI | Industry loops | ✓ checked |
| 2026-04 | Building a 100-Robot Data Factory Toward Factory-Ready AI100-robot Data Factory 1 trains the Ti0 VLA on teleop demos plus corrections of its own mistakes, then RL post-training from human-scored episodes; running one policy on 100 robots surfaces an 8-hour edge case in about 5 minutes. | Tutor Intelligence | Industry loops | ✓ checked |
| 2026-03-31 | Generalizable Dense Reward for Long-Horizon Robotic Tasks (VLLR: Vision-Language Long-horizon Reward)A VLM progress reward used briefly to warm-start the critic, then policy self-certainty as a bonus; PickUp 97.0% vs 91.8% for sparse-reward FLaRe (sim). | CMU, Amazon Robotics, UT Austin | Rewards & progress | ✓ checked |
| 2026-03-30 | ★ SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement LearningAn 8B video-language model writes chain-of-thought at each timestep, then outputs progress in [-100, 100] as the only reward for off-policy RL from a random policy. 40 sim and real tasks: >=50% success on 24 vs 7 for GPT-5. · code | MIT, RAI Institute | Rewards & progress | ✓ checked |
| 2026-03-27 | VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy DistillationDAgger with a neural teacher: reverse-KL on-policy distillation from an RL-tuned VLA teacher; LIBERO avg of a 1-demo SFT student 48.9 to 87.4 (distill only), 93.4 with GRPO (teacher 93.9). Sim only. · code | HKUST | Human corrections | ✓ checked |
| 2026-03-24 | ★ Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action ModelsControlled study of zero-shot sim-to-real for a VLA trained only in RoboTwin 2.0, isolating which randomization factors matter and adding binary-success GRPO in sim. Real average success: SFT 5.6% → SFT+RL 33.4% → SFT+RL+DR 42.8%. | The Chinese University of Hong Kong, Shenzhen; Shenzhen Loop Area Institute | Sim-to-real | ✓ checked |
| 2026-03-23 | PRM-as-a-Judge: A Dense Evaluation Paradigm for Fine-Grained Robotic AuditingReplaces binary success with dense process-reward-model scoring of trajectory progress (OPD metrics), validated on the RoboPulse diagnostic benchmark and used to audit mainstream policies on long-horizon tasks. | Ji et al. | Rewards & progress | ✓ checked |
| 2026-03-21 | StageCraft: Execution Aware Mitigation of Distractor and Obstruction Failures in VLA ModelsA training-free VLM rearranges clutter and obstructions before a frozen VLA executes. Across three real task domains with pi0.5 and SmolVLA, average success improves by 40 percentage points versus the obstructed workspaces. · code | Pangaonkar et al. | Test-time search | ✓ checked |
| 2026-03-21 | ★ Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action ModelsFine-tunes Bagel on ~50 collected rollouts per task to predict multi-view frames 10 steps ahead plus task success, then runs PPO on short imagined branches from dataset states. Sim LIBERO average 76.8 → 85.9, vs 82.6 for equal-budget online RL. | Nanjing University; Mila / Universite de Montreal | World models & eval | ✓ checked |
| 2026-03-19 | AcceRL: A Distributed Asynchronous Reinforcement Learning and World Model Framework for Vision-Language-Action ModelsAsynchronous VLA RL with plug-in world models. On 4×H200, estimated time to 2,000 steps: 6.5h versus RLinf 15.8h. LIBERO-Spatial: DIAMOND-backed training needs ~200x fewer simulator interactions than model-free AcceRL for comparable return. · code | Lu et al. | World models & eval | ✓ checked |
| 2026-03-19 | ★ Scaling Sim-to-Real VLA Reinforcement Learning with Generative 3D WorldsGenerates 100 physical ManiSkill 3 scenes with a GPT-4o designer and EmbodiedGen, runs PPOFlow on a BridgeV2-pretrained pi0 across them, then transfers zero-shot. Real WidowX, 12 scenes, 240 trials: 21.7% → 75% success, compared only with the imitation starting policy. | Horizon Robotics | Sim-to-real | ✓ checked |
| 2026-03-17 | Conservative Offline Robot Policy Learning via Posterior-Transition ReweightingPTR: reward-free weighted regression for VLA post-training; a transition scorer's identification posterior-to-uniform ratio reweights heterogeneous mixed-quality samples, compatible with diffusion and flow-matching action heads. | Zhang et al. | Advantage-weighted BC | ✓ checked |
| 2026-03-17 | Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language ModelsSeparately LoRA-tunes Qwen3-VL-8B reward models for contrastive progress, absolute progress or completion. After 30 PPO iterations on pi0.5 in ManiSkill3: 56.88% baseline to at most 60.93%, versus 66.87% with environment reward. · code | USC PSI Lab, TRI | Rewards & progress | ✓ checked |
| 2026-03-16 | Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learning (OmniReset)OmniReset: diverse programmatic simulator resets let large-scale RL learn long-horizon dexterous manipulation without hand-crafted curricula or demos; scales with compute and transfers to real robots above baselines. · code | University of Washington; Microsoft Research | Sim-to-real | ✓ checked |
| 2026-03-16 | ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior PriorsDSRL + FastTD3 noise steering in massively parallel sim with binary reward only. On an 8-task AnyTask mean: ExpertGen 85.3 vs DP prior 40.3, residual RL 53.7, DSRL-SAC 37.7. | RAI Institute; UT Austin; Sony AI | Residual & steering | ✓ checked |
| 2026-03-16 | Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation (SimDist)Pretrain a latent world model in simulation; freeze encoder, reward and value, adapt dynamics on real data and plan with MPPI. Go2 slippery slope: 14/15 successes across three speeds after 35.7 minutes, versus IQL/RLPD 0/15. · code | UT Austin; University of Washington | World models & eval | ✓ checked |
| 2026-03-16 | ★ You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise VectorReplaces per-chunk Gaussian noise in a frozen diffusion or flow policy with one fixed noise vector found by black-box search. Real Franka block/cube pick: 80 → 98% success on 50 held-out episodes after 150 search episodes (~30 min). · code | Robotics and AI Institute; Arizona State University; Northeastern University | Black-box search | ✓ checked |
| 2026-03-14 | Chunk-Guided Q-LearningOffline RL method (CGQ) regularizing a single-step critic toward a chunk-level critic trained with temporally extended backups; tighter optimality bounds, strong long-horizon OGBench results. · code | Song et al. | Off-policy critics | ✓ checked |
| 2026-03-14 | SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness OptimizationGRPO fine-tuning of a VLA with a hybrid reward: sparse task success plus dense jerk-based intrinsic smoothness term; 13.8% smoother than standard RL on LIBERO. | Li et al. | On-policy PG | ✓ checked |
| 2026-03-14 | REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement LearningDPPO on a humanoid diffusion planner, alternating with PPO on the low-level controller. Sim 50-70% → >90%. Booster T1 hardware 70% / 50% / 75%. | Georgia Tech | On-policy PG | ✓ checked |
| 2026-03-13 | Beyond Imitation: Reinforcement Learning Fine-Tuning for Adaptive Diffusion Navigation PoliciesGRPO (value-network-free) fine-tuning of an imitation-pretrained diffusion navigation policy (NavDP) with selective layer updates; Isaac Sim PointGoal unseen-scene success 52.0% to 58.7%, zero-shot transfer to a real Unitree Go2 quadruped. | Sheng et al. | On-policy PG | ✓ checked |
| 2026-03-13 | RoboStereo: Dual-Tower 4D Embodied World Models for Unified Policy OptimizationRGB/pointmap 4D world model supports test-time action verification and imagined GRPO. Combining all three components lifts OpenVLA-OFT mean MimicGen success 27.7% to 54.0%; online GRPO reaches 29.2%, under the same 128-trajectories-per-task interaction budget. | Tsinghua; X Square Robot; HKUST | World models & eval | ✓ checked |
| 2026-03-12 | RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic TasksVLM agent unifies collection, learning and execution; Entangled Action Pairs enable self-resetting collection. Vanity-table success reaches 30% versus a 5% end-to-end VLA baseline after five rounds (25 percentage points); human collection time falls 53.7%. · code | Li et al. | Human corrections | ✓ corrected |
| 2026-03-12 | ★ HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot PoliciesFreezes a sim piano rollout, corrects lateral finger joints from MIDI errors, then trains bounded residual RL with MIDI rewards. Five real songs: mean best-validation key-press F1 77.6 versus open-loop simulation trajectory 41.8 (chart reading). · code | Stanford University; Amazon FAR | Sim-to-real | ✓ checked |
| 2026-03-12 | ★ Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement LearningFine-tunes a VLA on each new task in sequence with GRPO on sparse 0/1 success and a rank-32 LoRA adapter, with no replay or regularizer. Sim LIBERO-Spatial: AVG 81.2, forgetting 0.3, vs 7.3 and 40.9 with full fine-tuning. · code | UT Austin, UCLA, NTU, Sony AI | On-policy PG | ✓ checked |
| 2026-03-11 | World2Act: Latent Action Post-Training from World Model DynamicsFrozen GR00T-N1.6-ft plus a residual policy trained in action latents contrastively aligned with world-model latents. RoboCasa success 70.1% to 72.6%; real-robot average improves 6.7 points over the same fine-tuned baseline. · code | MBZUAI | World models & eval | ✓ checked |
| 2026-03-10 | ★ Update-Free On-Policy Steering via VerifiersTrains a small verifier on labeled evaluation rollouts of the deployed policy, then reranks 10 samples from the frozen diffusion or flow policy. Real ALOHA, 5 tasks, 20 trials each: mean success 38% → 87% (chart reading). | University of Toronto; Google DeepMind; University of Alberta; Harvard University; UT Austin | Test-time search | ✓ checked |
| 2026-03-10 | Quality over Quantity: Demonstration Curation via Influence Functions for Data-Centric Robot Learning (QoQ)Hessian-free max-cosine influence against a small validation set. Robomimic Can-paired 99.2% vs 64.8% with all data; real cabinet opening 25.0% → 43.3%. | KAIST; UC Berkeley; NYU | Advantage-weighted BC | ✓ checked |
| 2026-03-10 | DexHiL: A Human-in-the-Loop Framework for Vision-Language-Action Model Post-Training in Dexterous ManipulationArm-plus-hand takeovers for a dexterous VLA (Being-H0.5), with intervention samples reweighted to 50% of the training distribution; Tissue Extraction after three rounds 19/20 vs DAgger* 16/20 vs data-matched offline 15/20. | CASIA; Shanghai Jiao Tong University | Human corrections | ✓ checked |
| 2026-03-10 | ★ From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL FinetuningFreezes a diffusion/flow BC prior, trains a residual on the same noise with TD3+BC, then executes the best-Q sample. Real UR5 BeltAssembly, 30 trials/checkpoint: 56.67% → 93.33% after ~420 training episodes (chart reading). · code | Stanford University | Residual & steering | ✓ checked |
| 2026-03-09 | Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLAIMCopilot RL skills assist teleoperation and serve as MoDE-VLA execution primitives; residual expert modules add force and tactile signals to pi0. Mean success across four real tasks is 34% versus pi0’s 15%. | SJTU / Shanghai AI Lab / NUS | Human corrections | ✓ checked |
| 2026-03-09 | OmniGuide: Universal Guidance Fields for Enhancing Generalist Robot PoliciesTest-time guidance of VLA action sampling via differentiable attractor/repeller energy fields from 3D foundation models, semantic reasoners and human pose estimators; improves success and safety in sim and real. · code | UPenn | Test-time search | ✓ checked |
| 2026-03-09 | ★ PlayWorld: Learning Robot World Models from Autonomous PlayTrains a video world model on 30 hours of autonomous robot play, then improves a diffusion policy inside it with DSRL. Real towel-folding success peaks at 75% from 10% (2000 steps); human-demo-model peak: 40% (chart reading). · code | Princeton University | World models & eval | ✓ checked |
| 2026-03-09 | Interactive World Simulator for Robot Policy Training and EvaluationPer-task consistency-model world model from about 6 h of play. Evaluation r 0.8455-0.9908 across checkpoints of DP/ACT/pi0/pi0.5; runs 10+ minutes at 15 FPS on a 4090. · code | Columbia; TRI; Amazon; UIUC | World models & eval | ✓ checked |
| 2026-03-09 | AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World ModelsOffline GRPO scores sampled action chunks using V-JEPA2 latent distances to subgoals and goals. LIBERO simulation average: 93.0% SFT to 97.0% after post-training without online environment rollouts. | INFIFORCE; HUST; others | World models & eval | ✓ checked |
| 2026-03-05 | Critic in the Loop: A Tri-System VLA Framework for Robust Long-Horizon ManipulationTri-system hierarchy: VLM planner, fast VLA executor, and lightweight visual critic that routes control, triggering VLM replanning on failures or stalls in long-horizon manipulation. | Yi et al. | Test-time search | ✓ checked |
| 2026-03-05 | RoboPocket: Improve Robot Policies Instantly with Your PhoneRobot-free DAgger-style loop: an iPhone handheld gripper previews the policy's predicted trajectory in AR and streams corrections to online finetuning; in a four-user distributed test, Scene 2 rose 0.42 to 0.82 with 12 corrections. | Shanghai Jiao Tong University et al. | Human corrections | ✓ checked |
| 2026-03-05 | Latent Policy Steering through One-Step Flow PoliciesLatent actor picks the noise for a one-step MeanFlow policy, and the action-space Q-gradient is backpropagated through the generator. Real average: BC-FM 25/80, DSRL 28/80, LPS 45/80. · code | Yonsei; Microsoft Research | Residual & steering | ✓ checked |
| 2026-03-04 | Force-Aware Residual DAgger via Trajectory Editing for Precision Insertion with Impedance Control (TER-DAgger)Force-mismatch trigger plus trajectory-edited residual corrections under impedance control for insertion; average success 77.2% vs 40.0% for the best baseline (Finetune) over two simulated and three real insertion tasks. | Xi'an Jiaotong-Liverpool University | Human corrections | ✓ checked |
| 2026-03-03 | How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human PreferenceHuman quality scores train a per-step reward, and a residual is fitted with exp(beta*r)-weighted BC. Apple 60% → 100%, while IQL-weighted BC scored 0%. · code | UC Berkeley | Advantage-weighted BC | ✓ checked |
| 2026-03-02 | SEAR: Sample Efficient Action Chunking Reinforcement LearningSEAR combines a multi-horizon causal critic with random receding-horizon replanning for off-policy action-chunk RL. Outperforms prior methods on the twenty hardest MetaWorld tasks; adding it to QC improves OGBench cube-triple offline-to-online learning. | Nagy et al. | Off-policy critics | ✓ checked |
| 2026-03-02 | ★ Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory ComparisonsQwen3-VL-4B reward model with progress, success and preference heads, trained on RBM-1M including failures. Real DSRL on frozen pi0: 20% → 85% in ≤45 min (10k steps) vs 55% with RoboReward; 0 false-positive successes vs 45. · code | USC, UT Dallas, MIT, University of Washington, Ai2, NVIDIA | Rewards & progress | ✓ checked |
| 2026-03-02 | π-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAsCritic-free, likelihood-free step-wise ranking on Flow-SDE rollouts. pi0 LIBERO 57.6 → 90.5, trailing piRL PPO at 96.0. ManiSkill OOD 50.4 vs PPO 39.3. · code | GigaAI, CASIA, UCAS, Tsinghua | On-policy PG | ✓ checked |
| 2026-02-28 | Keyframe-Guided Structured Rewards for Reinforcement Learning in Long-Horizon Laboratory RoboticsExtracts kinematics-aware keyframes from demos, diffusion-predicts stage targets, builds geometric progress rewards for online VLA fine-tuning; 82% after 40-60 min on four lab tasks vs HG-DAgger 42%, HiL-ConRFT 47%. | Qiu et al. | Rewards & progress | ✓ checked |
| 2026-02-26 | SPARR: Simulation-based Policies with Asymmetric Real-world Residuals for AssemblyFrozen sim PPO assembly policy plus a real-world image and force residual trained RLPD-style, seeded with 20 successes from the prior. Reaches 95–100% on 10 AutoMate tasks after 0.5 h of residual training per task, without human takeover. | NVIDIA; U Washington | Residual & steering | ✓ corrected |
| 2026-02-25 | FlowCorrect: Efficient Interactive Correction of Generative Flow Policies for Robotic ManipulationVR 'nudges' train a tiny LoRA patch on a frozen flow policy; hard Pick-and-Place cases 0/40 to 32/40. | Karlsruhe Institute of Technology | Human corrections | ✓ checked |
| 2026-02-24 | ★ IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic ManipulationTrains pi0.5 with 60 demos per task, offline advantage-weighted flow matching from a learned critic, then human-in-the-loop correction rounds with the same weighting. Four real long-horizon tasks, 20 trials each: 18.8% → 40.0% → 85.0%. | Zhejiang University; Torch Kernel | Advantage-weighted BC | ✓ checked |
| 2026-02-23 | PhysMem: Scaling Test-Time Memory for Embodied Physical ReasoningA VLM planner tests physical hypotheses before reusing them as memory, without weight updates. Simulated medium-difficulty brick insertion: 76% success versus 61% free-form memory, 53% no memory and 23% direct experience retrieval. | UC San Diego / Stanford | Test-time search | ✓ checked |
| 2026-02-23 | AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic ManipulationCouples world model, action expert and force predictor as flow-matching DiTs; online adaptive learning alternates action generation and future imagination to update modules under visual/physical domain shift. | Yuan et al. | World models & eval | ✓ corrected |
| 2026-02-23 | ★ What Matters for Simulation to Online Reinforcement Learning on Real RobotsEmpirical study of continuing sim-pretrained SAC on real robots: keep or warm-start replay, update the actor once per 20 critic steps at LR 1e-5, restore optimizer state. Franka vision grasping reaches near-perfect success in about ten real minutes. · code | ETH Zurich; Google DeepMind | Sim-to-real | ✓ checked |
| 2026-02-22 | LLMs Can Learn to Reason Via Off-Policy RLOAPL: off-policy LLM post-training that embraces lagged inference policies (400+ gradient steps) instead of importance-sampling; beats GRPO+IS on competition math, matches DeepCoder with 3x fewer generations. · code | Ritter et al. | General RL | ✓ checked |
| 2026-02-22 | ★ TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for RoboticsReads a frozen Qwen3-VL-8B's log-probability of 'True' that a video prefix completes the task as a training-free progress curve. Real ManiRewardBench progress subset (113 tasks, 497 episodes): VOC 0.942 vs GVL 0.316 on the same model, trained Robometer-4B 0.858. · code | University of Washington, Allen Institute for AI, Amazon, UNC Chapel Hill | Rewards & progress | ✓ checked |
| 2026-02-21 | RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot LearningFilters generated robot videos by comparing their motion with simulator replays of annotated actions. Relative success gains over real-data-only: +70.1% GR-1 Tabletop (300 demos), +16.1% DexMimicGen pretraining, +179.9% real ALLEX humanoid. | Kim et al. | World models & eval | ✓ corrected |
| 2026-02-19 | Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMsVCPO scales the learning rate by effective sample size and adds a critic-free minimum-variance baseline for stale-rollout policy gradients; stable even 128 steps off-policy, and matches synchronous performance 2.5x faster on a long-horizon tool-use task. · code | Huang et al. | General RL | ✓ checked |
| 2026-02-17 | World Action Models are Zero-shot Policies (DreamZero)14B video-action diffusion model; optimized Flash inference reaches ~7Hz on GB200. Real-robot task progress exceeds 2× the best tested pretrained VLA; 10–20 minutes of video-only cross-embodiment data yields >42% relative improvement on unseen tasks. · code | Ye et al. | World models & eval | ✓ corrected |
| 2026-02-15 | ★ WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RLRuns GRPO entirely inside an action-conditioned Wan2.2-5B world model, starting imagined episodes near failure keyframes and refreshing the model with policy-aligned data to curb hallucination. Real Franka: 51.1 → 80.0 average (WMPO 64.5), no online robot RL. · code | CASIA; Zhongguancun Academy; Tsinghua University | World models & eval | ✓ checked |
| 2026-02-13 | Dual-Granularity Contrastive Reward via Generated Episodic Guidance for Efficient Embodied RL (DEG)Video generator adapted with a small set of expert videos produces per-episode task guidance. Coarse- and fine-grained contrastive latent rewards drive manipulation RL without reward annotations, evaluated across 18 simulated and real tasks. | Liu et al. | Rewards & progress | ✓ corrected |
| 2026-02-13 | ★ Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA ModelsCo-trains the VLA on real plus MimicGen sim demos, then runs PPO in a rough ManiSkill3 task copy with a weighted real-demo BC loss; no real-robot RL. Real average success, pi0.5: SFT co-training 45.9 → 66.2. · code | Tsinghua University, Harbin Institute of Technology, Peking University, Carnegie Mellon University, Shanghai AI Laboratory, Zhongguancun Academy | Sim-to-real | ✓ checked |
| 2026-02-13 | ★ ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-TrainingLearns a chunk-level Q of the current pi0.5 policy by TD bootstrapping with a pessimistic ensemble, instead of Monte Carlo state values, to weight policy updates. Phone Assembly, 60 trials, 10.0% BC start: ALOE 95.0, AWR 80.0. | AgiBot, HKUST, Fudan University, Nanjing University | Off-policy critics | ✓ checked |
| 2026-02-12 | ★ VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World ModelGrounds a world model on 50 real pi0.5 rollouts per task each iteration, labels synthetic rollouts with a strict Qwen3-VL-4B, and fine-tunes on real plus synthetic successes. 5 real DROID tasks: 0.460 → 0.868 vs filtered BC 0.752. · code | Stanford University; Tsinghua University | World models & eval | ✓ checked |
| 2026-02-12 | ★ Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment (CoVer / CoVer-VLA)Keeps pi0 frozen, samples K instruction rephrasings times M action chunks, and picks one with a CLIP-style contrastive verifier trained on BridgeV2. SIMPLER red-teaming, ID/OOD: pi0 41.5/29.7, pi0 fine-tuned on augmented data 44.0/48.7, pi0 (rephrase) + CoVer 65.5/62.0. · code | Stanford University; NVIDIA Research | Test-time search | ✓ checked |
| 2026-02-12 | Accelerating Robotic Reinforcement Learning with Agent Guidance (AGPS: Agent-guided Policy Search)Replaces the HIL-SERL human with a failure-detector-gated VLM agent; USB insertion 100% at 8 min with zero human interventions. | Peking University; PKU-PsiBot Joint Lab | Human corrections | ✓ checked |
| 2026-02-12 | GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement LearningRECAP-style conditioning plus world-model future latents and value. Box Packing about 60 pretrain / 75 AWR / 60 RECAP / 95% RAMP (bar readings). | GigaAI | Advantage-weighted BC | ✓ checked |
| 2026-02-11 | ★ What Makes Value Learning Efficient in Residual Reinforcement Learning?Speeds up residual SAC with 20K base-policy replay transitions and LayerNorm in the critic. ManiSkill sim PegInsertionSide reaches 90% success around 0.3M environment steps versus roughly 2.3M for Policy Decorator (chart reading). · code | Nanyang Technological University; Mila / Universite de Montreal; National University of Singapore | Residual & steering | ✓ checked |
| 2026-02-11 | ★ RISE: Self-Improving Robot Policy with Compositional World ModelReplaces online physical rollouts with a learned action-conditioned video model and a pi0.5-based value model that scores imagined chunk outcomes, driving advantage-conditioned training of pi0.5. Real dual-arm, 20 trials per task: box closing 35% → 95% vs RECAP 60%. · code | CUHK; Kinetix AI; HKU; Shanghai Innovation Institute; Horizon Robotics; Tsinghua University | World models & eval | ✓ checked |
| 2026-02-10 | Where Autonomy Works: Evaluating Robot Capabilities in 2026Survey of reported robot reliability: warehouse picking 99%+ (per roboticist interviews), DYNA-1 napkin folding 99.4% over a 24-hour run, Figure package handling 95% (claimed, one-hour demo); autonomy works where environments are controlled or forgiving. | Epoch AI | Industry loops | ✓ checked |
| 2026-02-10 | Preference Aligned Visuomotor Diffusion Policies for Deformable Object Manipulation (RKO)Fine-tunes diffusion policies on preferred/dispreferred cloth-folding demonstrations, combining KTO sample labels with RPO similarity weighting. RKO leads 4/9 real garment/preference settings; Trousers Pref 1 normalized folding score is 0.910 versus vanilla DDPM’s 0.701. | KTH; INCAR Robotics | Advantage-weighted BC | ✓ corrected |
| 2026-02-10 | SERNF: Sample-Efficient Real-World Dexterous Fine-Tuning with Normalizing-Flow PoliciesNormalizing-flow chunk policy (exact likelihood) with a chunk critic; real dexterous scissors cut success 0.16 → 0.84 with 60 online rollouts. | ETH Zurich | Off-policy critics | ✓ checked |
| 2026-02-09 | χ0 (kai0): Resource-Aware Robust Manipulation via Taming Distributional InconsistenciesOpen garment-folding pipeline: model soup plus stage-aware advantage (RECAP-style, eps = 0.3) plus train-deploy alignment on pi0.5. Task A success about 27% → 97% (bar readings, 30 trials). · code | Kinetix AI / OpenDriveLab | Advantage-weighted BC | ✓ checked |
| 2026-02-09 | ★ TwinRL (TwinRL-VLA): Digital Twin-Driven Reinforcement Learning for Real-World Robotic ManipulationSFTs Octo-Small on 30 real demos plus twin trajectories, runs RL in a phone-scanned twin to seed the real buffer, then real HIL-SERL-style RL. 4 real Franka tasks: near-100% success, at least 30% less real interaction than HIL-SERL. · code | Peking University; HKUST; Simplexity Robotics; Tsinghua University | Sim-to-real | ✓ checked |
| 2026-02-08 | RLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AIInfrastructure, not an algorithm: a fully asynchronous system for real-robot online learning. Peg insertion (CNN, RLPD) converges in about 1500 s vs 8000+ s with a synchronous pipeline; pi0 Pick-and-Place 39/60 → 58/60 with HG-DAgger. · code | Tsinghua University and others | Off-policy critics | ✓ checked |
| 2026-02-07 | VGAS: Value-Guided Action-Chunk Selection for Few-Shot Vision-Language-Action AdaptationA chunk critic with expected-max backup and geometric regularization reranks eight SmolVLA samples. Simulated LIBERO, five demos/task: 39.8% BC to 49.0%; plain Q-chunking scores 37.9%. · code | AAII, University of Technology Sydney | Test-time search | ✓ checked |
| 2026-02-06 | DreamDojo: A Generalist Robot World Model from Large-Scale Human VideosWorld model pretrained on 44k h of human video with latent actions. Checkpoint ranking r = 0.995 / MMRV 0.003 on AgiBot fruit packing (human-scored). · code | NVIDIA and collaborators | World models & eval | ✓ checked |
| 2026-02-06 | World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA PolicyCosmos world model with a built-in reward head, trained on success plus near-success data and refreshed with policy rollouts. Real Place Cup 13.3% → 50.0% after 2 loops. | Show Lab, NUS | World models & eval | ✓ corrected |
| 2026-02-05 | RL-VLA³: A Flexible and Asynchronous Reinforcement Learning Framework for VLA TrainingFully asynchronous distributed RL framework for VLA post-training (decoupled simulation, inference, training; dynamic batching, environment sharding); up to 85.2% higher throughput than synchronous baselines, scaling 8 to 256 GPUs. · code | Sun et al. | On-policy PG | ✓ checked |
| 2026-02-03 | Reparameterization Flow Policy OptimizationRFO trains flow policies with reparameterization policy gradients through ODE action generation and differentiable dynamics. On simulated Soft Jumper, reward is 3023.96 versus 1597.99 for the best tested baseline, SAPO (about 1.89×). | Zhong et al. | On-policy PG | ✓ corrected |
| 2026-02-03 | ★ VLS: Steering Pretrained Robot Policies via Vision-Language ModelsTraining-free: a VLM names task objects, SAM/DINOv2 features plus depth give 3D keypoints, and VLM-generated reward gradients steer frozen diffusion/flow denoising. CALVIN (sim) movable objects: 0.94 versus DynaGuide 0.26 and base 0.13 (chart reading). · code | University of Washington; AI2; University of Oxford; NUS | Test-time search | ✓ checked |
| 2026-02-02 | Maximum Likelihood Reinforcement LearningUnder binary terminal feedback, expected-reward RL approximates maximum likelihood to first order. MaxRL approaches likelihood optimization as sampling compute grows; math-reasoning evaluations report up to 20× test-time sampling efficiency over GRPO with a perfect verifier. · code | CMU; ICML 2026 | General RL | ✓ corrected |
| 2026-02-02 | ★ RFS: Reinforcement Learning with Residual Flow Steering for Dexterous ManipulationOne PPO policy outputs both the starting noise for a frozen flow-matching base and an additive residual, so it can switch modes and leave the demo manifold. Sim, 6 IsaacLab dexterous tasks: 0.861 vs base 0.250, DSRL-only 0.483. | University of Washington; Amazon FAR | Residual & steering | ✓ checked |
| 2026-02-02 | World-Gymnast: Training Robots with Reinforcement Learning in a World ModelGRPO trains a VLA inside WorldGym with GPT-4o rewards. Real eggplant-to-blue-sink success: 4% SFT to 72%; World-Gymnast outperforms RL trained in SIMPLER on three of four evaluated real tasks. · code | NYU; NYU Shanghai; UC Berkeley | World models & eval | ✓ checked |
| 2026-02-02 | ★ Flow Policy Gradients for Robot Control (FPO++)Flow policy gradient using exp(old CFM loss - new CFM loss) as the PPO ratio, clipped per (tau, eps) sample, with zero-noise test-time sampling. Zero-sampling alone lifts the sim Robomimic Can base policy from 10.00% to 71.35% before RL. · code | Amazon FAR, UC Berkeley, Stanford, HKU, CMU | On-policy PG | ✓ checked |
| 2026-02-01 | TQL: Scaling Q-Functions with Transformers by Preventing Attention CollapseAttention-entropy regularization lets transformer critics improve with size (43% gain from 0.4M to 26M); 25-task average 40±7 vs floq 34±7. · code | Stanford University | Off-policy critics | ✓ corrected |
| 2026-01-31 | Green-VLA: Staged Vision-Language-Action Model for Generalist RobotsFive-stage VLA curriculum (VLM, grounding, multi-embodiment pretraining on 3,000 h, embodiment adaptation) ending in an RL policy-alignment stage; adds progress prediction and anomaly detection at inference. · code | Apanasevich et al. | Off-policy critics | ✓ checked |
| 2026-01-31 | SA-VLA: Spatially-Aware Flow-Matching for Vision-Language-Action Reinforcement LearningRL fine-tuning of flow-matching VLAs that preserves spatial grounding: spatial tokens fused with visual tokens, geometric progress dense rewards, SCAN spatially-conditioned exploration; better zero-shot spatial generalization on LIBERO/LIBERO-PLUS. · code | Wuhan University; A*STAR IHPC; NUS; CAS; NTU | On-policy PG | ✓ checked |
| 2026-01-28 | Reinforcement Learning via Self-DistillationSDPO distills a feedback-conditioned LLM’s next-token predictions back into its policy. On nine selected very-hard LiveCodeBench questions, test-time self-distillation matches best-of-k discovery probability with about 3× fewer attempts. · code | Hübotter et al. | General RL | ✓ corrected |
| 2026-01-28 | One Step Is Enough: Dispersive MeanFlow Policy Optimization (DMPO), renamed OGPO: One-Step Generative Policy Optimization for Real-Time Robot ControlOne-step MeanFlow policy with a dispersive loss, fine-tuned by PPO on a one-step Gaussian chain plus BC regularization. Square 0.83 at 1 NFE vs DPPO 0.78 at 20 NFE. Without the BC term (v1 ablation), success collapses to near zero. · code | Sun Yat-sen University | On-policy PG | ✓ checked |
| 2026-01-27 | Real-Time Robot Execution with Masked Action ChunkingREMAC (ICLR 2026): LoRA fine-tuning with masked action prefixes and a self-conditioned curriculum adapts pretrained policies to asynchronous inference. Prefix-preserved sampling maintains chunk continuity; reports higher completion rates and faster execution without added inference overhead. · code | Wang et al. | Test-time search | ✓ checked |
| 2026-01-27 | Self-Distillation Enables Continual LearningSDFT: a demonstration-conditioned model acts as its own teacher for on-policy distillation, learning new skills from demos with less forgetting than supervised fine-tuning (foundation models, not robot-specific). | Shenfeld et al. | General RL | ✓ checked |
| 2026-01-22 | Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and PlanningVideo-model policy that is also a world model and value function. Best-of-N planning refit on 648 real rollouts raised hard ALOHA tasks from 59.5 to 72.0 average score. Mainly a policy architecture. · code | NVIDIA; Stanford | Test-time search | ✓ checked |
| 2026-01-20 | ★ Q-learning with Adjoint Matching (QAM)Trains a second flow toward pi_beta(a|s)·exp(tau·Q(s,a)) via adjoint matching, using the critic's action gradient without backpropagating through the sampler. Offline RL over 50 OGBench tasks (sim): QAM 44 vs ReBRAC 40, FQL 36, FBRAC 11. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2026-01-15 | DeFlow: Decoupling Manifold Modeling and Value Maximization for Offline Policy ExtractionOffline RL with a flow-matching behavior policy plus a lightweight refinement module in a data-derived trust region, avoiding ODE-solver backprop; strong OGBench results and efficient offline-to-online adaptation. | Peking University | Off-policy critics | ✓ checked |
| 2026-01-12 | Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulation (FARL)World-model safety critic triggers an offline-trained recovery policy during PPO-based Uni-O4 refinement. Across three real Franka tasks, failure state-action pairs fall by 73.1% versus Uni-O4 (mean per-task reduction over 50 training episodes). | Shanghai Qi Zhi Institute, SJTU, Tsinghua and others | On-policy PG | ✓ corrected |
| 2026-01-06 | ★ SOP: A Scalable Online Post-Training System for Vision-Language-Action ModelsTen real AgiBot G1 robots run pi0.5 with human takeovers; a cloud learner mixes on-policy data with old demos, publishing weights every 25 steps. 3 h on-policy interaction: 0.571 → 0.800; 80 more demo hours: 0.576 → 0.612. | AgiBot Research; Shanghai Innovation Institute | Human corrections | ✓ checked |
| 2026-01-05 | CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk DecodingPredicts stop/progress as extra action dimensions; a VLM checks for failure near subtask ends, then backtracks and retries with MBR consensus over eight chunks. Simulation: LIBERO-Plus pi0.5 65.0% to 86.0%; LIBERO 96.9% to 98.5%. · code | University of Oxford; University of Cambridge | Test-time search | ✓ checked |
| 2026-01-02 | V-VLAPS: Value-Guided Planning for Vision-Language-Action ModelsAn offline-trained value head guides Octo-prior MCTS using the LIBERO simulator. Matches value-free VLAPS at 600 seconds of search per episode; at 1800 seconds, adds 6 points on LIBERO-Object and 4 on LIBERO-10. | Ren et al. | Test-time search | ✓ checked |
| 2026-01-02 | ★ RoboReward: General-Purpose Vision-Language Reward Models for RoboticsFine-tunes Qwen3-VL to score rollouts 1-5 on a rubric, adding counterfactual-relabeled and clipped negatives; in sim, reward accuracy tracks RL success (r=0.83). Real WidowX DSRL, open drawer, 20 trials: base 10%, RoboReward 8B 80%, human-reward oracle 90%. | Stanford University, UC Berkeley | Rewards & progress | ✓ checked |
| 2025-12-30 | Real-world Reinforcement Learning from Suboptimal Interventions (SiLRI)Per-state Lagrange multiplier keeps the policy near human interventions only where humans are consistent; reaches 90% at least 50% faster than HIL-SERL. · code | Beijing Innovation Center of Humanoid Robotics and others | Off-policy critics | ✓ checked |
| 2025-12-29 | ★ Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic ManipulationA VLM reward model predicts signed progress 'hops' from multi-view images, fused into a potential used as policy-invariant shaping, not raw reward. 8 real tasks: 95.2% final success vs 68.0% with sparse reward (ConRFT) and 9.8% for BC. · code | Peking University, BAAI, University of Sydney, CASIA | Rewards & progress | ✓ checked |
| 2025-12-18 | ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement LearningDecomposes tasks into motion-planner-linked skills, imitates data generated from 10 human demos, then fine-tunes pose prediction, skills and termination with online adaptation and RL; over 80% on every Robosuite visuomotor task tested, 84.6% vs 44.7% for the HSP baseline. | Zhou et al. | Residual & steering | ✓ checked |
| 2025-12-18 | Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL FinetuningPretrain BC to match a posterior over demonstrator actions so later RL can explore; Libero 16 tasks at 1000 rollouts best-of-N: BC 33.1, PostBC 55.1. | UC Berkeley; Stanford | Off-policy critics | ✓ checked |
| 2025-12-18 | ★ PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot PoliciesTurns a 2-5 minute monocular video into a splat-rendered IsaacSim twin for policy evaluation, with brief co-training to align visuals. Average Pearson r = 0.9 across 6 unseen scenes; in their tests the Ctrl-World video model mis-ranked policies (MMRV 0.22). · code | University of Washington; Princeton University; UC Berkeley; Stanford University; Toyota Research Institute; USC; Cornell University; Physical Intelligence | World models & eval | ✓ checked |
| 2025-12-16 | EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action ModelsTest-time GRPO with a pretrained progress critic (VLAC) instead of an oracle reward. LIBERO average 89.2 → 95.8 over the paper's own OpenVLA-OFT baseline; 1-shot SFT 43.6 → 61.3. | NUS Show Lab | On-policy PG | ✓ checked |
| 2025-12-12 | Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and LimitsEquivariant noise-steering actor and critics on equivariant diffusion policies. Square D2 peak success: base 0.275, DSRL 0.552, Equi-DSRL 0.644; but strict equivariance was lowest on Stack D1 (0.728 vs DSRL 0.740). Plain DSRL's Q diverged in 3/5 Lift seeds. | Yonsei; UC Berkeley | Residual & steering | ✓ checked |
| 2025-12-11 | ★ Evaluating Gemini Robotics Policies in a Veo World SimulatorRuns Gemini Robotics On-Device inside an action-conditioned, multi-view Veo 2 world model; humans score the videos. Eight nominal checkpoints: Pearson 0.88, MMRV 0.03 against real success, with under-predicted absolute rates (chart reading). | Google DeepMind | World models & eval | ✓ checked |
| 2025-12-11 | Decoupled Q-ChunkingLong-chunk value backups train a distilled partial-chunk critic for a shorter-output policy. Across six hard offline goal-conditioned OGBench environments, aggregate score is 82 versus n-step 68, SHARSA 44 and Q-chunking 25. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2025-12-08 | Scalable Offline Model-Based RL with Action Chunks (MAC)Action-chunk dynamics model predicts future state from an action sequence, reducing compounding error in model-based value expansion; rejection-samples a behavior policy; best offline MBRL on datasets up to 100M transitions. | Park et al. | World models & eval | ✓ checked |
| 2025-12-02 | Training Diffusion Policies via Prior-Mapping Co-EvolutionGoRL: online RL for generative policies by optimizing a tractable latent prior, then consolidating into an expressive prior-to-action mapping; HopperStand return over 870, 3x strongest baseline (ICML 2026). · code | Zhang et al. | General RL | ✓ checked |
| 2025-12-02 | ★ Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach (TACO)Samples M chunks from a flow VLA and runs the one with the highest pseudo-count in the fine-tuning data, from a small Coin Flipping Network on internal features. RoboTwin 2.0 (pi0.5, 41 tasks, M=50): 59.3 → 64.0. · code | TeleAI (China Telecom); USTC; Tsinghua University; HKUST | Test-time search | ✓ checked |
| 2025-12-01 | GR-RL: Going Dexterous and Precise for Long-Horizon Robotic ManipulationOffline distributional chunk critics filter demonstrations, followed by mirror-symmetry augmentation and latent-noise online RL around a frozen base. Real shoe lacing improves from 45.7% for GR-3 behavior cloning to 83.3%. | ByteDance Seed | Off-policy critics | ✓ checked |
| 2025-12 | State of Robot Learning — December 2025Practitioner essay: nearly all robot learning today is BC plus laborious DAgger-style data iteration, and real-world RL remains hard; in the author's reading, pi*0.6's advantage-weighted method gives minor gains over BC and many tasks still needed human corrections. | Industry loops | ✓ checked | |
| 2025-12 | ★ Open-World Dexterity and Live Demos around the World (DYNA-1i)Post-trains a VLA on 'only tens of hours' of office data to stay reliable in unseen venues; no objective or update rule is given. 30-minute shirt-folding trials (~40/hour): 22 consecutive failure-free folds in the office, 20 in unseen venues. | Dyna Robotics | Industry loops | ✓ checked |
| 2025-11-30 | Goal-Driven Reward by Video Diffusion Models for Reinforcement LearningFine-tunes a pretrained video diffusion model to generate goal videos; rewards combine video-level latent alignment with them and a forward-backward-representation reward toward a CLIP-selected goal frame; evaluated on Meta-World and Distracting Control Suite. · code | Wang et al. | Rewards & progress | ✓ checked |
| 2025-11-27 | ★ Beyond Success: Refining Elegant Robot Manipulation from Mixed-Quality Data via Just-in-Time InterventionTrains an offline Cal-QL critic on frozen VLA features; reranks chunks only when Q deviates from its moving average. LIBERO-Elegant, SmolVLA: ESR 49.8 → 67.2, versus 53.8 with guidance every step (chart reading). | Jilin University; Microsoft Research Asia | Test-time search | ✓ checked |
| 2025-11-25 | ★ Reinforcing Action Policies by Prophesying (Prophet / ProphRL: FA-GRPO + FlowScale)Trains flow VLAs with FA-GRPO and FlowScale inside a 2B Cosmos-initialized action-to-video world model, rewarded by a VLM majority vote. Real UR30e, 100 in-model RL updates, 3 runs x 20 trials: pi0.5-3B 52.1 → 82.1 (+30.0). · code | Fudan University; Shanghai Innovation Institute; Logos Robotics | World models & eval | ✓ checked |
| 2025-11-24 | Discover, Learn, and Reinforce: Scaling Vision-Language-Action Pretraining with Diverse RL-Generated TrajectoriesDLR makes RL generate multiple distinct high-success behavior patterns per task for VLA pretraining; beats equal-sized standard-RL data downstream and scales positively where single-pattern RL data does not. | Yang et al. | On-policy PG | ✓ checked |
| 2025-11-24 | SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-TuningTempo-augmented demonstrations and discounted sparse rewards optimize policy speed. DPPO handles simulation; real-arm PA-RL increases pick-and-place throughput from 11.4 to 20.76 successes/min after 16 minutes of interaction (v2 result). | KAIST, UNIST, DeepAuto.ai | On-policy PG | ✓ corrected |
| 2025-11-20 | ★ Evolution Strategies at the Hyperscale (EGGROLL: Evolution Guided GeneRal Optimisation via Low-rank Learning)Shares matrix multiplication across low-rank evolution-strategies perturbations. With pre-generated noise, one GH200 benchmark (8192-dimensional bf16 layer, batch 1024) reaches 91% of maximum batch-inference throughput, versus PPO 34% and OpenES 0.41% (chart reading). · code | University of Oxford (FLAIR, WhiRL); Mila; NVIDIA AI Technology Center; NormaCore.dev | Black-box search | ✓ checked |
| 2025-11-19 | SRPO: Self-Referential Policy Optimization for Vision-Language-Action ModelsScores failed rollouts by V-JEPA 2 latent distance to the batch's own successes and uses these rewards in a GRPO-style update; one-shot OpenVLA* on LIBERO goes from 48.9% to 99.2% average success. · code | Tongji, Shanghai Innovation Institute, Fudan | Rewards & progress | ✓ checked |
| 2025-11-18 | Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion (VLA-Pilot)GPT-4o writes a keypoint scoring function, then evolutionary search with truncated re-noising inside a diffusion VLA's samples. Real DiVLA 0.31 → 0.62. | CUHK | Test-time search | ✓ checked |
| 2025-11-18 | ★ π*0.6: a VLA That Learns From ExperienceFits a distributional value, binarizes n-step advantages into text conditioning, and supervises the whole flow VLA; CFG is optional. Same-data real laundry: about 2x AWR throughput; success ~96% versus AWR ~91%, PPO ~75% (chart reading). | Physical Intelligence | Advantage-weighted BC | ✓ checked |
| 2025-11-14 | Scalable Policy Evaluation with Video World ModelsAction-conditioned Cosmos-Predict2 evaluator. Sim Pearson 0.833-0.879; real Bridge only 0.687. Adding other policies' rollouts to world-model training matters. | NVIDIA; U Toronto; Vector Institute | World models & eval | ✓ checked |
| 2025-11-13 | ★ Harnessing Bounded-Support Evolution Strategies for Policy Refinement (TD-ES)Continues from a PPO checkpoint with antithetic evolution strategies whose bounded triangular per-coordinate noise keeps candidates in a small box around the weights. Sim IQM success, 3 Isaac Lab Franka tasks: PPO 67.2%, Gaussian ES 73.7%, TD-ES 85.0%. | The University of Sydney; NVIDIA | Black-box search | ✓ checked |
| 2025-11-12 | Diffusion Policies with Value-Conditional Optimization for Offline Reinforcement LearningDIVO (IROS 2025): trains a diffusion behavior model with binary advantage-based loss weights, then regularizes a TD3+BC actor toward diffusion samples with non-negative advantage; reports gains on D4RL locomotion and AntMaze. | Ma et al. | General RL | ✓ checked |
| 2025-11-12 | ★ WMPO: World Model-based Policy Optimization for Vision-Language-Action ModelsRuns critic-free GRPO for OpenVLA-OFT inside a pixel-space video world model fine-tuned on the policy's own rollouts, failures included, with a VideoMAE success classifier. MimicGen (sim), 4 tasks, 128-rollout budget: 33.6 → 47.1 vs online GRPO 33.2, offline DPO 37.3. · code | HKUST; ByteDance Seed | World models & eval | ✓ checked |
| 2025-11-11 | The Path Not Taken: RLVR Provably Learns Off the PrincipalsThree-Gate Theory for LLM RL with verifiable rewards: sparse-looking updates are a model-conditioned bias landing off principal weight directions with minimal spectral drift, unlike SFT. | Zhu et al. | General RL | ✓ checked |
| 2025-11-09 | ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and RetrievalFrozen-feature success/failure buffers, retrieval-augmented LoRA BC and a contrastive loss on failures. Real ID: naive FT 84.7% → 98.0%; OOD: 32% → 98%. | CMU | Advantage-weighted BC | ✓ corrected |
| 2025-11-06 | Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body InteractionsPhysTwin spring-mass twins plus Gaussian-splat rendering. Sim vs real success of ACT/DP/pi0/SmolVLA checkpoints correlates at Pearson r 0.944 / 0.901 / 0.915 on toy packing / rope routing / T-block pushing. · code | Columbia; SceniX; Google DeepMind | World models & eval | ✓ checked |
| 2025-11-05 | Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning EnvironmentsBenchmarks critic-free GRPO against PPO on classical control: all critic-free baselines underperform PPO except on short-horizon tasks like CartPole; high discount factors mostly help; smaller group sizes outperform larger ones. | Oliveira et al. | General RL | ✓ checked |
| 2025-11-02 | AgiBot Achieves First Real-World Deployment of Reinforcement Learning in Industrial RoboticsAgiBot's RW-RL system learns assembly skills directly on a Longcheer pilot production line; company claims training cut from weeks to minutes and 100% task completion over extended operation. The article links no paper or code. | AgiBot; Longcheer Technology | Industry loops | ✓ checked |
| 2025-11-01 | RL without TD learningTransitive RL: divide-and-conquer off-policy value learning over in-dataset subgoals with expectile regression; matches best TD-n on OGBench long-horizon (up to 3,000-step) goal-conditioned tasks without tuning n. | UC Berkeley | General RL | ✓ checked |
| 2025-11 | DYNA-1 Pre-Training: Zero-Shot Dexterity Is Here (adjacent)Company update, no paper: says the DYNA-1 base model folds laundry and sorts packages zero-shot in unseen environments, and that as little as one hour of demos fine-tunes it to roughly 100% success on new tasks. RL not discussed. | Dyna Robotics | Industry loops | ✓ checked |
| 2025-10-31 | End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by Autonomous Hand VLA Policy for Efficient Data CollectionShared autonomy combines human VR arm control with a tactile DexGrasp-VLA hand policy; collected data and corrective teleoperation train an end-to-end arm-hand VLA. Real pick-and-place across 50 objects: 88.7% success, averaging seen and unseen objects. | ByteDance Seed | Human corrections | ✓ checked |
| 2025-10-30 | ★ Human-in-the-loop Online Rejection Sampling for Robotic Manipulation (Hi-ORS)Keeps episodes above a rising return threshold, including human-corrected successes, and trains pi0 with reward-weighted flow matching. Three real tasks, two robots: 23.3 percentage points over offline BC (chart reading); about 1.5 hours of real training. · code | Tsinghua SIGS; Tencent Robotics X | Advantage-weighted BC | ✓ checked |
| 2025-10-30 | ★ Self-Improving Vision-Language-Action Models with Data Generation via Residual RL (PLD: Probe, Learn, Distill)Trains a bounded Gaussian residual with SAC on a frozen VLA, lets the VLA drift before the residual recovers, and distills successes into it. LIBERO Spatial/Object/Goal avg: OpenVLA-OFT 91.8 → 99.2; real Franka cube pick-up: 30/30 vs 10/30 (human data). | NVIDIA GEAR; CMU; UC Berkeley; UT Austin | Residual & steering | ✓ checked |
| 2025-10-29 | ★ πRL (piRL): Online RL Fine-tuning for Flow-based Vision-Language-Action ModelsMakes pi0/pi0.5 flow chunks PPO-trainable by adding a learned per-step noise net (Flow-Noise) or an equal-marginal SDE (Flow-SDE) for exact log-probs, dropped at deployment. Few-shot LIBERO (sim): pi0 57.6% → 97.6%, pi0.5 77.1% → 98.3%. · code | Tsinghua University; Peking University; CASIA; Carnegie Mellon University; Infinigence AI; Zhongguancun Academy | On-policy PG | ✓ checked |
| 2025-10-27 | On-Policy DistillationStudent samples its trajectories; teacher supplies per-token reverse-KL feedback. Thinking Machines estimates 9–30x lower cost than extrapolated off-policy SFT for raising Qwen3-8B-Base from 60% to 70% on AIME 2024. · code | Thinking Machines Lab | General RL | ✓ checked |
| 2025-10-26 | Transitive RL: Value Learning via Divide and ConquerDivide-and-conquer value update for offline goal-conditioned RL built on the triangle inequality; in principle needs O(log T) recursions versus TD's O(T), reducing bias accumulation; best results on long-horizon benchmarks. ICLR 2026. · code | Park et al. | General RL | ✓ checked |
| 2025-10-23 | GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulation3DGS plus ManiSkill3 photoreal twin for evaluation and DAgger. Sim underestimates real in all 8 ACT/pi0 cells. Sim-collected DAgger, evaluated on the real FR3: 63.75% vs 55% training from scratch at iteration 5 (4-task average). · code | UCSD; UCLA; Meta | World models & eval | ✓ checked |
| 2025-10-22 | Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning (RISE: Robust Imitation by Stitching from Experts)Offline IDQL with binary expert/non-expert rewards, a spectral-norm penalty and DINOv2-neighbor action augmentation widens the behavior policy to stitch play/failure data. Real lampshade: BC 17.5%, plain IDQL 10.0%, RISE 82.5%. · code | University of Washington; TRI | Off-policy critics | ✓ checked |
| 2025-10-16 | RM-RL: Role-Model Reinforcement Learning for Precise Robot ManipulationBest-of-scene self-imitation plus REINFORCE for a discrete pick-pose correction. Shelf placement 10/10 for pretrained RM-RL vs 5/10 for plain REINFORCE. This is a cross-method comparison, and the pretrained variant uses about 200 extra prior samples. · code | NTU Singapore | Advantage-weighted BC | ✓ checked |
| 2025-10-16 | ★ VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-TuningBC-trains a visuo-tactile diffusion policy on 30 real demos per task, fine-tunes it with DPPO and sparse success reward in an Isaac Gym twin simulating the same tactile pad, and redeploys. Real tabletop, 5 parts, visuo-tactile mean: 0.50 → 0.85. · code | Columbia University, NVIDIA, UC San Diego | Sim-to-real | ✓ checked |
| 2025-10-16 | ★ RL-100: Performant Robotic Manipulation with Real-World Reinforcement LearningBC-trains a diffusion policy on 29-400 teleop demos per task, then applies PPO-clip at every denoising step, offline with IQL advantages and an OPE gate, then online with GAE. Real, 8 tasks: 100 mean vs DP3 67.8; 1000/1000 trials. · code | Shanghai Qi Zhi Institute; Shanghai Jiao Tong University; HKU; IIIS, Tsinghua University; UNC Chapel Hill; Carnegie Mellon University; CASIA | On-policy PG | ✓ checked |
| 2025-10-15 | Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot ControlOffline-pretrained policies fine-tuned online with injected action perturbations under an EMA-based performance-aware curriculum; more robust than offline-only and faster than scratch on locomotion. | Chiba University | Off-policy critics | ✓ checked |
| 2025-10-15 | The Art of Scaling Reinforcement Learning Compute for LLMs400k+ GPU-hour study fitting sigmoidal compute-performance curves to LLM RL: recipes differ in asymptote, while details like loss aggregation, normalization and curriculum mainly change compute efficiency; ScaleRL predictably extrapolates to a 100k GPU-hour run. | Khatri et al. | General RL | ✓ checked |
| 2025-10-14 | Reflection-Based Task Adaptation for Self-Improving VLA (Reflective Self-Adaptation)GPT-4o converts failure videos into dense rewards for OpenVLA PPO. Standard LIBERO average: 83.6% versus VLA-RL 81.0%; on separate LIBERO-Adapt ablations, removing success-filtered SFT collapses success to zero. | Peking University, JD Explore, Tsinghua | Rewards & progress | ✓ checked |
| 2025-10-13 | Refinery: Active Fine-tuning and Deployment-time Optimization for Contact-Rich PoliciesA GP over start states plus Bayesian optimization picks where PPO trains, and a GMM steers deployment. Per-step assembly success 83.61% → 96.35% in sim. | USC; NVIDIA; University of Washington; University of Sydney | On-policy PG | ✓ checked |
| 2025-10-13 | RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action ModelA process reward model trained on perturbed expert actions scores candidates and predicts local-search directions. Real dual-arm Dobot with a Diffusion Policy base: 72.9% to 88.6% success across seven conditions with ten trials each. | SIAT; Peng Cheng Lab | Test-time search | ✓ checked |
| 2025-10-11 | Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models (FPO for pi0)Freezes the pi0 decoder; trains a latent flow actor with a CFM-loss ratio proxy, PPO clipping and a Q-ensemble. LIBERO average 87.2 (no plain-pi0 row). One-task ablation, SAC replacing the ratio proxy: 32.4 vs 78.5 (ICRA 2026). | Institute of Automation, CAS | On-policy PG | ✓ checked |
| 2025-10-11 | ★ Ctrl-World: A Controllable Generative World Model for Robot ManipulationFine-tunes a 1.5B multi-view video diffusion model on DROID so a VLA runs closed-loop inside it, then fine-tunes the policy on human-labeled successful imagined rollouts. pi0.5-DROID: 38.7% → 83.4% on novel instructions and objects, imagined rollouts only. · code | Stanford University; Tsinghua University | World models & eval | ✓ checked |
| 2025-10-10 | Failure Prediction at Runtime for Generative Robot Policies (FIPER)Failure detection combines RND out-of-distribution scores with action-chunk entropy, calibrated on successes. Time-varying thresholds achieve accuracy 0.78 and TWA 0.65; the CP-band variant with a false-alarm bound achieves TWA 0.62. · code | TU Munich | World models & eval | ✓ corrected |
| 2025-10-09 | DEAS: DEtached value learning with Action Sequence for Scalable Offline RLOffline RL with action-sequence critics (SMDP Q-learning) plus detached value learning to curb overestimation; beats baselines on OGBench and boosts VLAs on RoboCasa Kitchen and real-world manipulation. · code | Kim et al. | Off-policy critics | ✓ checked |
| 2025-10-09 | A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language ModelsSurvey of step-level process reward models for LLMs: process data generation, PRM construction, use in test-time scaling and RL; applications span math, code, multimodal, robotics, agents. | Zheng et al. | General RL | ✓ checked |
| 2025-10-09 | Value FlowsFlow-matching model of the return distribution trained with distributional TD; 1.3x average success gain over 62 tasks. · code | Stanford; Princeton | Off-policy critics | ✓ checked |
| 2025-10-08 | ★ RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action ModelsInfrastructure plus recipe for VLA RL: simulators, rollout and trainer share GPUs, split them or run pipelined, with chunk-aware PPO and GRPO tricks. ManiSkill 25 tasks, ID/OOD: OpenVLA 53.91/39.10 → GRPO 84.38/75.15 → PPO 96.09/81.93. · code | Tsinghua University; Zhongguancun Academy; Infinigence AI; Peking University; UC Berkeley; Harbin Institute of Technology; CASIA | On-policy PG | ✓ checked |
| 2025-10-07 | Verifier-free Test-Time Sampling for Vision-Language-Action Models (MG-Select: Masking Distribution Guided Selection)Selects actions by KL divergence from masked-condition predictions. Jointly trained MG-Select* on pi0-FAST: RoboCasa’s eight pick-and-place tasks, 30 demos, 5.3% to 14.2%; SIMPLER average 46.9% to 50.3%. | KAIST; SNU; RLWRLD | Test-time search | ✓ checked |
| 2025-10-04 | Group Policy GradientCritic-free policy gradient (GPG) replacing the value function with a group-based Monte Carlo advantage estimator inside PPO's clipped objective; matches or beats PPO and uses parallel simulation better. | Chen et al. | General RL | ✓ checked |
| 2025-10-02 | A Recipe for Efficient Sim-to-Real Transfer in Manipulation with Online Imitation-Pretrained World ModelsReward-free online imitation RL pretrains a TD-MPC2-style world model in sim for coverage, then a 15-30 demo real fine-tune. Real 32/40 vs 15/40 for the best baseline (v2 numbers; v1 differed). | UC San Diego, UC Berkeley, UNC Chapel Hill, Sudo AI | Sim-to-real | ✓ checked |
| 2025-10-01 | Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?M2PO constrains the second moment of importance weights for stable LLM RL on stale data. At staleness 256, clipped tokens average 0.06% versus GRPO’s 1.22%; experiments span six model scales. · code | CMU; Meta AI | General RL | ✓ checked |
| 2025-10-01 | ★ The Three Regimes of Offline-to-Online Reinforcement LearningCompares the pretrained policy's return with the dataset's average: if the policy is better, protect it with warm-up rollouts; if the data is better, replay offline data. Regime predictions matched outcomes in 45/63 D4RL settings (71%). | Mila - Quebec AI Institute; Universite de Montreal | Off-policy critics | ✓ checked |
| 2025-10-01 | Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Composition (GPC, General Policy Composition)Convex mix of two policies' scores at each denoising step, with the weight grid-searched on rollouts. E.g. 86.39 vs 78.84 for the better parent on Robomimic+PushT. · code | HKU; Shanghai AI Lab; SJTU | Test-time search | ✓ checked |
| 2025-10-01 | VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World SimulatorsGRPO for a flow VLA against a 138M world model with 'verified' reward (distance to the world model's rendering of the expert). LIBERO 86.6 → 91.1 in 400 steps. · code | Westlake University; Zhejiang University; OpenHelix Team; Fudan University; Zhengzhou University; Beijing University of Posts and Telecommunications; Hebei University of Technology | World models & eval | ✓ corrected |
| 2025-09-30 | VLA Model Post-Training via Action-Chunked PPO and Self Behavior CloningAction-chunked PPO plus auxiliary self-BC loss on a dynamically refreshed buffer of high-quality self-collected trials; 0.93 success and 42.17 steps-to-success on MetaWorld, beating supervised fine-tuning. | Wang et al. | On-policy PG | ✓ checked |
| 2025-09-30 | TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal DistanceLearns signed frame-pair temporal distances from action-free videos as a dense progress reward; added to the sparse success signal for DrQ-v2, it reaches near-perfect success on 9/10 Meta-World tasks within 200k interactions (sim). · code | Tsinghua, Shanghai Qi Zhi, SJTU, UPenn | Rewards & progress | ✓ checked |
| 2025-09-30 | SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential ModelingTreats flow sampling as a residual recurrent computation; GRU-style gated Flow-G or Transformer-decoded Flow-T stabilize direct SAC training. Reports up to 130% improvement on from-scratch MuJoCo HumanoidStandup. · code | Tsinghua, CMU, Li Auto, Shanghai AI Lab | Off-policy critics | ✓ checked |
| 2025-09-29 | XQC: Well-conditioned Optimization Accelerates Deep Reinforcement LearningSAC actor-critic combines BatchNorm, WeightNorm and a distributional cross-entropy critic, reducing condition numbers by orders of magnitude; authors report strong sample efficiency across 55 proprioceptive and 15 vision tasks with fewer parameters. · code | Palenicek et al. | General RL | ✓ corrected |
| 2025-09-29 | ★ Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement LearningPerturbs all weights of 30 model copies with tiny Gaussian noise, scores each with an outcome-only reward, and steps toward high-scoring noise, with no backprop or log-probs. Countdown, Qwen-2.5-3B (language-only): ES 60.5 vs Dr.GRPO-v 43.8 (base 10.0). · code | Cognizant AI Lab, MIT, UCLA, UT Austin | Black-box search | ✓ checked |
| 2025-09-29 | SARM: Stage-Aware Reward Modeling for Long Horizon Robot ManipulationStage-aware progress labels from subtask segmentation; reward-weighted BC on pi0 gets 10/12 medium and 8/12 hard T-shirt folds vs 1/12 and 0/12 for plain BC on the full dataset; available in LeRobot. · code | Stanford, UC Berkeley, xdof.ai | Rewards & progress | ✓ checked |
| 2025-09-29 | World-Env: Leveraging World Model as a Virtual Environment for VLA Post-TrainingRLOO+PPO for OpenVLA-OFT inside a video simulator with a VLM reflector. LIBERO 5-demo average 74.85 → 79.6, about equal to RIPT-VLA in the physics simulator. · code | Sun Yat-sen University; AMAP | World models & eval | ✓ corrected |
| 2025-09-28 | DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous ManipulationFreezes a diffusion imitation policy, trains a SAC residual scaled by 0.1 with shaped rewards, then filters successful rollouts, augments them and retrains imitation policies in a simulation data-generation loop. | Harbin Institute of Technology; Peking University; PsiBot; PKU-PsiBot Lab; Harbin Institute of Technology Suzhou Research Institute | Advantage-weighted BC | ✓ corrected |
| 2025-09-26 | VLA-Reasoner: Empowering Vision-Language-Action Models with Reasoning via Online Monte Carlo Tree SearchMCTS with a KDE action prior, a video world model and a progress value on a frozen VLA. LIBERO OpenVLA 76.0 → 81.0; real OpenVLA 22% → 41%. | Nanyang Technological University | Test-time search | ✓ checked |
| 2025-09-24 | Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA TrainingDiffusion-policy RL autonomously generates LIBERO training data; a VLA trained on it reaches 81.9% average success, +5.3 over human demos and +12.6 over Gaussian-RL data. | Yang et al. | On-policy PG | ✓ checked |
| 2025-09-24 | Selective Progress-Aware Querying for Human-in-the-Loop Reinforcement LearningSPARQ: query policy requests human feedback only when learning stagnates or worsens; near-perfect success on simulated UR5 cube picking (PyBullet) with about half the always-query feedback budget. | Muraleedharan et al. | Human corrections | ✓ checked |
| 2025-09-23 | Generalizable Domain Adaptation for Sim-and-Real Policy Co-TrainingUnbalanced optimal-transport loss aligns joint observation-action features across abundant sim and scarce real demos during BC co-training; up to 30 percentage points higher real-world success (NeurIPS 2025). · code | Cheng et al. | Sim-to-real | ✓ corrected |
| 2025-09-23 | World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic ManipulationPPO inside a diffusion world model trained on demos plus policy and random rollouts. Real Franka 68.3% → 93.3% (DP 88.3%). | CASIA | World models & eval | ✓ checked |
| 2025-09-23 | SOE: Sample-Efficient Robot Policy Self-Improvement via On-Manifold ExplorationInformation-bottleneck latent exploration for iterated success-filtered BC. Real Toaster Load 0.56 → 0.75 (0.84 with human steering) in one round. · code | SJTU; Noematrix | Advantage-weighted BC | ✓ checked |
| 2025-09-23 | ★ Residual Off-Policy RL for Finetuning Behavior Cloning Policies (ResFiT)Keeps an ACT chunked BC policy frozen and trains a small per-step residual with a tuned TD3-style off-policy recipe (demos in each batch, LayerNorm critic, n-step returns). Real 29-DoF humanoid: WoollyBallPnP 14% → 64% after 134 rollouts. · code | Amazon FAR; Stanford; CMU; UC Berkeley | Residual & steering | ✓ checked |
| 2025-09-19 | ★ A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning (VLAC)Fine-tunes InternVL into a pairwise critic outputting signed progress between two frames plus a done signal, labeled from video temporal order, and uses it for real-arm PPO. Four real tasks: about 30% → about 90% within 200 episodes. · code | Shanghai AI Laboratory | Rewards & progress | ✓ checked |
| 2025-09-18 | Self-Improving Embodied Foundation ModelsREINFORCE with a steps-to-go head that serves as both shaped reward and success detector. Real LanguageTable ~62-63% → ~87-88% with ~3% extra episodes (NeurIPS 2025). · code | Google DeepMind | On-policy PG | ✓ checked |
| 2025-09-17 | SHaRe-RL: Structured, Interactive Reinforcement Learning for Contact-Rich Industrial Assembly TasksScripted manipulation primitives and human corrections let SAC learn only a 3D insertion primitive. On real industrial connectors, reported cycle time is 3.5s versus HIL-SERL’s 7.4s after about three hours of interaction. · code | Bielefeld University; Fraunhofer IOSB-INA | Off-policy critics | ✓ checked |
| 2025-09-17 | Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop ApproachHuman-in-the-loop BC+Q fine-tuning uses a second actor for language-guided latent corrections. Three real tasks: 25/25 successes each (Wilson 95%: 86.7–100%) versus HG-DAgger’s 25.3% average; whether evaluation includes human language corrections is unclear. | Beijing Xiaomi Robot Technology | Human corrections | ✓ checked |
| 2025-09-11 | ★ SimpleVLA-RL: Scaling VLA Training via Reinforcement LearningGRPO on a tokenized-action OpenVLA-OFT: 8 rollouts per scene, binary success reward on every action token, group-normalized advantages, no critic, and all-agree groups dropped. LIBERO average 91.0 → 99.1; with one demo per task, 48.9 → 96.9. · code | Tsinghua University, Shanghai AI Lab, SJTU, PKU, HKU | On-policy PG | ✓ checked |
| 2025-09-09 | ★ RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and CorrectionChanges HG-DAgger takeovers: the human first recovers to a familiar state, then finishes the sub-task, and the episode ends with the takeover. Shirt-hanging: 78.3% with 5 hours of data vs ALOHA Unleashed's 75.0% with ~89 hours (RaC's estimate; cross-paper). | Carnegie Mellon University | Human corrections | ✓ checked |
| 2025-09-08 | floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RLTrains a flow-matching Q-function with TD targets; more integration steps increase critic computation. Best configuration reaches about 1.8x FQL on hard OGBench offline tasks, with separate gains in online fine-tuning. · code | CMU | General RL | ✓ checked |
| 2025-09-04 | Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models (ARFM: Adaptive Reinforced Flow Matching)Critic-free softmax-advantage-weighted flow-matching loss with adaptive temperature on pi0. LIBERO simulation average: 88.1% base to 92.1%; flow RWR 90.8%, flow ReinboT 91.2%. | Westlake; UCLA; XJTU | Advantage-weighted BC | ✓ checked |
| 2025-08-29 | Learn from What We HAVE: History-Aware VErifier that Reasons about Past Interactions OnlineDecouples generation from verification: a diffusion model proposes candidate actions and a history-aware verifier selects using past interactions, handling ambiguous articulated objects, multi-modal doors, uneven pick-up. CoRL 2025. · code | CMU | Test-time search | ✓ checked |
| 2025-08-20 | Compute-Optimal Scaling for Value-Based Deep RLStudies compute allocation across model size, batch size and UTD ratio in online value-based RL; identifies TD-overfitting, where large batches hurt small critics but not large ones. | Fu et al. | General RL | ✓ checked |
| 2025-08-17 | Improving Pre-Trained Vision-Language-Action Policies with Model-Based Search (VLAPS)MCTS uses Octo action priors and the LIBERO simulator as an assumed accurate environment model. Simulation only: at the 50k-step Octo checkpoint, LIBERO-Object success rises from 6% to 73% over Octo alone. · code | Neary et al. | Test-time search | ✓ checked |
| 2025-08-15 | Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Reward (AC3)TD3-style actor-critic over 16-action chunks with success-only actor updates; BiGym average 0.465 vs CQN-AS 0.349. | Fudan University and others | Off-policy critics | ✓ corrected |
| 2025-08-07 | IRL-VLA: Training an Vision-Language-Action Policy via Reward World ModelDriving VLA: imitation pretraining, then PPO fine-tuning against a lightweight inverse-RL reward world model instead of a simulator; top NAVSIM v2 results, 2nd in CVPR2025 Autonomous Grand Challenge. | Jiang et al. | General RL | ✓ checked |
| 2025-08-07 | Genie Envisioner: A Unified World Foundation Platform for Robotic ManipulationOne video world model platform for policy (GE-Act), simulation and closed-loop evaluation (GE-Sim), trained on about 3,000 h of teleop. · code | AgiBot | World models & eval | ✓ checked |
| 2025-08-05 | DiWA: Diffusion Policy Adaptation with World ModelsDPPO inside a frozen RSSM trained on play data, with no physical interaction during fine-tuning. CALVIN simulation skills gain roughly 17-41 points over the base policy (close-drawer 59.14% to 91.95%). · code | University of Freiburg; UTN | World models & eval | ✓ checked |
| 2025-08-04 | D2PPO: Diffusion Policy Policy Optimization with Dispersive LossDispersive-loss regularization (in-batch representations as negatives) during diffusion-policy pre-training, then DPPO fine-tuning. RoboMimic average success 0.94 vs 0.86 for DPPO (Transport 0.87 vs 0.60); real Franka Transport 70% vs 45% (AAAI 2026). · code | Sun Yat-sen University | On-policy PG | ✓ checked |
| 2025-08-04 | CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement LearningBehavior cloning on 30–60 demos precedes offline chunked Cal-QL with a Transformer critic. Across six real dexterous-hand grasping tasks, reports 57% average success-rate improvement over SFT; absolute versus relative change is unstated. | Beihang University; Tsinghua SIGS | Off-policy critics | ✓ checked |
| 2025-07-28 | Flow Matching Policy Gradients (Flow Policy Optimization, FPO)PPO ratio = exp(L_CFM_old - L_CFM_new), with no likelihoods. DM Control average 759.3 vs 667.8 for Gaussian PPO and 652.5 for DPPO. · code | UC Berkeley | On-policy PG | ✓ checked |
| 2025-07-24 | Group Sequence Policy OptimizationLLM RL algorithm replacing GRPO's token-level importance ratios with sequence-likelihood ratios and sequence-level clipping; more stable and efficient than GRPO, stabilizes MoE training, used for Qwen3. | Alibaba | General RL | ✓ checked |
| 2025-07-17 | Latent Policy Steering with Embodiment-Agnostic Pretrained World ModelsCross-embodiment-pretrained Dreamer world model plus critic ranks BC chunks. Real Franka, 30-50 demos: BC 21.2% → 36.2%. | CMU | Test-time search | ✓ checked |
| 2025-07-14 | ★ Finetuning Deep Reinforcement Learning Policies with Evolutionary Strategies for Control of Underactuated Robots (EvolSAC)Pretrains SAC on a dense surrogate reward, then fine-tunes the policy with SNES using the true task score as fitness. Sim cartpole swing-up time (10-agent mean): 2.085 s → 1.135 s, vs 1.565 s after 1000 more SAC episodes. | University of Padova | Black-box search | ✓ checked |
| 2025-07-11 | Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous ControlControl-theoretic analysis showing action chunking and exploratory (noise-injected) data collection each circumvent exponential compounding error in behavior cloning, in different regimes; validated on robot learning benchmarks. | Zhang et al. | General RL | ✓ checked |
| 2025-07-11 | Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data (PARS)Reward scaling with layer norm plus penalizing infeasible actions curbs Q-function extrapolation; strong offline and offline-to-online D4RL results, notably AntMaze Ultra (ICML 2025 spotlight). | Kim et al. | Off-policy critics | ✓ checked |
| 2025-07-11 | Online Pre-Training for Offline-to-Online Reinforcement LearningOPT adds an online pre-training phase that trains a new value function tailored for online fine-tuning; average 30% improvement on D4RL (MuJoCo, Antmaze, Adroit) with TD3/SPOT. ICML 2025. | Shin et al. | Off-policy critics | ✓ checked |
| 2025-07-10 | ★ Reinforcement Learning with Action Chunking (Q-chunking; algorithms QC and QC-FQL)Critic and policy act on h-step action chunks with an unbiased h-step TD target, while the policy stays near a flow-matching BC model of the data. OGBench, 25 tasks, offline → online: QC 52→86 vs FQL 37→58. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2025-06-27 | ★ An Introduction to Zero-Order Optimization Techniques for RoboticsTutorial framing derivative-free robotics methods as one loop: sample Gaussian perturbations, score them, move toward better samples; the sample weighting recovers MPPI, CMA-ES or CEM-like updates. Plots only: on 4 Hydrax sim tasks, MPPI-CMA consistently outperforms MPPI. · code | Machines in Motion Laboratory, New York University | Black-box search | ✓ checked |
| 2025-06-23 | ★ CUPID: Curating Data your Robot Loves with Influence FunctionsScores each demo's effect on closed-loop success by chaining influence functions with a REINFORCE estimate from labeled eval rollouts, then drops negative-influence demos and retrains. Real TuckBox mass shift, 66% filtered: base 0%, CUPID 84%, oracle 88% (bar readings). · code | Stanford; Toyota Research Institute; Cambridge; NVIDIA | Advantage-weighted BC | ✓ checked |
| 2025-06-21 | RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action ModelsStructured pruning of a VLA, recovery via SFT then RL (critic warm-up, BC-loss regularization), then quantization: up to 8x memory reduction, 2.3x speedup at original success rate. | Shanghai Jiao Tong University | On-policy PG | ✓ checked |
| 2025-06-21 | RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models7B VLM verifier trained on 20M synthetic preferences reranks Gaussian-perturbed OpenVLA samples. Real OOD 60% vs OpenVLA 35% and V-GPS 30%. · code | Stanford; UC Berkeley; NVIDIA | Test-time search | ✓ checked |
| 2025-06-20 | Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections (CR-DAgger)Compliant physical delta corrections train a force-aware residual on a frozen diffusion policy; real book flipping 40% to 100% success with on-policy delta data. · code | Stanford University | Human corrections | ✓ checked |
| 2025-06-18 | Steering Your Diffusion Policy with Latent Space Reinforcement Learning (DSRL)RL over the input noise of a frozen diffusion or flow policy. Real Franka 2/10 → 9/10 after about 40 episodes. Real pi0 'turn on toaster' 5/20 → 18/20 after 80 episodes. · code | UC Berkeley; U Washington; Amazon | Residual & steering | ✓ checked |
| 2025-06-16 | DynaGuide: Steering Diffusion Policies with Active Dynamic GuidanceA DINOv2-space dynamics model's gradient guides a frozen diffusion policy toward goal images. CALVIN articulated parts 70% average, 8.7x over base. · code | Stanford | Test-time search | ✓ checked |
| 2025-06-11 | SAFE: Multitask Failure Detection for Vision-Language-Action ModelsA tiny failure-detector head on frozen VLA hidden states; unseen-task ROC-AUC 78.00 vs 73.93 for the best baseline in sim. · code | University of Toronto, Vector, TRI | Rewards & progress | ✓ checked |
| 2025-06-10 | TGRPO: Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy OptimizationGRPO mixing step-level and trajectory-level group z-scores, with an LLM-written dense reward. LIBERO average 80.7 vs SFT 76.5 (v3), and not best on Goal. | Jilin University | On-policy PG | ✓ checked |
| 2025-06-09 | Reinforcement Learning via Implicit Imitation Guidance (Data-Guided Noise, DGN)Uses demos only to shape RLPD's exploration covariance, never as a BC loss; up to 2-3x improvement over prior methods on 7 sim tasks. | Stanford University | Off-policy critics | ✓ checked |
| 2025-06-07 | Self-Improving Loops for Visual Robotic Planning (SILVR)Video-generative planner (optionally composed with an internet-pretrained video model) iteratively fine-tuned on its own success-filtered rollouts; improves over iterations on unseen MetaWorld and two real-arm tasks, and still improves with VLM-judged filtering or suboptimal initial data (ICLR 2026). · code | Luo et al. | World models & eval | ✓ checked |
| 2025-06-07 | ★ Scaling Helix: a New State of the Art in Humanoid LogisticsImproves one deployed parcel-sorting job with more in-domain demos, richer System-1 inputs (stereo, vision memory, state history, force) and a 50% larger decoder head; the post describes no RL. Over three months, barcode-orientation success rose ~70% → ~95%. | Figure AI | Industry loops | ✓ checked |
| 2025-06-05 | A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to SearchMPPI residual search in a learned world model with an IRL reward and critic, then distillation. Sim only; e.g. Square 0.29 → 0.78 at 50 demos. · code | Mila; CMU; Cornell | Test-time search | ✓ checked |
| 2025-06-04 | Horizon Reduction Makes RL ScalableMany offline RL methods saturate despite datasets up to 1,000x larger than typical. Horizon-reduction experiments improve scaling on challenging tasks; SHARSA achieves the best asymptotic performance among the evaluated methods. · code | Park et al. | General RL | ✓ checked |
| 2025-06 | 1X World Model: Evaluating Bits, not Atoms (1XWM, policy evaluation with a learned action-conditioned video world model)Action-conditioned video world model that also predicts task success from the final generated frame, used to rank humanoid policy checkpoints. Evidence is predicted success per checkpoint, checked by real runs on selected checkpoints; no correlation coefficient is reported. | 1X Technologies | Industry loops | ✓ checked |
| 2025-06 | Dynamism v1 (DYNA-1) Model: A Breakthrough in Performance and Production-Ready Embodied AIReward-model-in-the-loop training over 6 weeks; self-reported 99.4% success over a 24-hour zero-intervention napkin run (850+ napkins, ~60% of human speed); method described only at a high level. | Dyna Robotics | Industry loops | ✓ checked |
| 2025-05-31 | WorldGym: World Model as An Environment for Policy Evaluation (earlier/code title: "Evaluating Robot Policies in a World Model")Video world model plus GPT-4o judge as a policy evaluator. Pearson r = 0.78 vs real over 17 Bridge tasks. · code | Stanford; NYU; Google DeepMind | World models & eval | ✓ checked |
| 2025-05-30 | ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language ModelsProlonged RL adds KL control, reference-policy resets and diverse tasks to DeepSeek-R1-Distill-Qwen-1.5B. Pass@k gains are largest where the base is weak; some high-baseline math tasks show unchanged or lower pass@128. · code | NVIDIA | General RL | ✓ checked |
| 2025-05-29 | ★ Diffusion Guidance Is a Controllable Policy Improvement Operator (CFGRL)Trains one flow policy by BC with an optimality token; classifier-free guidance samples reference policy times a non-decreasing advantage function. Sim ExORL walker-stand, same IQL critic: 782±8 vs AWR 603±8. OGBench: mixed vs AWR, beats goal-conditioned BC on most tasks. · code | UC Berkeley | Advantage-weighted BC | ✓ checked |
| 2025-05-28 | FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid ControlTD3 with parallel simulation, large-batch updates and a distributional critic; solves a range of HumanoidBench tasks in under 3 hours on one A100 GPU. · code | Seo et al. | Off-policy critics | ✓ checked |
| 2025-05-28 | The Entropy Mechanism of Reinforcement Learning for Reasoning Language ModelsShows LLM policy-gradient RL trades policy entropy for performance, driven by probability-logit covariance; Clip-Cov and KL-Cov restrict high-covariance token updates to prevent entropy collapse. | Cui et al. | General RL | ✓ checked |
| 2025-05-28 | SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation LearningA task-progress predictor removes suboptimal state-action pairs; deduplication removes redundant pairs. On Sirius-Fleet real-robot tasks, the paper text reports a 32.9-percentage-point success gain over training on the full dataset. · code | UT Austin; NVIDIA | Advantage-weighted BC | ✓ checked |
| 2025-05-28 | ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement LearningA learned noise network makes flow steps Gaussian for exact-likelihood PPO. Simulated image-based Can success: 57.83% to 98.50%; Transport: 30.17% to 88.67%. Reports 62.82% lower wall time than DPPO across tasks. · code | CMU, Tsinghua | On-policy PG | ✓ checked |
| 2025-05-27 | Hume: Introducing System-2 Thinking in Visual-Language-Action Modelpi0-style VLA with a built-in value-query head that picks among 5 partly denoised chunks, plus a fast refiner. LIBERO 98.6% vs pi0 94.2%. · code | Shanghai AI Lab; SJTU; Fudan; INSAIT | Test-time search | ✓ checked |
| 2025-05-26 | RFTF: Reinforcement Fine-tuning for Embodied Agents with Temporal FeedbackA temporal-ranking value model as potential-based shaping in PPO; CALVIN ABC-D Avg. Len. 4.283 to 4.296 (Seer-Large), while a sparse-reward PPO ablation fell below the starting policy (4.225). | Peking University | Rewards & progress | ✓ checked |
| 2025-05-26 | What Can RL Bring to VLA Generalization? An Empirical StudyControlled study where PPO beats GRPO and DPO. ID 0.781 (SFT-16k, plateaued) → 0.938. Robot-pose OOD 0.339 → 0.797. Zero-shot Franka pick-and-place 0.00 → 0.27. · code | Tsinghua | On-policy PG | ✓ checked |
| 2025-05-25 | WorldEval: World Model as Real-World Robot Policies EvaluatorConditions Wan 2.1 14B on the policy's own latent (Policy2Vec). Average r 0.942 vs 0.411 for a real-to-sim baseline. · code | Midea Group; East China Normal University | World models & eval | ✓ checked |
| 2025-05-24 | Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine GuidanceRewind-and-refine DAgger on the GO-1 VLA with a learned success detector; e.g. Microwave-Heating task score 0.97±0.01 at 80 trajectories vs 0.55±0.05 teleop. | AgiBot | Human corrections | ✓ checked |
| 2025-05-24 | VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement LearningPPO on OpenVLA-7B, treating a trajectory as a multi-turn conversation, with a VLM process reward model trained on automatically extracted pseudo-labels. LIBERO average 76.5 → 81.0 (OpenVLA SFT baseline); π0-FAST, listed for reference, scores 85.5. · code | Tsinghua SIGS, NTU | On-policy PG | ✓ checked |
| 2025-05-22 | Interactive Post-Training for Vision-Language-Action Models (RIPT-VLA)RLOO, PPO clipping and dynamic sampling use binary success rewards. LIBERO averages: QueST 82.7% to 93.6%, OpenVLA-OFT 96.7% to 97.5%. Best reported one-demo cross-scenario gain: 3.5% SFT to 97.2%. · code | UT Austin | On-policy PG | ✓ checked |
| 2025-05-20 | FlowQ: Energy-Guided Flow Policies for Offline Reinforcement LearningEnergy-guided flow matching learns a Q-guided velocity field during training, needing no inference-time guidance; policy training time is constant in flow sampling steps, with competitive offline RL performance. | Alles et al. | Off-policy critics | ✓ checked |
| 2025-05-18 | RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and CorrectionFine-tuned Qwen2.5-VL diagnoses failures and corrects VLA actions, using 9,440 failed trajectories and 78,623 QA pairs. Failure-analysis score: 79.10 versus GPT-4o 57.42; real SO-100 success after five attempts: 61.25% versus 47.5% without correction. · code | Shanghai Jiao Tong University | Test-time search | ✓ checked |
| 2025-05-16 | Reinforcement Learning Finetunes Small Subnetworks in Large Language ModelsAcross 7 RL algorithms (PPO, GRPO, DPO...) and 10 LLMs, RL fine-tuning updates only 5-30% of parameters; these sparse updates are nearly full-rank. NeurIPS 2025. | Mukherjee et al. | General RL | ✓ checked |
| 2025-05-16 | ReWiND: Language-Guided Rewards Teach Robot Policies without New DemonstrationsA language-conditioned progress reward with rewind augmentation from about 5 demos per task; real bimanual policy 12% to 68% in 1 hour of RL. · code | USC, KAIST | Rewards & progress | ✓ checked |
| 2025-05-14 | DataMIL: Selecting Data for Robot Imitation Learning with DatamodelsLinear datamodels with a target-demo BC-loss proxy select prior-data clusters. LIBERO-10 37.76 vs 27.72 for all-data co-training. · code | UT Austin; MIT; Stanford | Advantage-weighted BC | ✓ checked |
| 2025-05-12 | ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement LearningEnd-to-end VLA predicts dense returns to model data quality and condition actions on high return; authors report gains on CALVIN mixed-quality data and few-shot and real-world OOD generalization. | Zhang et al. | Advantage-weighted BC | ✓ corrected |
| 2025-05-12 | What Matters for Batch Online Reinforcement Learning in Robotics?Controlled study of deploy-collect-retrain loops; implicit Q best-of-N extraction beats AWR (Square: AWR 62 → 11 vs best-of-N 47 → 83). | Stanford University | Off-policy critics | ✓ checked |
| 2025-05-11 | X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-RealReconstructs a photorealistic sim from an RGBD human video, trains RL on object-centric rewards, distills into an image diffusion policy with online sim-real alignment; no teleoperation data. Average task progress +30% over hand-tracking and sim-to-real baselines on 5 real tasks. · code | Cornell | Sim-to-real | ✓ checked |
| 2025-05-02 | SIME: Enhancing Policy Self-Improvement with Modal-level ExplorationSuccess-filtered self-improvement with noise injected into the observation embedding to diversify modes. Square-20 (state): 0.534 → 0.620 vs 0.556 for plain self-improvement. · code | SJTU; Noematrix | Advantage-weighted BC | ✓ checked |
| 2025-04-18 | Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?RLVR-trained LLMs win at pass@1 but base models win at large-k pass@k: RL improves sampling efficiency within base-model capability, unlike distillation; NeurIPS 2025 Oral. · code | Tsinghua University; Shanghai Jiao Tong University | General RL | ✓ checked |
| 2025-04-04 | Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital TwinThe policy always acts on a synced dynamic digital twin. Failure states mined in sim plus 30 new demos lift real PushT from about 57% to 80%. | RAI Institute; QUT | World models & eval | ✓ checked |
| 2025-04 | Dyna Robotics Unveils DYNA-1: The First Commercial-Ready Robot Foundation Model Offering Fully Autonomous, Round-the-Clock DexterityPress release for DYNA-1: self-reported 800+ napkins folded in a continuous 24-hour run with zero interventions, 99.4% success at 60% of human throughput; credits a proprietary reward model for error recovery and training-data generation. | Dyna Robotics | Industry loops | ✓ checked |
| 2025-03-31 | HACTS: a Human-As-Copilot Teleoperation System for Robot LearningLow-cost, 3D-printed bilateral teleoperation system that syncs leader hardware with robot joints so humans can seamlessly intervene and collect correction data; improves imitation learning and human-in-the-loop RL. | Xu et al. | Human corrections | ✓ checked |
| 2025-03-31 | Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic ManipulationCo-trains one Diffusion Policy on 20-50 real demos per task plus MimicGen-multiplied sim 'digital cousins' and prior sim data. Across six real tasks, average success rose from 45.3% (real only) to 83.2% (real + cousins + prior). | UT Austin, NVIDIA, UC Berkeley, New York University | Sim-to-real | ✓ checked |
| 2025-03-26 | Understanding R1-Zero-Like Training: A Critical PerspectiveDr. GRPO removes a response-length optimization bias, improving token efficiency while maintaining reasoning performance. The paper’s complete minimalist R1-Zero recipe reaches 43.3% AIME 2024 accuracy with a 7B base model. · code | Sea AI Lab | General RL | ✓ checked |
| 2025-03-19 | 1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching CapabilitiesScaling contrastive goal-conditioned RL from 4 to 64 layers gives 2–50x gains on simulated tasks without demonstrations or rewards; Humanoid U-Maze improves further up to 1,024 layers. · code | Wang et al. | General RL | ✓ checked |
| 2025-03-18 | DAPO: An Open-Source LLM Reinforcement Learning System at ScaleDecoupled Clip and Dynamic Sampling Policy Optimization: four techniques for large-scale LLM RL; 50 points on AIME 2024 with Qwen2.5-32B; code on verl plus dataset released. · code | Yu et al. | General RL | ✓ checked |
| 2025-03-11 | MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action ModelsMixture-of-LoRA-experts quadruped VLA fine-tuned with an RL objective on mixed-quality data; outperforms baselines across six skills, generalizes OOD, validated on real robot (ICRA 2025). | Zhao et al. | Off-policy critics | ✓ checked |
| 2025-03-10 | RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot ManipulationCompliant bilateral teleoperation system that lets a human seamlessly take over from and hand back to the policy, collecting interactive correction data for bimanual manipulation in sim and real. | Wu et al. | Human corrections | ✓ checked |
| 2025-03-06 | Refined Policy Distillation: From VLA Generalists to RL ExpertsA small PPO student plus an MSE term toward a frozen VLA teacher's action. Even 1-10% success teachers make sparse-reward PPO work in ManiSkill3 (sim only). · code | UTN Nuremberg | On-policy PG | ✓ checked |
| 2025-03-05 | Curating Demonstrations using Online Experience (Demo-SCORE)A success classifier trained on policy rollouts flags demonstration strategies the policy cannot execute. Robomimic square peg (simulation, averaged over data mixtures): 79.9% base to 94.4%. · code | Stanford | Advantage-weighted BC | ✓ checked |
| 2025-03-05 | Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returns (T-SAC)Transformer critic over action prefixes trained with N-step returns without importance sampling; Box-Pushing dense 96.8% vs baselines at or below 85%. · code | KIT | Off-policy critics | ✓ checked |
| 2025-03-03 | Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning (DEMO3)Demonstration-augmented model-based RL (TD-MPC2 backbone) that learns stage-wise dense rewards online alongside policy and world model; 40% better average data efficiency (70% on the hardest tasks) than baselines across 16 simulated sparse-reward tasks, using 5-100 demos per task. · code | UC San Diego, Hillbot, ETH Zürich | Rewards & progress | ✓ checked |
| 2025-02-26 | Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion ModelsComputes uncertainty from a diffusion policy's own denoising process to request operator help only when needed at deployment; the same signal targets data collection for fine-tuning. | He et al. | Human corrections | ✓ checked |
| 2025-02-12 | Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic ManipulationReal-to-sim pipeline using 3D reconstruction and neural rendering inside a physics simulator; imitation policies trained only on its sim data exceed 58% average zero-shot real success. · code | Han et al. | Sim-to-real | ✓ checked |
| 2025-02-12 | Robot Data Curation with Mutual Information Estimators (DemInf)Label-free demo scoring by contribution to I(S;A), using k-NN estimators in VAE latents. The abstract reports a 5-10% improvement on RoboMimic. · code | Google DeepMind; Stanford | Advantage-weighted BC | ✓ checked |
| 2025-02-11 | From Action Labels to Sets: Rethinking Action Supervision for Imitation Learning from Corrective Feedback (CLIC)CLIC: builds set-valued desired-action targets from human corrections and trains the policy to put probability mass on those sets, giving robustness to noisy corrections and multi-modal behaviors. · code | Li et al. | Human corrections | ✓ checked |
| 2025-02-08 | ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency PolicyTrains a consistency-policy head over frozen Octo using BC+Q: Cal-QL offline, then human-in-the-loop online fine-tuning. Across eight real-robot tasks, mean success rises from 39.4% to 96.3%. · code | Institute of Automation, CAS | Off-policy critics | ✓ checked |
| 2025-02-06 | Value-Based Deep RL Scales PredictablyFits power laws relating batch size and learning rate to UTD and predicts data/compute trade-offs from small runs; evaluates off-policy value-based RL with SAC, BRO and PQL. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2025-02-05 | Learning from Active Human Involvement through Proxy Value Propagation (PVP)Reward-free +1/-1 proxy values on human vs rejected actions; MetaDrive test success 0.857 vs HG-DAgger 0.045 (NeurIPS 2023, arXiv 2025). · code | UCLA | Human corrections | ✓ checked |
| 2025-02-04 | DIME: Diffusion-Based Maximum Entropy Reinforcement LearningDerives a tractable lower bound on the MaxEnt RL objective for diffusion policies with provably convergent policy iteration; outperforms other diffusion-RL methods on high-dimensional control benchmarks. | Celik et al. | Off-policy critics | ✓ checked |
| 2025-02-04 | Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning (SGFT)Precursor. The frozen sim critic serves as a potential-based shaping reward and H-step terminal value for real-world SAC/TD-MPC2. 100% on hammering and pushing within about an hour, 70% on insertion within two hours. · code | University of Washington, Microsoft Research | Sim-to-real | ✓ checked |
| 2025-02-04 | Flow Q-Learning (FQL)Pure-BC flow policy plus a one-step actor that climbs Q under a distillation leash; OGBench cube-double 29 vs ReBRAC 12. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2025-02-03 | From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment (FOREWARN)Latent world model plus fine-tuned VLM verify 6 plan modes in language. Real Franka Fork task 0.10 → 0.70. · code | CMU | Test-time search | ✓ checked |
| 2025-02-02 | Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling (GPC: Generative Predictive Control; v1 titled "Strengthening Generative Robot Policies through Predictive World Modeling")World model plus reward predictor rank diffusion-policy chunks. State Push-T 0.812 → 0.934 (K=5000); vision Push-T 0.642 → 0.739 vs V-GPS 0.620. · code | Harvard; Georgia Tech | Test-time search | ✓ checked |
| 2025-01-28 | Improving Vision-Language-Action Model with Online Reinforcement Learning (iRe-VLA)Alternates online RL on an action head over a frozen VLM with full-model SFT on expert demos and RL successes. MetaWorld unseen-10 success: 0.51 SFT / 0.39 PPO-Replay / 0.80 iRe-VLA. | Tsinghua IIIS; UC Berkeley | Advantage-weighted BC | ✓ checked |
| 2024-12-18 | Policy Decorator: Model-Agnostic Online Refinement for Large Policy ModelBounded SAC residual plus a progressive exploration schedule on frozen BeT or Diffusion Policy. Bases at 13-78% reach near-perfect success on 8 sim tasks, with millions of env steps. · code | UC San Diego | Residual & steering | ✓ corrected |
| 2024-12-13 | RLDG: Robotic Generalist Policy Distillation via Reinforcement LearningReal-world HIL-SERL specialists generate successful rollouts that become SFT data for OpenVLA/Octo. On FMB Insertion, OpenVLA succeeds in 25/30 trials versus 15/30 when fine-tuned on human demonstrations. · code | UC Berkeley | Advantage-weighted BC | ✓ corrected |
| 2024-12-10 | Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data (WSRL: Warm-Start RL)Drop offline data, warm up with frozen-policy rollouts, then high-UTD RL; real Franka peg insertion 13/20 → 20/20 in 18 min, while SERL/RLPD was at 0/20 after 50 min. · code | UC Berkeley; CMU | Off-policy critics | ✓ checked |
| 2024-12-09 | Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone (PA-RL)Samples policy actions, selects high-Q candidates, refines them by Q-gradient ascent, then distills with the policy’s own loss. PA-RL+Cal-QL on simulated CALVIN: 28% offline success to 61% after online fine-tuning. · code | CMU; Stanford | Off-policy critics | ✓ checked |
| 2024-12-02 | Robot Learning with Super-Linear Scaling (CASHER)Crowdsourced real-to-sim scans plus sim RL bootstrapped by human demos; the improving generalist policy replaces human effort, giving zero/few-shot real transfer and scan-only fine-tuning. | Torne et al. | Sim-to-real | ✓ checked |
| 2024-11-28 | GRAPE: Generalizing Robot Policy via Preference AlignmentTrajectory-level preference optimization aligns OpenVLA using its successful and failed rollouts, ranked by task success, self-evaluation and VLM-generated stage/keypoint costs, with a KL anchor to the SFT reference policy. · code | UNC-Chapel Hill; University of Washington; University of Chicago | Advantage-weighted BC | ✓ checked |
| 2024-11-26 | PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online RefinementPredicts a Gaussian over task progress from initial, current and goal frames, pretrained on EPIC-KITCHENS video; during online RL, adversarial 'push-back' lowers estimates on the agent's own out-of-distribution rollouts to counter distribution shift. · code | UChicago, TTIC | Rewards & progress | ✓ checked |
| 2024-11-25 | Inference-Time Policy Steering through Human Interactions (ITPS)Human clicks, sketches or nudges define inference-time costs for steering pretrained generative policies. Compares sample ranking, gradient guidance and diffusion-specific stochastic sampling, studying intent alignment versus constraint satisfaction. · code | MIT CSAIL; NVIDIA | Test-time search | ✓ checked |
| 2024-11-19 | Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Reinforcement LearningCritic-only coarse-to-fine Q-network over action sequences; avoids actor-driven overestimation that breaks chunked SAC/TD3 (curves only). · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2024-11-07 | Vision Language Models are In-Context Value Learners (Generative Value Learning, GVL)Asks a frozen VLM (Gemini-1.5-Pro) for task progress on shuffled frames, zero- or few-shot; introduces VOC (0.74 on RT-1); one-shot GVL-weighted AWR beat diffusion policy on 4 of 7 real ALOHA tasks (1 tie, 2 losses). | Google DeepMind, UPenn, Stanford | Rewards & progress | ✓ checked |
| 2024-11-04 | So You Think You Can Scale Up Autonomous Robot Data Collection?CoRL 2024 study over 7 sim and real tasks: autonomous imitation-learning data collection inherits RL's environment-design burdens; scaling human demos often improves policies significantly more. | Mirchandani et al. | General RL | ✓ checked |
| 2024-11 | CHG-DAgger: Interactive Imitation Learning with Human-Policy Cooperative ControlCooperative HG-DAgger uses multilateral control so a human and learned visuomotor policy can correct task failures together without a full control handoff. | Toyota Motor Corporation; Toyota Research Institute | Human corrections | ✓ checked |
| 2024-10-30 | Multi-Task Interactive Robot Fleet Learning with Visual World Models (Sirius-Fleet)World-model-based failure monitor decides when humans step in; real autonomous success 0.250 to 0.715 over 3 rounds. · code | UT Austin | Human corrections | ✓ checked |
| 2024-10-29 | Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning (HIL-SERL)RLPD with human SpaceMouse corrections and a learned binary reward classifier, trained on real robots. RAM insertion: 100% after 1.5 h vs 29% for a data-matched HG-DAgger baseline; faster cycle times on every timed task. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2024-10-25 | On-Robot Reinforcement Learning with Goal-Contrastive Rewards (GCR)A VIP-style progress embedding kept honest with hard negatives from the agent's own failed episodes; solves about twice as many tasks as baseline reward learners. | RAI Institute, Northeastern | Rewards & progress | ✓ checked |
| 2024-10-23 | SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon ManipulationTAMP decomposes tasks into short segments; BC policies receive RL fine-tuning with a KL penalty. Across nine simulated tasks, average success is 87.8% versus 52.9% for TAMP-gated BC and 37.6% for planner-guided RL. | Zhou et al. | Off-policy critics | ✓ checked |
| 2024-10-18 | Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic ManipulationDiffusion training loss triggers human queries in robot-gated DAgger; reports 39.0% higher failure-prediction F1 and 20.6% task-completion improvement over evaluated baselines, with 7.8x lower average policy-update time than Ensemble-DAgger. · code | Lee et al. | Human corrections | ✓ checked |
| 2024-10-17 | Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance (V-GPS)Offline Cal-QL Q reranks K samples from frozen generalists. Real WidowX Octo-small-1.5 0.24 → 0.44 (+82%); SIMPLER gains are small for some policies (OpenVLA 0.24 → 0.27). · code | UC Berkeley; CMU | Test-time search | ✓ checked |
| 2024-10-12 | TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement LearningEnables off-policy updates for episodic RL by segmenting long action sequences and valuing each segment with a transformer critic and n-step returns; ICLR 2025 Spotlight. | Li et al. | Off-policy critics | ✓ checked |
| 2024-10-09 | Automated Creation of Digital Cousins for Robust Policy Learning (ACDC)Precursor. Sim 'cousin' scenes built from one image. Zero-shot real door opening: 25% for the exact twin vs 90% cousin-only and 95% twin+cousin. · code | Stanford University | Sim-to-real | ✓ checked |
| 2024-10-08 | Incremental Learning for Robot Shared AutonomyILSA continually fine-tunes an assistive arm's shared-autonomy policy on each interaction's limited new data while preserving prior knowledge. In a 20-person study it completed tasks faster than static baselines. · code | Carnegie Mellon University; NIST; University of Pittsburgh | Human corrections | ✓ checked |
| 2024-10-08 | Reinforcement Learning From Imperfect Corrective Actions And Proxy RewardsICoPro: value-based RL that combines sparse, imperfect human corrective actions (margin loss plus target-Q pseudo-labels) with proxy rewards; better alignment and sample efficiency on Atari and highway driving. | Jiang et al. | Human corrections | ✓ checked |
| 2024-10-06 | The Promise of Generalist Robotic PoliciesEssay: generalist robot foundation models bootstrap a deployment data flywheel; language feedback and RL from real-world outcomes then drive autonomous self-improvement beyond imitation. | Sergey Levine | General RL | ✓ checked |
| 2024-10-06 | Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress (Sentinel)Runtime failure detector (action-chunk consistency plus VLM progress check). Detects 18% more failures than either detector alone. · code | Stanford; NVIDIA | World models & eval | ✓ checked |
| 2024-10-02 | Run-time Observation Interventions Make Vision-Language-Action Models More Visually RobustBYOVLA: training-free run-time intervention finds image regions a VLA is sensitive to and edits task-irrelevant distractors out (VLM + segmentation + inpainting); on hardware, VLAs nearly retain nominal performance despite distractors that otherwise cut success by up to 40%. · code | Princeton | Test-time search | ✓ checked |
| 2024-10-01 | AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic ManipulationFine-tunes a VLM on FailGen's procedurally perturbed sim failures to detect and explain manipulation failures; beats GPT-4o in-context learning by 10.3%; as failure feedback it lifts success in three LLM/VLM pipelines 21.4% on average over GPT-4 models. · code | Duan et al. | Rewards & progress | ✓ checked |
| 2024-09-30 | Towards Effective Utilization of Mixed-Quality Demonstrations in Robotic Manipulation via Segment-Level Selection and Optimization (S2I)S2I segments mixed-quality demos, selects good segments via contrastive learning and optimizes poor ones' trajectories; 3 expert demos as reference improve downstream policies on six tasks (ICRA 2025). · code | Chen et al. | Advantage-weighted BC | ✓ checked |
| 2024-09-30 | RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation LearningPrecursor. A 3DGS-rendered twin lets image-based SAC transfer zero-shot. small_cube grasping 96.88% sim → 96.88% real, vs 12.50% real for mesh renders. · code | Shanghai Jiao Tong University and others | Sim-to-real | ✓ checked |
| 2024-09-25 | FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-TuningPPO fine-tunes a BC transformer in AI2THOR using a low learning rate, no entropy bonus and a separate critic. CHORES-S success: ObjectNav 85.0%, Fetch 66.9%, PickUp 91.8%, RoomVisit 70.4%. · code | Ai2, UT Austin, UW, Sony AI | On-policy PG | ✓ checked |
| 2024-09-23 | RACER: Rich Language-Guided Failure Recovery Policies for Imitation LearningAugments expert demos with automatically generated failure-recovery trajectories and rich language; a VLM supervisor guides a language-conditioned visuomotor actor, outperforming RVT on RLBench. · code | Dai et al. | Human corrections | ✓ checked |
| 2024-09-16 | SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian SplattingPrecursor. Gaussian-splat rendering over PyBullet physics. A zero-shot RGB diffusion policy averages 86.25% sim-to-real vs 97.5% real-to-real. · code | Carnegie Mellon University | Sim-to-real | ✓ checked |
| 2024-09-10 | DemoStart: Demonstration-led Auto-Curriculum Applied to Sim-to-Real with Multi-Fingered RobotsUses a few sim demos and a sparse reward to build an auto-curriculum for RL on a three-fingered hand, then distills and transfers zero-shot to real. Needs 100x fewer demos than real-robot baselines. | Google DeepMind | Sim-to-real | ✓ checked |
| 2024-09-01 | Diffusion Policy Policy Optimization (DPPO)PPO over diffusion denoising steps. Low-randomness Furniture-Bench simulation: One-leg 57% → 97%, Lamp 12% → 87%, Round-table 1% → 86%. Zero-shot hardware One-leg: 16/20 versus 0/20 for Gaussian PPO. · code | Princeton, MIT, TRI, CMU | On-policy PG | ✓ corrected |
| 2024-09 | 1X World Model (announcement of a learned simulator for evaluating humanoid policies, plus the 1X World Model Challenge)Blog and challenge arguing for a learned simulator as a repeatable evaluator. No quantitative link to real success. · code | 1X Technologies | Industry loops | ✓ checked |
| 2024-08-30 | Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling (BID)Training-free selection among N chunks by coherence with the last chunk and contrast with a weak checkpoint. 32% average relative gain over vanilla closed loop on 7 sim tasks. · code | Stanford | Test-time search | ✓ checked |
| 2024-08-10 | Representation Alignment from Human Feedback for Cross-Embodiment Reward Learning from Mixed-Quality DemonstrationsShows cross-embodiment IRL (e.g., XIRL) fails on mixed-quality demos; studies representation learning aligned with human feedback to improve transferable reward learning and shaping. | University of Utah | Rewards & progress | ✓ checked |
| 2024-07-30 | Autonomous Improvement of Instruction Following Skills via Foundation Models (SOAR)VLM-proposed tasks and VLM success filtering for goal-conditioned BC on autonomous data. Average real-world success 28% to 57% across 10 scenes, with no human annotations. · code | UC Berkeley | Advantage-weighted BC | ✓ checked |
| 2024-07-23 | From Imitation to Refinement -- Residual RL for Precise Assembly (ResiP)Per-step PPO residual on a frozen chunked diffusion policy. Sim assembly goes from 5-54% to 88-99% (low randomization) with 50 demos, while diffusion BC plateaus at about 80% on one_leg even with 100K RL-generated demos. · code | MIT Improbable AI; Harvard | Residual & steering | ✓ checked |
| 2024-07-01 | Residual-MPPI: Online Policy Customization for Continuous ControlOnline MPPI planning adapts a pretrained policy to new objectives using only its action distribution, no retraining; customizes champion GT Sophy racing agent few-shot and zero-shot. · code | UC Berkeley; Sony Research; UT Austin | Residual & steering | ✓ checked |
| 2024-06-22 | To Err is Robotic: Rapid Value-Based Trial-and-Error during DeploymentBellman-Guided Retrials: a demo-trained value function detects stalled progress at deployment and resamples alternative strategies, avoiding failed ones; raises average absolute success rates by more than 20 points in simulation and more than 50 in real-world tests. | Du et al. | Test-time search | ✓ checked |
| 2024-06-20 | IRASim: A Fine-Grained World Model for Robot ManipulationAn action-conditioned video model ranks candidate chunks using predicted-outcome IoU. Push-T IoU: 0.637 to 0.961 with 50 candidates and 1,000 world-model training rollouts; without rollout training data, 50 candidates score 0.418. · code | HKUST; ByteDance Seed | Test-time search | ✓ checked |
| 2024-06-17 | AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic ManipulationMLLM predicting SE(3) poses for articulated objects corrects itself from interaction failures via visual-mask and text prompts, a feedback extraction module, and test-time adaptation. | Xiong et al. | Test-time search | ✓ checked |
| 2024-05-31 | Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (DAC)KL-constrained policy iteration written as diffusion noise regression with a Q-gradient nudge; locomotion total 836.4 vs DTQL 798.3. · code | HKUST; SUSTech | Off-policy critics | ✓ checked |
| 2024-05-30 | Video-Language Critic: Transferable Reward Functions for Language-Conditioned RoboticsA CLIP-initialized video-text critic learns contrastive and temporal-ranking rewards. At matched hand-reward convergence budgets on MetaWorld: 58% sparse-only success, 78% with in-domain MW50 VLC, 65% with cross-embodiment Open X VLC. · code | Aalto University and others | Rewards & progress | ✓ checked |
| 2024-05-25 | Diffusion-based Reinforcement Learning via Q-weighted Variational Policy OptimizationQVPO (NeurIPS 2024): Q-weighted variational loss plus entropy regularization and an action-selected behavior policy trains diffusion policies in online RL; authors report improved reward and sample efficiency over their MuJoCo baselines. | Ding et al. | Off-policy critics | ✓ corrected |
| 2024-05-24 | Diffusion Actor-Critic with Entropy RegulatorOnline RL (DACER) uses the diffusion reverse process as the policy, with GMM-estimated entropy tuning exploration noise; authors report leading results among their baselines on most tested MuJoCo benchmarks. | Wang et al. | Off-policy critics | ✓ corrected |
| 2024-05-16 | TRANSIC: Sim-to-Real Policy Transfer by Learning from Online CorrectionGated residual trained by max likelihood on human SpaceMouse corrections over a frozen sim policy. Real Franka average 81% over four tasks; beats BC fine-tuning, and HG-DAgger and IWR, which the authors find forget the base policy. · code | Stanford | Residual & steering | ✓ checked |
| 2024-05-09 | Evaluating Real-World Robot Manipulation Policies in Simulation (SIMPLER)Real-to-sim evaluation via sysID plus visual matching. Average Pearson r 0.924 vs 0.308 for offline action MSE. Introduced MMRV. · code | UCSD; Stanford; UC Berkeley; Google DeepMind | World models & eval | ✓ checked |
| 2024-05-02 | IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation LearningExpands 10 human interventions into 1,000 synthetic corrections. In simulated Coffee with injected pose-estimation error and full observability on contact, success improves from 2% base to 80%. | NVIDIA; UC Berkeley | Human corrections | ✓ checked |
| 2024-04-25 | DrS: Learning Reusable Dense Rewards for Multi-Stage TasksLearns stage-aware dense rewards from sparse rewards and optional demos, reusable on unseen tasks; across 1000+ manipulation task variants it sometimes matches hand-engineered rewards. · code | Mu et al. | Rewards & progress | ✓ checked |
| 2024-04-23 | Rank2Reward: Learning Shaped Reward Functions from Passive VideoTemporal ranking of demo frames plus a GAIL discriminator for pessimism; real xArm tasks learned in under 2 hours. · code | MIT, UW, Bristol | Rewards & progress | ✓ checked |
| 2024-03-21 | Bayesian Optimization for Sample-Efficient Policy Improvement in Robotic Manipulation (BOpt-GMM)Imitate from about 10 demos with a GMM dynamical system, then BO (SMAC3) on sparse binary success. In sim it reaches 80% success 87 episodes earlier than SAC-GMM. · code | University of Freiburg; UTN | Black-box search | ✓ checked |
| 2024-03-19 | Yell At Your Robot: Improving On-the-Fly from Language CorrectionsHigh-level policy over language-conditioned low-level skills takes real-time verbal corrections and is iteratively retrained on them, improving long-horizon dexterous manipulation without extra teleoperation. · code | Shi et al. | Human corrections | ✓ checked |
| 2024-03-06 | Stop Regressing: Training Value Functions via Classification for Scalable Deep RLTrains value functions with categorical cross-entropy instead of MSE regression, improving performance and scalability on Atari, Q-transformer robot manipulation, chess and Wordle by mitigating noisy, non-stationary targets. | Google DeepMind | General RL | ✓ checked |
| 2024-03-06 | Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation (RialTo)Precursor. Phone-scan twin, then PPO with a BC term starting from 15 demos, then distill back. Real success averages 91%/77%/75% (poses/distractors/disturbances) vs 25%/11%/5% for BC. · code | MIT, University of Washington, TU Darmstadt | Sim-to-real | ✓ checked |
| 2024-02-27 | Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation LearningReplaces DAgger's extra expert data collection with diffusion-synthesized off-distribution eye-in-hand samples; 80% pushing success from 8 demos vs 20% for BC (RSS 2024). · code | Zhang et al. | Human corrections | ✓ checked |
| 2024-02-09 | Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction FollowingTest-time evolutionary search over a frozen trajectory diffusion model, mutating by partial re-noise and denoise. It matches the hand-designed SOTA planner on nuPlan (driving score 92 vs 92). · code | Carnegie Mellon University | Black-box search | ✓ checked |
| 2024-02-08 | Offline Actor-Critic Reinforcement Learning Scales to Large Models (Perceiver-Actor-Critic, PAC)Up to about 1B-parameter offline actor-critic with one BC-to-RL knob; self-improvement on real CHEF from 69.8% to 93.2%. | Google DeepMind | Off-policy critics | ✓ checked |
| 2024-02-05 | Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation ProblemRL fine-tuning can forget pretrained capabilities on states not visited early; knowledge-retention methods mitigate this. NetHack Human Monk exceeds 10K points versus the previous neural-model best of 5K. | Wołczyk et al. | General RL | ✓ checked |
| 2024-01-29 | SERL: A Software Suite for Sample-Efficient Robotic Reinforcement LearningTuned real-robot RLPD stack with reward classifiers; 100/100 success on 3 tasks with 20-105 min of training. · code | UC Berkeley; University of Washington; Stanford; Intrinsic | Off-policy critics | ✓ checked |
| 2024-01-26 | RESPRECT: Speeding-up Multi-fingered Grasping with Residual Reinforcement LearningLearns a residual policy on top of a DRL-pretrained multi-fingered grasping policy, reusing its components; about 5x fewer timesteps than from scratch, validated on the real iCub. | Ceola et al. | Residual & steering | ✓ checked |
| 2024-01-17 | Learning from Imperfect Demonstrations with Self-Supervision for Robotic ManipulationSSDF: self-supervised quality scoring of failed-trajectory segments using expert plus imperfect data; high-scoring segments augment offline BC training, improving success on ManiSkill2 and a real Franka, reward-free. | Wu et al. | Advantage-weighted BC | ✓ checked |
| 2023-12-18 | Learning a Diffusion Model Policy from Rewards via Q-Score Matching (QSM)Trains a diffusion policy's score to match alpha * grad_a Q, avoiding backprop through the sampler; matches SAC/TD3 on DMC. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2023-11-22 | Guided Flows for Generative Modeling and Decision MakingAdds classifier-free guidance to flow matching. Improves image and speech generation, and for offline-RL planning runs about 10x faster than diffusion with comparable returns. | Meta, UCLA, Weizmann Institute | General RL | ✓ checked |
| 2023-11-21 | RLIF: Interactive Imitation Learning as Reinforcement Learning-1 reward on the step before each takeover plus off-policy RL; sim Walker2d normalized score 103.83±4.48 vs HG-DAgger 36.75±4.7 (averaged over three expert levels, same intervention rule), and it improves on suboptimal experts. · code | UC Berkeley | Human corrections | ✓ checked |
| 2023-11-17 | Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections (DROC)LLM system distills reusable knowledge from language corrections (plan-level and skill-parameter) and retrieves it later; half the first-round corrections of LLM code-generation baselines, few or none after two iterations. · code | Zha et al. | Human corrections | ✓ checked |
| 2023-11-03 | Imitation Bootstrapped Reinforcement Learning (IBRL)Frozen BC policy and TD3 actor both propose; the critic picks, also inside the TD target. Real Hang task: BC 65%, RLPD 15%, IBRL 85%. · code | Stanford University | Off-policy critics | ✓ checked |
| 2023-10-26 | Model-Based Runtime Monitoring with Interactive Imitation LearningA latent dynamics model and failure classifier predict problems and request human supervision before failure. Evaluated as a human-robot interactive-imitation system in simulation and on hardware; the human decides whether to intervene. · code | UT Austin | Human corrections | ✓ checked |
| 2023-10-23 | Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning (RoboFuME)A MiniGPT-4 success detector rewards reset-free Cal-QL fine-tuning. Across five real tasks, mean task score rises from 50.4% offline to 76.2% after 2–4 hours; the aggregate mixes success rates and candy-sweeping completion. | Stanford | Rewards & progress | ✓ checked |
| 2023-10-11 | RoboCLIP: One Demonstration is Enough to Learn Robot PoliciesRewards a PPO agent at episode end with a frozen S3D video-language model's similarity between its rollout and a single video or text demo; in simulated Meta-World/Franka Kitchen, 2-3x higher zero-shot task reward than GAIL/AIRL. | USC, Stanford | Rewards & progress | ✓ checked |
| 2023-10-09 | UniSim: Learning Interactive Real-World SimulatorsA 5.6B video simulator. REINFORCE raises VLA success from 0.58 to 0.81 (pointing: 0.12 to 0.71), assessed qualitatively on 48 Language Table tasks using video rollouts inside UniSim itself. | Google DeepMind; UC Berkeley; MIT; U Alberta | World models & eval | ✓ checked |
| 2023-09-22 | Robotic Offline RL from Internet Videos via Value-Function Pre-Training (V-PTR)TD value pretraining on action-free human video initializes a CQL agent; WidowX scenario 1 total 24/48 vs PTR 5/48. | UC Berkeley; Google DeepMind | Off-policy critics | ✓ checked |
| 2023-09-20 | Text2Reward: Reward Shaping with Language Models for Reinforcement LearningLLM generates dense reward code from language goals, refinable with human feedback; matches or beats expert rewards on 13 of 17 ManiSkill2/MetaWorld tasks, 94%+ on six locomotion behaviors. · code | Xie et al. | Rewards & progress | ✓ checked |
| 2023-09-18 | Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-FunctionsPer-dimension autoregressive Transformer Q-function trained offline on demos plus failures; 56% vs RT-1 25% on 72 real tasks. | Google DeepMind | Off-policy critics | ✓ checked |
| 2023-09-06 | Robotic Table Tennis: A Case Study into a High Speed Learning SystemBGS, an ARS-derived search, trains a roughly 1,000-parameter policy in simulation for zero-shot real table tennis. Mean hit-plus-land score: 1.83/2 with measured latency modeled, versus 1.33 at half latency. | Google DeepMind | Black-box search | ✓ checked |
| 2023-07-12 | Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy AdaptationCounterfactual demonstrations let users flag task-irrelevant concepts. The flagged concepts drive data augmentation to adapt policies under distribution shift, which reduced fine-tuning demos in user studies (ICML 2023). | MIT | Human corrections | ✓ checked |
| 2023-07-01 | DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution MisalignmentLLM produces plans plus execution constraints; a VLM continuously detects constraint violations during low-level skill execution and triggers immediate replanning, raising success and cutting completion time on arm and humanoid tasks. | Guo et al. | Test-time search | ✓ checked |
| 2023-06-27 | REFLECT: Summarizing Robot Experiences for Failure Explanation and CorrectionQueries an LLM on a hierarchical summary of multisensory robot experience to explain failures and guide a language planner's corrections; introduces the RoboFail dataset (CoRL 2023). · code | Liu et al. | Test-time search | ✓ checked |
| 2023-06-20 | RoboCat: A Self-Improving Generalist Agent for Robotic ManipulationSelf-improvement loop for a 1.18B visual-goal-conditioned transformer: fine-tune on 100-1000 demonstrations, collect new robot trajectories, hindsight-relabel goals including failed episodes, and retrain the generalist. | Google DeepMind | Advantage-weighted BC | ✓ checked |
| 2023-06-01 | LIV: Language-Image Representations and Rewards for Robotic ControlVIP plus CLIP InfoNCE gives language-conditioned value and reward from EPIC-KITCHENS. · code | UPenn, Meta AI | Rewards & progress | ✓ checked |
| 2023-05-05 | Deep RL at Scale: Sorting Waste in Office Buildings with a Fleet of Mobile ManipulatorsPI-QT-Opt on a 23-robot fleet with sim and curriculum bootstrapping; 84.35% sorting vs a 71% hand-engineered policy. | Everyday Robots; Robotics at Google | Off-policy critics | ✓ checked |
| 2023-04-20 | IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion PoliciesDiffusion behavior policy trained by BC separately from a generalized IQL critic; at evaluation, picks the highest-Q of N sampled actions. D4RL total 1213.8 vs IQL 1070.4 (reported-results setting). · code | UC Berkeley | Test-time search | ✓ checked |
| 2023-03-30 | Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical SolutionsStudies policy collapse when fine-tuning offline-RL agents online (data diversity, algorithm choice, replay distribution) and proposes conservative policy optimization for stable, sample-efficient online learning. | Luo et al. | Off-policy critics | ✓ checked |
| 2023-03-13 | Vision-Language Models as Success Detectors (SuccessVQA)Fine-tunes Flamingo 3B to answer 'did the robot successfully X?' on video clips; in-domain gear-insertion balanced accuracy 94-96% (bespoke ResNet 98-99%), but it degrades less under camera and distractor shifts; Ego4D remains hard (62%). | DeepMind, UC Berkeley | Rewards & progress | ✓ checked |
| 2023-03-09 | Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-TuningCalibrated CQL that keeps Q at or above a reference value to avoid unlearning; 11-task average 44 → 90 vs CQL 42 → 71. · code | UC Berkeley; Stanford | Off-policy critics | ✓ checked |
| 2023-03-02 | Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning (MEDAL++)Forward and backward image-based actor-critics with classifier rewards for reset-free real practice; cube grasping from OOD starts 0.08 → 0.82 vs BC. · code | Stanford University | Off-policy critics | ✓ checked |
| 2023-02-06 | Efficient Online Reinforcement Learning with Offline Data (RLPD: Reinforcement Learning with Prior Data)SAC with 50/50 offline/online batches, LayerNorm critics and ensembles; 2.5x the best reported result on sparse Adroit Door. The engine of SERL and HIL-SERL. · code | Oxford; UC Berkeley | Off-policy critics | ✓ checked |
| 2023-02-02 | Policy Expansion for Bridging Offline-to-Online Reinforcement Learning (PEX)Freeze the offline policy, add a new policy, and let a shared critic choose between their proposals by Boltzmann sampling; curves only. · code | Horizon Robotics | Off-policy critics | ✓ checked |
| 2023-01-10 | DreamerV3: Mastering Diverse Domains through World ModelsLearns a world model and trains behavior by imagined futures. DreamerV3 combines normalization and robust value learning to use one configuration across more than 150 diverse tasks. · code | Google DeepMind; University of Toronto | World models & eval | ✓ checked |
| 2023-01-03 | DADAgger: Disagreement-Augmented Dataset AggregationDAgger variant that queries the expert only on out-of-distribution states flagged by dropout-ensemble variance; matches DAgger on Car Racing and Half Cheetah with fewer expert queries. | Haridas et al. | Human corrections | ✓ checked |
| 2022-11-15 | Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment (Sirius)Deployment-time weighted BC; real Gear Insertion 14/32 to 29/32 over three rounds. · code | UT Austin | Human corrections | ✓ checked |
| 2022-10-11 | Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials (PTR)CQL-based offline RL pretraining on diverse multi-task robot data, then offline and autonomous online fine-tuning; learns new tasks from as few as 10 demos on a real WidowX. · code | UC Berkeley, Stanford | Off-policy critics | ✓ checked |
| 2022-09-30 | VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-TrainingA goal-conditioned value embedding from Ego4D via a dual RL objective; VIP-weighted RWR gets 100/90/60/90% on 4 real tasks. · code | Meta FAIR, UPenn | Rewards & progress | ✓ checked |
| 2022-08-22 | Self-Improving Robots and the Importance of DataLevine essay: a Roomba-scale fleet (~10 million service robots) would collect ~10 billion hours of embodied experience a year; offline RL on self-supervised, goal-conditioned tasks is proposed as the way to learn from it. | Sergey Levine | General RL | ✓ checked |
| 2022-08-12 | Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning (Diffusion-QL)Diffusion policy trained with a denoising BC loss plus a Q-maximizing term backpropagated through the chain; D4RL Gym average 88.0 vs CQL 77.6. · code | UT Austin; Twitter | Off-policy critics | ✓ checked |
| 2022-07-28 | Graph Inverse Reinforcement Learning from Diverse VideosGraphIRL abstracts diverse (including human) videos into object-interaction graphs and temporally matches them to measure task progress as a reward; outperforms hand-crafted rewards on real-robot pushing. · code | UC San Diego | Rewards & progress | ✓ checked |
| 2022-06-29 | Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human SupervisionAllocates M humans across N robots; defines Return on Human Effort and claims up to 8.8x higher ROHE than baselines. · code | UC Berkeley | Human corrections | ✓ checked |
| 2022-06-28 | DayDreamer: World Models for Physical Robot LearningDreamer on four real robots with no simulator: an A1 learns to walk in about 1 hour, while SAC on the same budget only rolls over. · code | UC Berkeley | World models & eval | ✓ checked |
| 2022-04-05 | Jump-Start Reinforcement Learning (JSRL)A guide policy runs the first h steps and RL learns the tail, handing over earlier as success rises; sim indiscriminate grasping with 20 demos: QT-Opt+JSRL 0.91 vs QT-Opt 0.00. | Google, UC Berkeley and others | Off-policy critics | ✓ checked |
| 2022-02-17 | Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization (HACO)Human-in-the-loop RL for driving: human takeovers during training yield proxy state-action values from partial demonstrations, giving safe, sample-efficient learning that beats RL and IL baselines. · code | Li et al. | Human corrections | ✓ checked |
| 2022-02-13 | Supported Policy Optimization for Offline Reinforcement LearningSPOT: VAE density estimator models behavior-policy support, giving a simple density-based regularizer pluggable into off-policy RL; supports offline-to-online fine-tuning without architectural changes. · code | Tsinghua University | Off-policy critics | ✓ checked |
| 2021-10-12 | Offline Reinforcement Learning with Implicit Q-Learning (IQL)Expectile value regression that never queries unseen actions; D4RL antmaze total 378.0 vs CQL 303.6. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2021-09-17 | ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation LearningRobot-gated on novelty plus risk; sim peg insertion 73/100 autonomous success vs HG-DAgger 57/100, with fewer training interventions than HG-DAgger but more human-teleoperated actions. · code | UC Berkeley AUTOLAB | Human corrections | ✓ checked |
| 2021-07-21 | Demonstration-Guided Reinforcement Learning with Learned SkillsSkiLD: demo-guided RL that follows demonstrated skills learned from offline data rather than primitive actions; substantial gains over prior demo-guided RL on simulated long-horizon maze navigation and robot manipulation. · code | Pertsch et al. | Off-policy critics | ✓ checked |
| 2021-07-08 | RMA: Rapid Motor Adaptation for Legged Robots (adjacent)Sim-trained base policy plus adaptation module lets an A1 quadruped adapt to new terrains in fractions of a second, deployed without real-world fine-tuning (RSS 2021). · code | Kumar et al. | Sim-to-real | ✓ checked |
| 2021-06-09 | PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingOff-policy preference-based RL: learns a reward model from human preference queries over behavior clips, relabels replay experience as it changes, and uses unsupervised pre-training for feedback efficiency. · code | UC Berkeley | Rewards & progress | ✓ corrected |
| 2021-06-07 | XIRL: Cross-embodiment Inverse Reinforcement LearningLearns task-progress visual embeddings from cross-embodiment demo videos via temporal cycle-consistency; negative embedding distance to goal serves as RL reward, generalizing to unseen embodiments (CoRL'21 oral). · code | Google; Stanford | Rewards & progress | ✓ checked |
| 2021-03-31 | LazyDAgger: Reducing Context Switching in Interactive Imitation LearningLazyDAgger adds asymmetric takeover/release thresholds and noisy supervisor actions to reduce switching. On a physical ABB YuMi fabric task, execution succeeds in 8/10 trials versus SafeDAgger’s 2/10, with 21 versus 53 switches. | Hoque et al. | Human corrections | ✓ checked |
| 2021-03-31 | Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos (DVD: Domain-agnostic Video Discriminator)A 'same task?' video classifier trained on human plus robot videos gives a reward from a single human demo; used with visual MPC, not RL. · code | Stanford | Rewards & progress | ✓ checked |
| 2020-12-12 | Human-in-the-Loop Imitation Learning using Remote Teleoperation (Intervention Weighted Regression, IWR)Balanced sampling of intervention vs robot data; Threading final 87.3±5.0 vs HG-DAgger 75.3±8.1 and Full Demos 76.7±2.3 (sim). | Stanford; UT Austin | Human corrections | ✓ checked |
| 2020-11-19 | Parrot: Data-Driven Behavioral Priors for Reinforcement LearningPre-trains a behavioral prior from successful trials across many tasks, then uses it to speed RL on new image-based, sparse-reward manipulation tasks; beats prior methods by a substantial margin. | UC Berkeley | Off-policy critics | ✓ checked |
| 2020-11-14 | PLAS: Latent Action Space for Offline Reinforcement LearningOffline RL whose actor outputs a tanh-bounded latent through a frozen CVAE behavior decoder, constraining actions through its learned data distribution; evaluated on simulated continuous control and real deformable-object manipulation. · code | CMU | Residual & steering | ✓ checked |
| 2020-10-29 | Recovery RL: Safe Reinforcement Learning with Learned Recovery ZonesSeparate task and recovery policies with a safety critic pretrained on offline data; 2-20x better constraint-violation/success trade-off in sim, 3x on a physical robot. · code | Thananjeyan et al. | Off-policy critics | ✓ checked |
| 2020-10-22 | Accelerating Reinforcement Learning with Learned Skill PriorsSPiRL: jointly learns a skill embedding space and state-conditioned skill prior from offline data, then uses the prior to regularize max-entropy RL on new navigation and manipulation tasks (CoRL 2020). · code | USC | Off-policy critics | ✓ checked |
| 2020-07-21 | EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RLBackup operator taking the max Q over N actions sampled from a behavior model, keeping the policy in the data's support with no explicit policy network. Matches or beats offline SOTA on D4RL and is competitive with SAC online. | Google Brain / University of Toronto | Off-policy critics | ✓ checked |
| 2020-07-08 | Self-Supervised Policy Adaptation during DeploymentPAD: keeps updating the policy with a self-supervised objective at deployment, no reward needed; improves generalization in 31 of 36 environments, including real robot manipulation. · code | Hansen et al. | Test-time search | ✓ checked |
| 2020-07 | Learning from Interventions: Human-robot interaction as both explicit and implicit feedbackExpert Intervention Learning (EIL) uses both interventions and non-interventions as feedback on state quality and action optimality; learns collision avoidance from about one minute of expert control. | Princeton; University of Washington | Human corrections | ✓ checked |
| 2020-06-26 | Critic Regularized Regression (CRR)Offline critic-filtered BC (binary or exp weights). Humanoid Run: CRR exp 586±6 vs BC 382±2 and D4PG 1±1. · code | DeepMind | Advantage-weighted BC | ✓ checked |
| 2020-06-16 | AWAC: Accelerating Online Reinforcement Learning with Offline DatasetsAWAC combines a TD critic with advantage-weighted actor regression for offline pretraining and online fine-tuning. Sparse simulated Adroit success after fine-tuning: door 95% at 800k steps, relocate 54% at 4M steps. · code | UC Berkeley | Off-policy critics | ✓ checked |
| 2020-06-08 | Conservative Q-Learning for Offline Reinforcement Learning (CQL)Pessimistic critic that pushes Q down on policy actions and up on data actions; big wins on mixed-quality D4RL data. · code | UC Berkeley; Google Brain | Off-policy critics | ✓ checked |
| 2020-04-27 | The Ingredients of Real-World Robotic Reinforcement Learning (R3L)SAC from pixels plus a goal-image classifier reward and a perturbation controller for reset-free real-world practice. | UC Berkeley; University of Oxford; University of Washington | Off-policy critics | ✓ checked |
| 2019-10-01 | Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement LearningThe base recipe: regress V, then do exp(A/beta)-weighted BC on buffer actions. Hopper-v2 3405±121 vs SAC 2769±552 and PPO 1391±304. · code | UC Berkeley | Advantage-weighted BC | ✓ checked |
| 2019-09-26 | Scaling data-driven robotics with reward sketching and batch reinforcement learningHuman-drawn progress curves train a ranking-loss reward that relabels a 400+ hour robot log for offline distributional RL. · code | DeepMind | Rewards & progress | ✓ checked |
| 2019-04-16 | End-to-End Robotic Reinforcement Learning without Reward Engineering (VICE-RAQ)A success classifier from 80 goal images plus 25-75 yes/no queries; 100% real success on pushing, draping and bookshelf, where a naive classifier gets 0%. · code | UC Berkeley | Rewards & progress | ✓ checked |
| 2019-04-12 | Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations (T-REX)T-REX learns a reward function from approximately ranked suboptimal demonstrations and extrapolates beyond them, often exceeding twice the best demonstration's performance on benchmark tasks. · code | Brown et al. | Rewards & progress | ✓ checked |
| 2018-12-15 | Residual Policy Learningpi = pi_base + f_theta with DDPG+HER on a frozen hand controller or MPC. On sim Push it reaches 0.9 success with about 10x fewer samples than learning from scratch. · code | MIT | Residual & steering | ✓ checked |
| 2018-12-07 | Residual Reinforcement Learning for Robot ControlTD3 residual on a hand-engineered controller. Real Sawyer insertion with misaligned blocks: 15/20 vs 2/20 for the controller, in about 8K samples (about 3 h). | Siemens; UC Berkeley; TU Hamburg | Residual & steering | ✓ checked |
| 2018-10-05 | HG-DAgger: Interactive Imitation Learning with Human ExpertsHuman takeovers supply labels for iterative imitation learning. In the real-car test’s first 5,000 samples, HG-DAgger records no collisions or road departures, versus DAgger’s one each and BC’s one collision and six departures. | Stanford University; UIUC | Human corrections | ✓ checked |
| 2018-10-02 | CEM-RL: Combining evolutionary and gradient-based methods for policy searchCEM over actor weights, where half the population gets TD3 gradient steps before evaluation. HalfCheetah 10725 vs TD3 9630 at 1M steps. · code | Sorbonne / CNRS ISIR | Black-box search | ✓ checked |
| 2018-07-22 | EnsembleDAgger: A Bayesian Approach to Safe Imitation LearningDAgger extension using neural-network ensemble variance as a confidence measure to decide when the novice acts versus the expert; tested on inverted pendulum and MuJoCo HalfCheetah (IROS 2019). | Menda et al. | Human corrections | ✓ checked |
| 2018-06-27 | QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic ManipulationActor-free Q-learning with CEM on 580k real grasps; adding 28k on-policy grasps took unseen-object grasping from 87% to 96%. · code | Google Brain; X; UC Berkeley | Off-policy critics | ✓ checked |
| 2018-03-19 | Simple random search provides a competitive approach to reinforcement learning (Augmented Random Search, ARS). NeurIPS version title: "Simple random search of static linear policies is competitive for reinforcement learning"Antithetic random search on linear policies, with reward-std scaling, state normalization and top-direction selection, is competitive on MuJoCo locomotion. · code | UC Berkeley | Black-box search | ✓ checked |
| 2017-09-28 | Overcoming Exploration in Reinforcement Learning with DemonstrationsDDPG + HER plus a demo replay buffer, a Q-filtered behavior-cloning loss and optional resets to demo states; in simulation it learns multi-block stacking where HER and BC alone fail, often outperforming the demonstrator (ICRA 2018). | Nair et al. | Off-policy critics | ✓ checked |
| 2017-09-28 | Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations (DAPG: Demo Augmented Policy Gradient)BC on 25 demos, then NPG with a decaying demo-likelihood term. Reaches 90% on Adroit tasks in 30-55 iterations (3.33-6.1 robot-hours), where sparse-reward RL from scratch never gets there on 3 of 4 tasks. · code | UW, OpenAI, UC Berkeley, IIT | On-policy PG | ✓ checked |
| 2017-07-27 | Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards (DDPGfD)About 100 human demos kept permanently in DDPG's prioritized replay buffer; learns sparse-reward insertion without a shaped reward. | DeepMind | Off-policy critics | ✓ checked |
| 2017-07-17 | Reverse Curriculum Generation for Reinforcement LearningStarts from one goal state and adaptively expands training starts outward, forming a reverse curriculum for sparse-reward simulated navigation and fine-grained manipulation (CoRL 2017). | UC Berkeley | General RL | ✓ checked |
| 2017-06-12 | Deep Reinforcement Learning from Human PreferencesFits a Bradley-Terry reward model online from trajectory-clip comparisons. Evaluates 700 human queries on eight MuJoCo tasks; synthetic feedback nearly matches true-reward RL, with human feedback typically slightly less effective. | OpenAI, DeepMind | Rewards & progress | ✓ checked |
| 2017-06-06 | Parameter Space Noise for ExplorationClassic precursor. Episode-fixed weight noise for coherent exploration in DQN, DDPG and TRPO. DQN with parameter noise beats ES on 15 of 21 Atari games with 25x less data. · code | OpenAI; KIT; UC Berkeley | Black-box search | ✓ checked |
| 2017-04-12 | Deep Q-learning from DemonstrationsDQfD combines TD updates with supervised classification of demonstrator actions using prioritized replay of demo data; better initial scores than PDD DQN on 41 of 42 Atari games. | Hester et al. | Off-policy critics | ✓ checked |
| 2017-03-27 | DART: Noise Injection for Robust Imitation LearningInjects optimized noise into the supervisor's policy during demonstration to collect recovery data, an off-policy alternative to DAgger; 62% gain over behavior cloning on real robot grasping. | Laskey et al. | Human corrections | ✓ checked |
| 2017-03-10 | Evolution Strategies as a Scalable Alternative to Reinforcement LearningWeight-space ES as a scalable alternative to RL. Humanoid reached a score of 6000 in 10 minutes on 1,440 cores vs 657 minutes on one 18-core machine. · code | OpenAI | Black-box search | ✓ checked |
| 2016-12-20 | Unsupervised Perceptual Rewards for Imitation LearningSplits a handful of demo videos into steps and scores progress with features selected from a frozen ImageNet Inception network; this staged reward replaces an IMU door sensor for real-robot PI2 door opening, even from human-hand demos. | Google Brain | Rewards & progress | ✓ checked |
| 2016-05-20 | Query-Efficient Imitation Learning for End-to-End Autonomous Driving (SafeDAgger)SafeDAgger extends DAgger with a safety policy predicting when the learner deviates from the reference, cutting expert queries; converges faster in a car racing simulator. | Zhang et al. | Human corrections | ✓ checked |
| 2016-04-04 | The CMA Evolution Strategy: A TutorialReference derivation of CMA-ES. Default population lambda = 4 + floor(3 ln n); covariance is accumulated across generations. · code | Inria | Black-box search | ✓ checked |
| 2014-07-13 | Robots that can adapt like animals (Intelligent Trial and Error, IT&E)MAP-Elites builds a simulation behavior map; Bayesian optimization searches it on a damaged real hexapod. Compensatory gaits are 3–7x faster than the reference gait under the same damage, found in a median eight trials. | UPMC/CNRS ISIR; Inria; University of Wyoming | Black-box search | ✓ checked |
| 2010-11-02 | A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (DAgger)Aggregates expert labels on the learner’s own states and retrains. In Super Tux Kart, the policy almost never fell off after five iterations, while supervised behavior cloning did not improve with more expert data. | Carnegie Mellon University | Human corrections | ✓ checked |
| 2010-11 | A Generalized Path Integral Control Approach to Reinforcement Learning (PI2, Policy Improvement with Path Integrals)Path-integral sample-perturb-reweight update with no gradient and no learning rate. The paper reports it learning about an order of magnitude faster than the gradient-based methods it compares against. | USC | Black-box search | ✓ checked |
| 2008-12 | Policy Search for Motor Primitives in Robotics (PoWER: Policy learning by Weighting Exploration with the Returns)Imitate once, then return-weighted parameter perturbation on the real robot. Ball-in-a-cup regularly succeeds after about 75 rollouts. | MPI for Biological Cybernetics | Black-box search | ✓ checked |
| 2004-05 | Policy Gradient Reinforcement Learning for Fast Quadrupedal LocomotionFinite-difference hill climbing of a 12-parameter gait on 3 real Aibos: 291 ± 3 mm/s after about 3 hours. A roughly hand-tuned start worked best. | UT Austin | Black-box search | ✓ checked |
| Not stated | Welcome to the Era of ExperiencePosition essay (preprint chapter of MIT Press book "Designing an Intelligence"): agents will gain superhuman ability by learning predominantly from their own experience via streams, grounded actions, grounded rewards, and planning. | General RL | ✓ corrected | |
| Not stated | Cargo Cults, Data Flywheels, and Novelty PumpsEssay: more robots repeating the same tasks yield redundant data; real improvement needs a labor-intensive "novelty pump" of ops teams seeking new states and tasks, not an automatic flywheel. | Industry loops | ✓ checked | |
| Not stated | Enact: Real-world RL environments for robotics models (YC S26)Startup that rolls out customer policies, finds failures and collects targeted recovery data; pi0.5 packing went from 90/100 (200 demos) to 99/100 with 50 added recovery demos (self-reported). | Enact | Industry loops | ✓ checked |
| Not stated | Fern (Ishiki Labs): RL environments for robotics companiesYC W26 startup: world-model RL environments replicating production scenes; adds RL heads atop customers' existing policies to raise speed and success weekly. No quantitative results published. | Fern / Ishiki Labs | Industry loops | ✓ checked |