TL;DR of the paper, generated automatically.

A paper says a weak decision model can still train a good RL policy.

• Finding: as actors, all six models drift to an easier subgoal and skip the prerequisite.
• Judging: as judges that see the full trajectory, they credit the steps toward that prerequisite.
• Result: as teacher (JEV), Qwen2.5-1.5B success rises: ALFWorld 72.8% to 82.9%, WebShop 56.8% to 71.4%.