Issue evidence · Run · huggingface/trl

Iterable datasets with at least as many shards as processes skip and duplicate prompts in multi-process GRPO

GitHub issue: https://github.com/huggingface/trl/issues/7690

Reproduced

With an iterable dataset that has at least as many shards as processes, multi-process GRPO trains on some prompts twice and never sees others.

What Badgr ran

Ran the reporter's script as two CPU ranks (rendezvous over loopback) with TRL 1.15.0, num_shards 1 and 4, 16 prompts, on macOS. TRL main could not be used here because it requires Triton.

What came back

  • num_shards=1: trained on 16/16 prompts, none repeated.
  • num_shards=4: trained on 8/16 prompts; never seen P1, P3, P4, P6, P9, P11, P12, P14; seen more than once P0, P2, P5, P7, P8, P10, P13, P15.

The corrected-output check was not run, and TRL main was not tested.

Checked 2026-10-14 in Badgr’s local development environment.

← All issue evidence