Align tiny Gemma2 config with google/gemma-2-2b-it

[albertvillanova]

This PR aligns the tiny-Gemma2ForCausalLM generator config with its reference model, google/gemma-2-2b-it.

Part of #7137. Depends on #7227, which fixes the head_dim scale-down; this PR is branched off it and only the last commit belongs here.

Motivation

The script builds Gemma2Config from architecture arguments only, so eos_token_id falls back to the class default of 1, a scalar, where the reference ships the list [1, 107]. This is the case #7137 opens with: generation stops on a single id in the tiny model, while every user of the real model stops on either, so the multi-id stopping path has no coverage.

Solution

Mirror the reference values:

• eos_token_id=[1, 107] (reference value; class default is the scalar 1)
• vocab_size=256000, matching len(tokenizer.vocab) today and pinned so the config no longer depends on the tokenizer files at generation time

cache_implementation and hidden_act stay as differences: the reference repo carries both, but Gemma2Config models neither, the same situation as rope_scaling in #7217, use_mrope in #7135 and use_mambapy in #7139.

Before, with the head_dim reduction already applied by #7227

[config_diff] google/gemma-2-2b-it vs tiny (11 differences)
  cache_implementation                             hybrid                             → <missing>
  eos_token_id                                     [2 items] [1, 107]                 → 1
  head_dim                                         256                                → 2
  hidden_act                                       gelu_pytorch_tanh                  → <missing>
  hidden_size                                      2304                               → 8
  intermediate_size                                9216                               → 32
  layer_types                                      [26 items] ['sliding_attention', … → [2 items] ['sliding_attention', '…
  num_attention_heads                              8                                  → 4
  num_hidden_layers                                26                                 → 2
  num_key_value_heads                              4                                  → 2
  query_pre_attn_scalar                            256                                → 2

After

[config_diff] google/gemma-2-2b-it vs tiny (10 differences)
  cache_implementation                             hybrid                             → <missing>
  head_dim                                         256                                → 2
  hidden_act                                       gelu_pytorch_tanh                  → <missing>
  hidden_size                                      2304                               → 8
  intermediate_size                                9216                               → 32
  layer_types                                      [26 items] ['sliding_attention', … → [2 items] ['sliding_attention', '…
  num_attention_heads                              8                                  → 4
  num_hidden_layers                                26                                 → 2
  num_key_value_heads                              4                                  → 2
  query_pre_attn_scalar                            256                                → 2

Every remaining row is the deliberate size reduction, apart from cache_implementation and hidden_act. Produced with print_config_diff at transformers==4.56.2, the version check_transformers_version() pins. The Hub repo needs regenerating for this to take effect.

Changes

• Pin vocab_size to the reference's 256000
• Set eos_token_id from the reference config

---

Note

Low Risk
Only changes the tiny-model generation script and published test artifact config; no runtime library or production inference paths.

Overview
Updates the tiny Gemma2 generator so its Gemma2Config matches google/gemma-2-2b-it on fields that affect generation semantics, not just shrunk architecture.

eos_token_id is set to [1, 107] instead of the class default scalar 1, so the tiny Hub model exercises the same multi-EOS stopping behavior as the reference. vocab_size is pinned to 256000 rather than len(tokenizer.vocab), decoupling config generation from tokenizer files while staying aligned with the reference today.

Remaining print_config_diff gaps vs the full model are intentional mini dimensions and known unsupported fields (cache_implementation, hidden_act). The Hub tiny repo must be regenerated for the change to land in published artifacts.

^{Reviewed by Cursor Bugbot for commit 9ba41bb. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[albertvillanova]

The CI model has been regenerated and the corresponding Hub PR is open. It needs to be merged once this PR is approved:

• https://huggingface.co/trl-internal-testing/tiny-Gemma2ForCausalLM/discussions/2

[qgallouedec]

apart from cache_implementation and hidden_act

why not aligning them then?

[albertvillanova]

Tested with MODEL_REVISIONS pointing at the regenerated model, trl-internal-testing/tiny-Gemma2ForCausalLM at refs/pr/2, on top of current main: https://github.com/huggingface/trl/actions/runs/35091947908

The suite is green.

[albertvillanova]

Merging this as part of the approved series.

The rows left un-aligned here (cache_implementation and hidden_act) are under discussion in #7137. Merging does not settle it: mirroring them later is additive, not a revert.