fix: preserve configured model order

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417

[bussyjd]

Summary

This PR removes the hidden model-quality ranking heuristic from Hermes/OpenClaw model selection.

Instead of guessing the best primary model from provider names, parameter-count tags, or hardcoded model-family defaults, model.Rank now treats the configured LiteLLM model_list order as the source of truth. The only remaining adjustment is that known embedding-only model names containing embed are kept as fallbacks behind chat-capable models so they do not become the default chat model by accident.

What we have today

Today the primary model is selected by a hidden heuristic:

flowchart TD
A["Available model names"] --> B["model.Rank"]
B --> C{"Cloud-looking name?"}
C -->|"claude / gpt / o-series"| D["Sort by cloudPrecedence"]
C -->|"everything else"| E["Local model path"]
E --> F{"Has parseable size token?"}
F -->|"llama3.2:1b / qwen3.5:9b"| G["Bigger parameter count wins"]
F -->|"no size token"| H{"Known family prefix?"}
H -->|"llama3.3 / qwen3 / etc"| I["Use untaggedFamilyDefaults"]
H -->|"unknown"| J["rank = 0"]
D --> K["Primary + fallbacks"]
G --> K
I --> K
J --> K

Loading

That means the stack can override the order the user configured because the code thinks, for example, that a cloud-looking name or a larger-size local tag is better. That is too much hidden product policy for a generic model router.

Proposed behavior in this PR

flowchart TD
A["Configured LiteLLM model_list"] --> B["Preserve order"]
B --> C["First chat-capable model is primary"]
B --> D["Remaining models stay fallbacks in configured order"]
E["Embedding-only entries"] --> F["Move behind chat-capable models"]

Loading

This makes model selection predictable:

Pattern	Behavior
obol model setup --provider openai --model gpt-4.1	gpt-4.1 becomes primary.
obol model setup --provider anthropic --model claude-sonnet...	The configured Claude model becomes primary.
obol model setup custom --model qwen36-fast on a fresh stack	qwen36-fast becomes primary.
Multiple configured models	The first configured chat-capable model wins; the rest are fallbacks in order.
Embedding model before chat model	Embedding model is moved behind chat models.

UX question

@OisinKyne, what UX would you prefer here?

The implementation in this PR assumes configured order should be the contract, but there are two product choices worth making explicitly:

1. Should we add a first-class command like obol model prefer <model> so users can reorder the primary without remove/re-add?
2. Should obol model setup always place explicitly selected models before wildcard entries such as anthropic/* or openai/*, so the model the user selected is visibly the primary?

My recommendation is yes to both, but I kept this PR scoped to removing the hidden ranking heuristic and documenting the new selection contract.

Validation

Passed:

go test ./internal/model ./internal/hermes ./internal/openclaw -run 'TestRank' -count=1
git diff --check

Known unrelated note: go test ./... on current main still hits the existing Hermes extras expectation mismatch. That is handled in the separate flow/RBAC QA branch, not here, to keep this PR focused.

[OisinKyne]

favouring #417