Skip to content

MCP Server Part 6: Format callback results for LLM consumption - #3748

Merged
KoolADE85 merged 9 commits into
mcpfrom
feature/mcp-formatted-tool-results
May 8, 2026
Merged

MCP Server Part 6: Format callback results for LLM consumption#3748
KoolADE85 merged 9 commits into
mcpfrom
feature/mcp-formatted-tool-results

Conversation

@KoolADE85

Copy link
Copy Markdown
Contributor

Summary

  • Add formatters that enrich callback responses with LLM-friendly content (images, markdown tables) for certain data types
  • PlotlyFigureResult — renders Graph.figure outputs as PNG images via kaleido, returned as ImageContent
  • DataFrameResult — renders DataTable.data and AgGrid.rowData outputs as markdown tables, returned as TextContent

Manual verification:

importplotly.expressaspxfromdashimportDash, html, dcc, Input, Output, dash_tabledf=px.data.gapminder().query("year == 2007 and continent == 'Europe'")
app=Dash(__name__)
app.layout=html.Div([
dcc.Dropdown(id="metric", options=["gdpPercap", "lifeExp", "pop"], value="gdpPercap"),
dcc.Graph(id="chart"),
dash_table.DataTable(id="table"),
])
@app.callback(Output("chart", "figure"), Input("metric", "value"))defupdate_chart(metric):
"""Bar chart of the selected metric across European countries."""returnpx.bar(df, x="country", y=metric, title=f"{metric} in Europe (2007)")
@app.callback(Output("table", "data"), Input("metric", "value"))defupdate_table(metric):
"""Top 5 countries for the selected metric."""returndf.nlargest(5, metric)[["country", metric]].to_dict("records")
withapp.server.app_context():
fromdash.mcp.primitives.tools.callback_adapter_collectionimportCallbackAdapterCollectionfromdash.mcp.primitives.tools.callback_utilsimportrun_callbackfromdash.mcp.primitives.tools.resultsimportformat_callback_responsecollection=CallbackAdapterCollection(app)
foradapterincollection:
response=run_callback(adapter, {"metric": "lifeExp"})
result=format_callback_response(response, adapter)
foriteminresult.content:
ifitem.type=="text":
print(f"[text] {item.text[:200]}...")
elifitem.type=="image":
print(f"[image] {item.mimeType}, {len(item.data)} chars base64")
# update_chart: [text] JSON + [image] PNG# update_table: [text] JSON + [text] markdown table

@github-actions

github-actionsBot commented Apr 16, 2026

Copy link
Copy Markdown

Thank you for your contribution to Dash! 🎉

This PR is exempt from requiring a linked issue due to its labels.

@KoolADE85
KoolADE85force-pushed the feature/mcp-pattern-matching-schemas branch from 9b3063f to 7451883CompareApril 21, 2026 17:27
@KoolADE85
KoolADE85force-pushed the feature/mcp-formatted-tool-results branch from 0e54d74 to 8a41b76CompareApril 21, 2026 17:28
@KoolADE85
KoolADE85force-pushed the feature/mcp-pattern-matching-schemas branch from 7451883 to 5fbf497CompareApril 22, 2026 21:13
@KoolADE85
KoolADE85force-pushed the feature/mcp-formatted-tool-results branch from 8a41b76 to 4fb9d4aCompareApril 22, 2026 21:31
@KoolADE85
KoolADE85force-pushed the feature/mcp-pattern-matching-schemas branch from 5fbf497 to b564279CompareApril 23, 2026 20:04
@KoolADE85
KoolADE85force-pushed the feature/mcp-formatted-tool-results branch from 4fb9d4a to 10a544bCompareApril 23, 2026 20:15
@KoolADE85
KoolADE85force-pushed the feature/mcp-pattern-matching-schemas branch from b564279 to e2aec3dCompareApril 30, 2026 15:05
@KoolADE85
KoolADE85force-pushed the feature/mcp-formatted-tool-results branch 3 times, most recently from 2acb39d to 04ca2feCompareApril 30, 2026 16:32
@KoolADE85
KoolADE85 changed the base branch from feature/mcp-pattern-matching-schemas to mcpMay 6, 2026 21:29
@KoolADE85
KoolADE85force-pushed the feature/mcp-formatted-tool-results branch from 04ca2fe to 8f9d5a0CompareMay 6, 2026 21:31

@camdecostercamdecoster left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I add a few minor suggestions.

formatters are called per output property and may add additional
content items (images, markdown, etc.).
"""
content: list[Any] = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would using a dict instead of a list make more sense here? Then you wouldn't have to iterate through the list down below (assuming you could figure out which formatter to use before calling it).

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this further, I'd like to keep the current structure for two reasons:

  1. consistency: the pattern of "iterate providers, each decides if it applies" is already used for input schemas, tool descriptions, and resource providers; I think it's valuable to keep a consistent pattern here too.
  2. flexibility: iterating lets each formatter decide dynamically whether it applies. While it's fairly static today, we are leaving room for multiple formatters to enrich the same output without restructuring later.

Comment threaddash/mcp/primitives/tools/results/result_dataframe.py Outdated
Comment threaddash/mcp/primitives/tools/results/result_dataframe.py Outdated
Comment threaddash/mcp/primitives/tools/results/result_dataframe.py Outdated
)

GENERIC_FIGURE = PropRole(
PLOTLY_FIGURE = PropRole(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this change necessary?

@KoolADE85KoolADE85May 8, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a style choice: I wanted it to read more clearly when it's subsequently used in the result formatter, that we are actually concerned with formatting Plotly figures, not more "generic" figures.

Comment threaddash/mcp/primitives/tools/results/result_dataframe.py Outdated
Comment threaddash/mcp/primitives/tools/results/result_plotly_figure.py Outdated
Comment threaddash/mcp/primitives/tools/results/result_plotly_figure.py Outdated
@KoolADE85
KoolADE85 merged commit 7a55005 into mcpMay 8, 2026
26 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@KoolADE85@camdecoster