Skip to content

improvement: extract and deliver chart PNGs from code_run output#5

Open
hjpinheiro wants to merge 1 commit into
daytona:mainfrom
hjpinheiro:improvement/run-code-chart-delivery
Open

improvement: extract and deliver chart PNGs from code_run output#5
hjpinheiro wants to merge 1 commit into
daytona:mainfrom
hjpinheiro:improvement/run-code-chart-delivery

Conversation

@hjpinheiro

Copy link
Copy Markdown

Summary

When Python code generates charts (matplotlib, seaborn, plotly, etc.), the Daytona SDK captures them as base64 PNG images in response.artifacts.charts. These were previously discarded.

This PR extracts the chart PNGs and delivers them as blob messages so users see rendered charts directly in the Dify chat.

Changes

Chart PNG delivery

# Extract PNG from chart elements and deliver as image
for chart in response.artifacts.charts:
    for element in chart.elements:
        if element.png:
            png_bytes = base64.b64decode(element.png)
            yield self.create_blob_message(blob=png_bytes, meta={"mime_type": "image/png"})

Chart metadata in JSON response

{
    "exit_code": 0,
    "output": "...",
    "charts_count": 2,
    "charts": [{"type": "bar", "title": "Sales by Quarter"}, ...]
}

Text output

Added create_text_message(response.result) so the agent sees code output in chat (previously only JSON was returned).

SDK structure

The chart data path (verified against daytona==0.187.0):

ExecuteResponse.artifacts: ExecutionArtifacts
  └── .charts: list[Chart]
       └── .elements: list[GeneratedChartElement]
            ├── .png: str (base64-encoded PNG)
            ├── .type: str (chart type)
            └── .title: str (chart title)

All field access uses defensive getattr with None defaults.

Testing

  • py_compile passes

The code_run response includes chart artifacts (matplotlib/seaborn plots,
etc.) as base64 PNGs in response.artifacts.charts, but these were
previously ignored.

Now extracts chart PNGs and delivers them as blob messages so users see
the rendered charts directly in the chat. Also adds chart metadata
(type, title) and charts_count to the JSON response.

Additionally adds create_text_message for the text output (previously
only JSON was returned, so agents couldn't see the output in chat).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants