Created

Hands-on evaluation · James Meadlock & Milo · Flint 0.4.1

Flint-chart: Putting a Compiler Between Agents and Bad Plots

Bottom line. Flint gets the architecture right: let a probabilistic model describe meaning, then let deterministic software handle scales, formats, labels, spacing, and backend syntax. Our compiler and hardened MCP tests passed. Our deliberately unfair architecture-diagram stress test did not. Flint is useful infrastructure for charts—not an excuse to stop looking at the picture.

Daniel Lougen’s post put Flint on my radar. His point was simple and correct: AI can write plenty of chart code while still making a bad plot. A repository designed specifically for agentic chart-making is worth attention.

Flint0.4.1
MCP tools5 / 5
TopologyPASS
Pure heroFAIL
npm audit0 known

What Flint changes

Today an agent usually writes a native Vega-Lite, ECharts, Plotly, Chart.js, or Excel specification directly. That forces the model to coordinate dozens of coupled details: field parsing, aggregation, axes, domains, baselines, legends, label density, padding, colors, and renderer-specific syntax.

Flint inserts a compact intermediate language:

{
  "semantic_types": {
    "period": "YearMonth",
    "change": "Profit"
  },
  "chart_spec": {
    "chartType": "Heatmap",
    "encodings": {
      "x": {"field": "period"},
      "color": {"field": "change"}
    }
  }
}

YearMonth tells the compiler that 202601 is not an arbitrary integer. Profit tells it that positive and negative values have a meaningful zero. Flint can derive parsing, labels, a diverging scale, a zero midpoint, and layout choices in ordinary code instead of hoping the model remembers every renderer knob.

The paper reports Flint specifications were 85% shorter on average than native backend code in its exploration case. That is not just token thrift. A smaller output surface means fewer opportunities for malformed JSON and subtle visual contradictions.

How we wired Flint into Hermes

We did not add Flint to the production Hermes configuration. For this trial, Hermes launched the pinned MCP server as an isolated local stdio process, sent semantic intent plus inline public-safe data, and received validated backend artifacts. File-backed data was disabled at the MCP boundary.

Hermes and Flint pipeline: Hermes sends semantic chart intent and inline data through a hardened local stdio MCP process to Flint's semantic compiler, which emits ECharts SVG and PNG artifacts for tests and mandatory human visual review.

The tested path. Hermes makes the semantic choices; Flint validates and compiles them; ECharts renders repeatable artifacts; tests and a human visual-quality gate decide whether anything is publishable. Select the image for the editable SVG.

The deliberately unfair stress test

We tested something harder than a bar chart: recreate our real Local LLM Stack routing diagram from an edge list. This is exactly where we expected Flint’s abstraction to strain. Rich architecture cards, fixed hierarchy, annotations, fallback semantics, and carefully routed connectors are not the same problem as plotting quantitative data.

Improved custom Local LLM Stack diagram: Hermes Gateway routes directly to the dual-Spark default, M5 specialists, M3 bench, and a dashed cloud text fallback.

The target and final editorial diagram. This is custom SVG—not Flint output. It makes Hermes the routing hub, keeps compute lanes distinct, and treats the route guide as documentation rather than a fourth compute node. Select the image for the editable SVG.

Method

What actually happened

Pure Flint circular Network Graph candidate with sparse radial nodes and rotated labels.
Pure Flint Network Graph. Correct nodes and edges; poor editorial hierarchy, radial labels, and a clipped Spark label.
Pure Flint vertical Sankey candidate with one wide source band and four destination bands.
Pure Flint Sankey. Correct topology and clean bands; clipped destination labels and a misleading implication that route priority is flow volume.
GateNetworkSankey
Compiles / rendersPASSPASS
All approved nodes / edgesPASSPASS
Repeatable outputPASSPASS
Route meaning is obviousFAILPARTIAL
Blog-width labelsFAILFAIL
Better than designed SVGNONO
The important distinction: Flint succeeded as a compiler. It did not succeed as the design system for this particular artifact. A valid backend spec and a rendered image prove mechanical correctness—not semantic clarity or editorial quality.

The better diagram

The custom v2 above is meaningfully better than our earlier diagram in three ways:

It would be dishonest to call that image “made by Flint.” Fixed placement, rich cards, gutter routing, typography, and annotation structure are the design. Flint’s pure outputs remain above as the actual evidence.

Where Flint fits

WorkFitWhy
Benchmark bars, lines, heatmapsStrongSemantic types and automatic layout attack common agent mistakes directly.
Operational plots and compact dashboardsStrongShort specs, validation, deterministic compilation, multiple backends.
Editable Excel chartsPromisingOne semantic input can target native editable artifacts.
Simple networks / flowsConditionalUseful when a standard graph layout communicates enough.
Editorial architecture diagramsWeakRich nodes, fixed composition, annotations, and route semantics exceed the abstraction.

My default policy: use Flint for charts whose truth lives in data encodings. Use designed SVG for diagrams whose truth lives in composition.

Safety and limits

Verdict. Daniel was right to be excited. Flint moves brittle chart expertise out of prompts and into testable software. That is the right direction. Just keep the visual-review gate—and keep a real diagram tool nearby.

Sources