Hermes Agent Tracing with Litefuse

Hermes Agent is Nous Research’s terminal-based AI assistant with a pluggable tool-calling and gateway architecture. This integration installs as a Hermes plugin under ~/.hermes/plugins/litefuse/. The plugin runs in-process inside Hermes’ Python interpreter, subscribes to ten lifecycle hook events, and emits one Litefuse trace per user turn (the same trace shape as the Pi and Claude Code integrations, so one dashboard query works across all your agents).

The plugin works on every Hermes surface — CLI (hermes -z, hermes chat -q), Gateway (Feishu, Discord, Slack, Telegram, WhatsApp, IRC), and TUI.

For AI — automated install

If you’re chatting with Hermes Agent right now, paste this prompt and the agent will handle the whole install end-to-end:

Read https://litefuse.ai/SKILL.md and follow the instructions to install and configure Litefuse for Hermes Agent.

The skill will ask for your Litefuse API keys (or walk you through signing up if you don’t have an account yet), then configure everything in place. For step-by-step manual setup instead, continue below.

What gets captured

Trace structure

A multi-tool turn that also delegates to a subagent produces a trace shaped like:

Hermes Agent — Turn 7                     (AGENT, root, real turn duration)
├── plan (2 tools) #1                     (GENERATION, usage + real latency, thinking + toolCalls)
├── tool: terminal (grep) #2              (TOOL, input=args, output=result)
├── tool: read_file (index.ts) #3         (TOOL)
├── plan (1 tool) #4                      (GENERATION, usage)
├── tool (1 subagent) #5                  (TOOL — the delegate_task batch container)
│   └── subagent                          (AGENT — child container, input=task, output=child answer)
│       ├── plan (1 tool) #1              (step numbering restarts per container)
│       ├── tool: read_file (README.md) #2
│       └── subagent response             (GENERATION — child usage rolls into trace cost)
└── response                              (GENERATION — the final LLM call itself, usage + latency)

Notes on the design:

• Behaviour-based generation names. plan (n tools) #N when the model requested tool calls, response for the final answer, think #N for thinking-only steps. The model name is the generation’s model attribute — switching models never breaks your name-based filters or dashboards.
• One shared step counter. Generations and tools draw #N from the same per-container sequence, so the number in the name is agent_step_index in metadata — one numbering scheme, not two.
• Tools link to their plan via metadata, not tree depth. Each tool carries agent_plan_step (the step index of the generation that requested it) and agent_tool_call_id. The tree stays flat; the join is queryable.
• Real wall-clock timestamps. Generation spans open at pre_api_request and close with the time captured at post_api_request — the timeline shows actual LLM latency. Tool spans run pre_tool_call → post_tool_call.
• Full I/O via end-of-turn backfill. Hermes’ hook payloads carry counts, not content — so the plugin holds generation spans open and backfills input (the complete message array) and output (thinking / text / toolCall blocks) from post_llm_call’s conversation history before exporting. Each span is still sent exactly once.
• Trace header rides on every span. The trace appears in Litefuse (name / session / input / tags) as soon as the first observation completes, with no extra bootstrap observation.
• Subagent subtrees. A delegate_task call produces a tool (n subagents) #N span wrapping one subagent AGENT container per child. The gap between the tool span and the container is your real delegation overhead (process spawn, result parsing).

Quick Start

~/.hermes/hermes-agent/venv/bin/pip install 'langfuse>=4,<5'
 
# Verify
~/.hermes/hermes-agent/venv/bin/python3 -c "import langfuse; print(langfuse.__version__)"
# Expect: 4.x.y

mkdir -p ~/.hermes/plugins/litefuse
curl -fsSL https://litefuse.ai/integrations/hermes-agent/plugin.yaml \
  -o ~/.hermes/plugins/litefuse/plugin.yaml
curl -fsSL https://litefuse.ai/integrations/hermes-agent/__init__.py \
  -o ~/.hermes/plugins/litefuse/__init__.py

cat >> ~/.hermes/.env <<'EOF'
 
# Litefuse observability
LITEFUSE_PUBLIC_KEY=pk-lf-xxx
LITEFUSE_SECRET_KEY=sk-lf-xxx
LITEFUSE_BASE_URL=https://litefuse.cloud
EOF

hermes plugins enable litefuse
hermes plugins list   # confirm: status = enabled

hermes gateway restart  # only if you have `hermes gateway` running

hermes -z "use the shell tool to count the lines in ~/.hermes/config.yaml, then tell me the number"
tail -3 ~/.hermes/state/litefuse_plugin.log
# Expected: "litefuse plugin v0.2.0 registered (10 hooks, spec v1.2)"
#         + "turn closed session=YYYYMMDD_HHMMSS_xxxxxx turn=1 steps=3 api=2 tools=1 final=True"

Environment variables

Credential lookup order: process env → ~/.hermes/state/litefuse.env → ~/.hermes/litefuse.env (first existing file wins; env always takes precedence over files).

How it works

The plugin subscribes to ten Hermes plugin-hook events via ctx.register_hook(...):

Subagent subtrees. Hermes runs delegate_task children in-process on worker threads, and its hooks don’t carry a parent-session link — so the plugin uses a heuristic: a session whose first turn starts while a delegate_task tool span is open is treated as that delegation’s child. The child’s container (subagent, AGENT type) is parented under the delegate tool span via the trace context; child steps renumber from #1; the child’s generations carry their own usage, which Litefuse rolls into the parent trace’s total cost. The linkage is recorded as agent_subagent_link: "heuristic" in the container metadata.

The plugin is fail-open: any unexpected error is logged to ~/.hermes/state/litefuse_plugin.log and the callback returns silently so it never blocks Hermes’ main loop. A single Langfuse client is held in module-level state across the lifetime of the Hermes process, guarded by a thread lock so multiple concurrent gateway sessions share it safely.

Trace metadata reference

All metadata is flat, agent_-prefixed scalars — the same key set as every other Litefuse agent integration, so metadata.agent_duration_ms > 60000 filters work across Hermes, Pi, and Claude Code traces alike. Fields absent from the source data are omitted entirely (no null padding).

Trace root (AGENT observation):

• agent_turn_number, agent_session_id, agent_cwd, agent_model, agent_platform, agent_history_messages
• closed with: agent_api_calls, agent_tool_calls, agent_steps, agent_message_count, agent_duration_ms

Generations (plan / response / think):

• agent_turn_number, agent_step_index, agent_provider, agent_api (api_mode), agent_stop_reason, agent_api_duration_ms, agent_tool_call_count, agent_message_count
• agent_thinking_chars when the message carried reasoning

Tools (tool: ...):

• agent_tool_name, agent_step_index, agent_plan_step (join key: equals the requesting generation’s agent_step_index), agent_tool_call_id, agent_duration_ms, agent_is_error
• agent_output_truncated / agent_output_orig_len when truncation hit

Subagent containers (subagent):

• agent_subagent: true, agent_subagent_link: "heuristic", agent_model, agent_platform, agent_session_id (the child’s)

Troubleshooting

No traces appear in Litefuse. Tail ~/.hermes/state/litefuse_plugin.log:

tail -20 ~/.hermes/state/litefuse_plugin.log

Expected: at startup "litefuse plugin v0.2.0 registered (10 hooks, spec v1.2)", then "Litefuse client ready" and per-turn "turn closed session=... turn=N steps=N api=N tools=N final=True". If the file is empty, the plugin isn’t loading:

hermes plugins list           # confirm: litefuse → enabled
hermes plugins enable litefuse  # if not enabled
hermes gateway restart        # if gateway is running

Plugin log says Litefuse credentials not set. The plugin couldn’t find keys in the environment or fallback env files. Add them to ~/.hermes/.env:

cat >> ~/.hermes/.env <<'EOF'
LITEFUSE_PUBLIC_KEY=pk-lf-...
LITEFUSE_SECRET_KEY=sk-lf-...
LITEFUSE_BASE_URL=https://litefuse.cloud
EOF

Then re-run any hermes -z "..." or restart the gateway. (LANGFUSE_* names also work as a fallback.)

Plugin log says langfuse SDK not importable. The Langfuse SDK isn’t installed in Hermes’ venv:

~/.hermes/hermes-agent/venv/bin/pip install --upgrade 'langfuse>=4,<5'

Generations show up only when the turn ends. By design: generation content (full messages, thinking) is only available at end of turn, so generation spans export then — with their real recorded timestamps. Tool spans export live as each tool finishes, and the trace header appears with the first completed observation.

Old trace shape (api: <model> #N, user message events, metadata.hermes_agent.*). You’re running plugin v0.1.0. Re-fetch both files from https://litefuse.ai/integrations/hermes-agent/ and restart the gateway.

TUI or hermes -z oneshot doesn’t produce traces. Only affects Hermes versions older than v0.12 — upgrade Hermes, or use hermes chat -q "..." which has always loaded plugins.

Limitations

• In-flight steps are invisible until they end. Each span is sent exactly once, at its end — a long-running tool or streaming LLM call won’t show in the live view until it completes. During a delegation, child steps can appear before their parent container (“orphan” rows at the list tail); the tree is complete and correct once the turn ends.
• Subagent linking is heuristic. On a busy multi-user gateway, an unrelated session starting in the same instant as a delegation could be mis-attached (it would be tagged agent_subagent_link: "heuristic"). Single-user setups are unaffected.
• No TTFT / sampling params. Hermes hooks don’t expose first-token timing or temperature/top_p, so completion_start_time and full model_parameters (beyond max_tokens) are absent.

Resources

• Hermes Agent hooks user guide
• Hermes Agent plugin reference
• Langfuse Python SDK v4
• Litefuse Cloud
• Plugin source: plugin.yaml · __init__.py