Hermes Agent Tracing with Litefuse

Hermes Agent is Nous Research’s terminal-based AI assistant with a pluggable tool-calling and gateway architecture. This integration installs as a Hermes plugin under ~/.hermes/plugins/litefuse/. The plugin runs in-process inside Hermes’ Python interpreter and subscribes to lifecycle hook events (pre_llm_call, post_api_request, pre_tool_call, post_tool_call, post_llm_call, etc.), emitting one Litefuse trace per user turn with real wall-clock per-event timestamps.

The plugin works for every Hermes surface — CLI (hermes chat -q), Gateway (Feishu, Discord, Slack, Telegram, WhatsApp, IRC), and (with PR #24237) the TUI.

For AI — automated install

If you’re chatting with Hermes Agent right now, paste this prompt and the agent will handle the whole install end-to-end:

Read https://litefuse.ai/SKILL.md and follow the instructions to install and configure Litefuse for Hermes Agent.

The skill will ask for your Litefuse API keys (or walk you through signing up if you don’t have an account yet), then configure everything in place. For step-by-step manual setup instead, continue below.

What gets captured

Trace structure

A typical multi-tool turn produces a trace shaped like:

Hermes Agent — Turn 7                (SPAN, root, real duration)
├── user message                     (EVENT, flushes trace metadata immediately)
├── api: MiniMax-M2.7 #1             (GENERATION, usage_details, real latency)
├── tool: web_search (#1)            (TOOL, real duration)
├── api: MiniMax-M2.7 #2             (GENERATION, usage_details)
├── tool: web_extract (#2)           (TOOL, real duration)
├── api: MiniMax-M2.7 #3             (GENERATION, usage_details)
└── assistant response               (GENERATION, final answer)

Notes on the design:

• Real wall-clock timestamps. Every observation’s start/end is set by the hook callback firing moment, so the Litefuse timeline reflects actual LLM latency, tool execution time, and inter-call gaps — no synthetic grid.
• Single root parent. The container span is the unique trace root; every api / tool / response observation nests underneath. The timeline view stays clean even for 50+ step turns.
• Trace header is stable. Trace name / input / session_id flush to Litefuse at turn START (via the user message event with real wall-clock end + lf.flush()) and never get overwritten mid-stream as new observations land.
• Tool numbering. (#N) suffix on tool: observation names disambiguates repeated calls to the same tool within a turn.
• Namespaced metadata. All Hermes-specific fields live under metadata.hermes_agent.* to keep the top-level metadata dict clean for Litefuse-standard keys.

Quick Start

~/.hermes/hermes-agent/venv/bin/pip install 'langfuse>=4,<5'
 
# Verify
~/.hermes/hermes-agent/venv/bin/python3 -c "import langfuse; print(langfuse.__version__)"
# Expect: 4.x.y

mkdir -p ~/.hermes/plugins/litefuse
curl -fsSL https://litefuse.ai/integrations/hermes-agent/plugin.yaml \
  -o ~/.hermes/plugins/litefuse/plugin.yaml
curl -fsSL https://litefuse.ai/integrations/hermes-agent/__init__.py \
  -o ~/.hermes/plugins/litefuse/__init__.py

cat >> ~/.hermes/.env <<'EOF'
 
# Litefuse observability
LANGFUSE_PUBLIC_KEY=pk-lf-xxx
LANGFUSE_SECRET_KEY=sk-lf-xxx
LANGFUSE_HOST=https://litefuse.cloud
EOF

hermes plugins enable litefuse
hermes plugins list   # confirm: status = enabled

hermes gateway restart  # only if you have `hermes gateway` running

hermes chat -q "say hello in five words exactly"
tail -3 ~/.hermes/state/litefuse_plugin.log
# Expected: "turn complete session=YYYYMMDD_HHMMSS_xxxxxx turn=1 api_calls=1 tool_calls=0"

Environment variables

How it works

The plugin subscribes to nine Hermes plugin-hook events via ctx.register_hook(...):

The plugin is fail-open: any unexpected error is logged to ~/.hermes/state/litefuse_plugin.log and the callback returns silently so it never blocks Hermes’ main loop. A single Langfuse client is held in module-level state across the lifetime of the Hermes process (gateway daemon, CLI invocation, TUI subprocess), guarded by a thread lock so multiple concurrent gateway sessions share it safely.

Trace metadata reference

Trace-level metadata (under metadata.hermes_agent.*):

• turn_number — 1-based, monotonically increasing per session lifetime
• platform — cli, gateway, tui, feishu, etc. (set by Hermes)
• model — the model name at turn start
• is_first_turn, history_messages — turn position in session

Generation observation metadata (api: ..., under metadata.hermes_agent.*):

• provider, base_url, api_mode, response_model — provider config
• api_call_count, api_duration, finish_reason, message_count — per-call stats
• assistant_content_chars, assistant_tool_call_count — output shape
• step_index — the (#N) step counter

Tool observation metadata (under metadata.hermes_agent.*):

• tool_name, task_id, tool_call_id
• duration_ms, is_error
• step_index

Troubleshooting

No traces appear in Litefuse. Tail ~/.hermes/state/litefuse_plugin.log:

tail -20 ~/.hermes/state/litefuse_plugin.log

Expected: at startup "litefuse plugin registered (9 hooks)" then per-turn "Litefuse client ready" and "turn complete session=... turn=N api_calls=N tool_calls=N". If the file is empty, the plugin isn’t loading.

hermes plugins list           # confirm: litefuse → enabled
hermes plugins enable litefuse  # if not enabled
hermes gateway restart        # if gateway is running

Plugin log says Litefuse credentials not set. The plugin couldn’t find LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY in os.environ. Add them to ~/.hermes/.env:

cat >> ~/.hermes/.env <<'EOF'
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_HOST=https://litefuse.cloud
EOF

Then re-run any hermes chat -q "..." or restart the gateway.

Plugin log says langfuse SDK not importable. The Langfuse SDK isn’t installed in Hermes’ venv:

~/.hermes/hermes-agent/venv/bin/pip install --upgrade 'langfuse>=4,<5'

Trace name / session / input missing during a long turn. This was a real bug we fixed by emitting a tiny user message observation (typed as event, structured as a span so Langfuse propagates the AS_ROOT + TRACE_* attributes) and force-flushing it at turn start. If you see this, you’re probably running an older copy of __init__.py — re-fetch from https://litefuse.ai/integrations/hermes-agent/__init__.py.

TUI sessions don’t produce traces. Hermes v0.12 doesn’t load plugins in the TUI process (tui_gateway/entry.py). Either:

• Upgrade to a Hermes version with PR #24237 merged, OR
• Apply the 3-line patch locally to ~/.hermes/hermes-agent/tui_gateway/entry.py (gets clobbered on hermes update — apply again after each update).

CLI (hermes chat -q, hermes -z) and Gateway (Feishu, Slack, Discord, Telegram, etc.) work without the patch.

hermes -z "..." (oneshot mode) doesn’t produce traces either. Same root cause — oneshot’s entry point doesn’t call discover_plugins. Use hermes chat -q "..." instead (full CLI path that does load plugins), or apply the same 3-line treatment to hermes_cli/oneshot.py. We’ll file a companion PR for this.

Limitations

• No assistant reasoning capture. Hermes plugin-hook callbacks don’t expose the model’s reasoning / chain-of-thought text directly — post_api_request only gives us assistant_content_chars (the count). The thinking text exists in Hermes’ conversation history but harvesting it would require subscribing to per-message events that aren’t currently available.
• No image-block summary. If a turn involves image inputs, the plugin records them as JSON-flattened text in the trace input; it doesn’t surface media types or counts as separate metadata.

Resources

• Hermes Agent hooks user guide
• Hermes Agent plugin reference
• Langfuse Python SDK v4
• Litefuse Cloud
• Plugin source: plugin.yaml · __init__.py
• Upstream Hermes Agent PR for TUI plugin discovery: NousResearch/hermes-agent#24237