在 Litefuse 中 Trace BeeAI Agent

本 notebook 演示如何使用 OpenTelemetry instrumentation，结合 Litefuse 对 BeeAI 框架应用进行 trace 与观测。

什么是 BeeAI？ BeeAI 框架是由 IBM Research 开发的全面工具集，用于构建智能、自主的 Agent 与多 Agent 系统。它提供 Python 和 TypeScript 中创建可推理、可执行操作并能协作解决复杂问题的 Agent 所需的一切。

什么是 Litefuse？ Litefuse 是一个用于 LLM 可观测性与监控的开源平台。它通过捕获元数据、prompt 详情、token 使用、延迟等信息，帮助你 trace 和监控 AI 应用。

第 1 步：安装依赖

开始前，请在你的 Python 环境中安装所需依赖：

%pip install beeai-framework langfuse openinference-instrumentation-beeai "beeai-framework[wikipedia]"

第 2 步：配置 Langfuse SDK

接下来设置你的 Litefuse API Key。你可以通过注册免费的 Litefuse Cloud 账号或自托管 Litefuse 获取这些 Key。这些环境变量是 Langfuse 客户端鉴权并向你的 Litefuse 项目发送数据所必需的。

你还需要配置 LLM 提供方的凭证。BeeAI 框架支持多种提供方，包括 OpenAI、Ollama、watsonx.ai 等。

import os
 
# Get keys for your project from the project settings page: https://litefuse.cloud
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..." 
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..." 
os.environ["LANGFUSE_BASE_URL"] = "https://litefuse.cloud"
 
# Your LLM provider API key (example with OpenAI, adjust for your provider)
os.environ["OPENAI_API_KEY"] = "sk-proj-..."
# For Ollama (local), no API key needed
# For other providers, set appropriate environment variables

设置好环境变量后，我们就可以初始化 Langfuse 客户端了。get_client() 会使用环境变量中提供的凭证来初始化 Langfuse 客户端。

from langfuse import get_client
 
# Initialise Langfuse client and verify connectivity
langfuse = get_client()
assert langfuse.auth_check(), "Langfuse auth failed - check your keys ✋"

第 3 步：OpenTelemetry Instrumentation

使用 BeeAIInstrumentor 库来包装 BeeAI 框架的调用，并将 OpenTelemetry span 发送到 Litefuse。

from openinference.instrumentation.beeai import BeeAIInstrumentor
 
BeeAIInstrumentor().instrument()

第 4 步：运行示例

import asyncio
from beeai_framework.agents.react import ReActAgent
from beeai_framework.agents.types import AgentExecutionConfig
from beeai_framework.backend.chat import ChatModel
from beeai_framework.backend.types import ChatModelParameters
from beeai_framework.memory import TokenMemory
from beeai_framework.tools.search.wikipedia import WikipediaTool
from beeai_framework.tools.weather.openmeteo import OpenMeteoTool
 
# Initialize the language model
llm = ChatModel.from_name(
    "openai:gpt-4o-mini",  # or "ollama:granite3.3:8b" for local Ollama
    ChatModelParameters(temperature=0.7),
)
 
# Create tools for the agent
tools = [
    WikipediaTool(),
    OpenMeteoTool(),
]
 
# Create a ReAct agent with memory
agent = ReActAgent(
    llm=llm, 
    tools=tools, 
    memory=TokenMemory(llm)
)
 
# Run the agent
async def main():
    response = await agent.run(
        prompt="I'm planning a trip to Barcelona, Spain. Can you research key attractions and landmarks I should visit, and also tell me what the current weather conditions are like there?",
        execution=AgentExecutionConfig(
            max_retries_per_step=3, 
            total_max_retries=10, 
            max_iterations=5
        ),
    )
    print("Agent Response:", response.result.text)
    return response
 
# Run the example
response = await main()

在 Litefuse 中查看 Trace

执行应用后，进入你的 Litefuse Trace 表。你将看到应用执行的详细 trace，包括 Agent 对话、工具调用、LLM 交互、输入、输出以及性能指标的洞察。

Litefuse Trace

在 Litefuse 中查看 trace

Interoperability with the Python SDK

You can use this integration together with the Litefuse SDKs to add additional attributes to the observation.

The @observe() decorator provides a convenient way to automatically wrap your instrumented code and add additional attributes to the observation.

from langfuse import observe, propagate_attributes, get_client
 
langfuse = get_client()
 
@observe()
def my_llm_pipeline(input):
    # Add additional attributes (user_id, session_id, metadata, version, tags) to all spans created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-observation"],
        metadata={"email": "user@litefuse.ai"},
        version="1.0.0"
    ):
 
        # YOUR APPLICATION CODE HERE
        result = call_llm(input)
 
        return result
 
# Run the function
my_llm_pipeline("Hi")

Learn more about using the Decorator in the Langfuse SDK instrumentation docs.

Troubleshooting

No observations appearing

First, enable debug mode in the Python SDK:

export LANGFUSE_DEBUG="True"

Then run your application and check the debug logs:

OTel observations appear in the logs: Your application is instrumented correctly but observations are not reaching Litefuse. To resolve this:
1. Call langfuse.flush() at the end of your application to ensure all observations are exported.
2. Verify that you are using the correct API keys and base URL.
No OTel spans in the logs: Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.

Unwanted observations in Litefuse

The Langfuse SDK is based on OpenTelemetry. Other libraries in your application may emit OTel spans that are not relevant to you. These still count toward your billable units, so you should filter them out. See Unwanted spans in Litefuse for details.

Missing attributes

Some attributes may be stored in the metadata object of the observation rather than being mapped to the Litefuse data model. If a mapping or integration does not work as expected, please raise an issue on GitHub.

Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application:

Manage prompts in Litefuse Add evaluation scores Run LLM-as-a-judge Evaluators Create datasets Create custom dashboards Test queries in the Playground

AutoGen Claude Agent SDK

这个页面对你有帮助吗？

支持