用 Litefuse 为 Portkey LLM Gateway 提供可观测性
本指南介绍如何把 Portkey 的 AI 网关与 Litefuse 集成。Portkey 的 API 端点完全 兼容 OpenAI SDK，可以无缝地追踪和监控你的 AI 应用。
什么是 Portkey？ Portkey 是一个 AI 网关，提供统一接口访问 250+ AI 模型，并为生成式 AI 应用提供高级的控制、可见性与安全工具。
什么是 Litefuse？ Litefuse 是开源的 AI Agent 可观测性和评估平台，帮助团队追踪 LLM 调用、监控性能并调试 AI 应用中的问题。
第 1 步：安装依赖
%pip install openai langfuse portkey_ai第 2 步：配置环境变量
接下来配置你的 Litefuse API Key。可以通过免费注册 Litefuse Cloud 或 自托管 Litefuse 获取。这些环境变量是 Langfuse 客户端鉴权和把数据发送到你的 Litefuse 项目所必需的。
import os
# Get keys for your project from the project settings page: https://litefuse.cloud
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_BASE_URL"] = "https://litefuse.cloud"from langfuse import get_client
get_client().auth_check()True
第 3 步：使用 Litefuse 的 OpenAI Drop-in 替代品
接下来，你可以使用 Litefuse 的 OpenAI 兼容客户端（from langfuse.openai import OpenAI）来追踪所有经 Portkey 网关发送的请求。LLM 网关与虚拟 LLM key 的详细配置说明，请参考 Portkey 文档。
from langfuse.openai import OpenAI
from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL
client = OpenAI(
api_key="xxx", #Since we are using a virtual key we do not need this
base_url = PORTKEY_GATEWAY_URL,
default_headers = createHeaders(
api_key = "***",
virtual_key = "***"
)
)第 4 步：运行一个示例
response = client.chat.completions.create(
model="gpt-4o", # Or any model supported by your chosen provider
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What are the benefits of using an AI gateway?"},
],
)
print(response.choices[0].message.content)
# Flush via global client
langfuse = get_client()
langfuse.flush()第 5 步：在 Litefuse 中查看 trace
运行示例后，登录 Litefuse 即可查看详细 trace，包括：
- 请求参数
- 响应内容
- token 用量与延迟指标
- 通过 Portkey 网关的 LLM 模型信息
Interoperability with the Python SDK
You can use this integration together with the Litefuse SDKs to add additional attributes to the observation.
The @observe() decorator provides a convenient way to automatically wrap your instrumented code and add additional attributes to the observation.
from langfuse import observe, propagate_attributes, get_client
langfuse = get_client()
@observe()
def my_llm_pipeline(input):
# Add additional attributes (user_id, session_id, metadata, version, tags) to all spans created within this execution scope
with propagate_attributes(
user_id="user_123",
session_id="session_abc",
tags=["agent", "my-observation"],
metadata={"email": "user@litefuse.ai"},
version="1.0.0"
):
# YOUR APPLICATION CODE HERE
result = call_llm(input)
return result
# Run the function
my_llm_pipeline("Hi")Learn more about using the Decorator in the Langfuse SDK instrumentation docs.
Troubleshooting
No observations appearing
First, enable debug mode in the Python SDK:
export LANGFUSE_DEBUG="True"Then run your application and check the debug logs:
- OTel observations appear in the logs: Your application is instrumented correctly but observations are not reaching Litefuse. To resolve this:
- Call
langfuse.flush()at the end of your application to ensure all observations are exported. - Verify that you are using the correct API keys and base URL.
- Call
- No OTel spans in the logs: Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.
Unwanted observations in Litefuse
The Langfuse SDK is based on OpenTelemetry. Other libraries in your application may emit OTel spans that are not relevant to you. These still count toward your billable units, so you should filter them out. See Unwanted spans in Litefuse for details.
Missing attributes
Some attributes may be stored in the metadata object of the observation rather than being mapped to the Litefuse data model. If a mapping or integration does not work as expected, please raise an issue on GitHub.
Next Steps
Once you have instrumented your code, you can manage, evaluate and debug your application: