OpenRouter 集成

本指南介绍如何把 Litefuse 与 OpenRouter 集成。

什么是 OpenRouter？ OpenRouter 提供 OpenAI 兼容的 completion API，可以访问 280+ 语言模型与 provider，可直接调用，也可通过 OpenAI SDK 调用。这让开发者通过单一统一接口就能访问各种 LLM。

什么是 Litefuse？ Litefuse 是开源的 AI Agent 可观测性和评估平台，帮助团队追踪 LLM 调用、监控性能并调试 AI 应用中的问题。

将 Litefuse 与 OpenRouter 集成有两种方式：

使用 OpenRouter Broadcast 功能，无需任何代码改动即可自动将 trace 发送到 Litefuse。
使用 Litefuse 的 OpenAI SDK 包装器 手动把 trace 发送到 Litefuse。由于 OpenRouter 使用 OpenAI API schema，可以利用 Litefuse 与 OpenAI SDK 的原生集成，Python 与 TypeScript 都可用。

下面的指南还包含了成本追踪、与 Litefuse Python SDK 互通以及后续步骤等更多信息。

1. OpenRouter Broadcast Tracing 与 Litefuse（低代码）

OpenRouter 的 Broadcast 功能可以无代码地把 trace 自动发送到 Litefuse。要启用这个集成，进入你的 OpenRouter 设置并接入你的 Litefuse API Key。配置完成后，所有经 OpenRouter 处理的请求都会被追踪并出现在你的 Litefuse 项目中。

OpenRouter settings showing the Litefuse Broadcast configuration

注意： 如果你希望以最少配置把所有 OpenRouter 请求记录到 Litefuse，又不需要自定义 trace metadata 或嵌套 tracing 等高级特性，推荐使用此方法。

如何配置 Litefuse API Key 或可用选项的更多信息，见 Litefuse 入门指南。

2. SDK 集成

如果需要嵌套 tracing、自定义 metadata 以及对 trace 数据的完整控制等高级特性，请使用 Litefuse 的 OpenAI SDK 包装器。本指南后续部分介绍 SDK 集成方法。

通过 SDK 集成快速开始

pip install langfuse openai

import os
 
# Set your Litefuse API keys
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_BASE_URL="https://litefuse.cloud"
# LANGFUSE_BASE_URL="https://litefuse.cloud"
 
# Set your OpenRouter API key (OpenRouter uses the 'OPENAI_API_KEY' environment variable)
os.environ["OPENAI_API_KEY"] = "<YOUR_OPENROUTER_API_KEY>"

示例 1：简单的 LLM 调用

由于 OpenRouter 提供 OpenAI 兼容 API，我们可以使用 Litefuse 的 OpenAI SDK 包装器自动把 OpenRouter 调用记录为 Litefuse 中的 generation。

base_url 设置为 OpenRouter 的 API 端点。
你可以把 "qwen/qwen-plus" 替换为 OpenRouter 上的任意模型。
default_headers 可包含 OpenRouter 文档列出的可选 header。
extra_body={"usage": {"include": True}} 让 OpenRouter 在响应中返回成本。

# Import the Langfuse OpenAI SDK wrapper
from langfuse.openai import openai
 
# Create an OpenAI client with OpenRouter's base URL
client = openai.OpenAI(
    base_url="https://openrouter.ai/api/v1",
    default_headers={
        "HTTP-Referer": "<YOUR_SITE_URL>",  # Optional: Your site URL
        "X-Title": "<YOUR_SITE_NAME>",      # Optional: Your site name
    }
)
 
# Make a chat completion request
response = client.chat.completions.create(
    model="qwen/qwen-plus",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a fun fact about space."}
    ],
    extra_body={"usage": {"include": True}},
    name="fun-fact-request"  # Optional: Name of the generation in Litefuse
)
 
# Print the assistant's reply
print(response.choices[0].message.content)

示例 2：嵌套 LLM 调用

通过 @observe() 装饰器，我们可以捕获任意 Python 函数的执行细节，包括嵌套 LLM 调用、输入、输出和执行时间。这样能在极少代码改动下获得深入的可观测性。

@observe() 装饰器捕获函数的输入、输出和执行细节。
嵌套函数 summarize_text 与 analyze_sentiment 也被装饰，从而构成 trace 的层级结构。
函数中的每次 LLM 调用都会被记录，提供详细的执行流程 trace。

from langfuse import observe
from langfuse.openai import openai
 
# Create an OpenAI client with OpenRouter's base URL
client = openai.OpenAI(
    base_url="https://openrouter.ai/api/v1",
)
 
@observe()  # This decorator enables tracing of the function
def analyze_text(text: str):
    # First LLM call: Summarize the text
    summary_response = summarize_text(text)
    summary = summary_response.choices[0].message.content
 
    # Second LLM call: Analyze the sentiment of the summary
    sentiment_response = analyze_sentiment(summary)
    sentiment = sentiment_response.choices[0].message.content
 
    return {
        "summary": summary,
        "sentiment": sentiment
    }
 
@observe()  # Nested function to be traced
def summarize_text(text: str):
    return client.chat.completions.create(
        model="openai/gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You summarize texts in a concise manner."},
            {"role": "user", "content": f"Summarize the following text:\n{text}"}
        ],
        extra_body={"usage": {"include": True}},
        name="summarize-text"
    )
 
@observe()  # Nested function to be traced
def analyze_sentiment(summary: str):
    return client.chat.completions.create(
        model="openai/gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You analyze the sentiment of texts."},
            {"role": "user", "content": f"Analyze the sentiment of the following summary:\n{summary}"}
        ],
        extra_body={"usage": {"include": True}},
        name="analyze-sentiment"
    )
 
# Example usage
text_to_analyze = "OpenAI's GPT-4 model has significantly advanced the field of AI, setting new standards for language generation."
analyze_text(text_to_analyze)

Example trace in Litefuse

Litefuse 中示例 trace 的公开链接

成本追踪

Litefuse 的 token 与成本追踪可以直接采用 OpenRouter 返回的成本，而不再自行计算。这能提升成本追踪的准确度，对小众模型尤为有用。

要启用这个能力，需要：

像上面那样使用 Litefuse 的 OpenAI 集成
在 OpenRouter 中启用 Usage Accounting，让 OpenRouter 在每次调用中返回成本

Interoperability with the Python SDK

You can use this integration together with the Litefuse SDKs to add additional attributes to the observation.

The @observe() decorator provides a convenient way to automatically wrap your instrumented code and add additional attributes to the observation.

from langfuse import observe, propagate_attributes, get_client
 
langfuse = get_client()
 
@observe()
def my_llm_pipeline(input):
    # Add additional attributes (user_id, session_id, metadata, version, tags) to all spans created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-observation"],
        metadata={"email": "user@litefuse.ai"},
        version="1.0.0"
    ):
 
        # YOUR APPLICATION CODE HERE
        result = call_llm(input)
 
        return result
 
# Run the function
my_llm_pipeline("Hi")

Learn more about using the Decorator in the Langfuse SDK instrumentation docs.

Troubleshooting

No observations appearing

First, enable debug mode in the Python SDK:

export LANGFUSE_DEBUG="True"

Then run your application and check the debug logs:

OTel observations appear in the logs: Your application is instrumented correctly but observations are not reaching Litefuse. To resolve this:
1. Call langfuse.flush() at the end of your application to ensure all observations are exported.
2. Verify that you are using the correct API keys and base URL.
No OTel spans in the logs: Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.

Unwanted observations in Litefuse

The Langfuse SDK is based on OpenTelemetry. Other libraries in your application may emit OTel spans that are not relevant to you. These still count toward your billable units, so you should filter them out. See Unwanted spans in Litefuse for details.

Missing attributes

Some attributes may be stored in the metadata object of the observation rather than being mapped to the Litefuse data model. If a mapping or integration does not work as expected, please raise an issue on GitHub.

Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application:

Manage prompts in Litefuse Add evaluation scores Run LLM-as-a-judge Evaluators Create datasets Create custom dashboards Test queries in the Playground

LiteLLM Proxy Portkey

这个页面对你有帮助吗？

支持