Anannas AI 集成

本指南介绍如何把 Litefuse 与 Anannas AI 集成。

什么是 Anannas AI？ Anannas AI 是一个统一的推理网关，通过 OpenAI 兼容 API 提供对 500+ 模型（OpenAI、Anthropic、Mistral、Gemini、DeepSeek 等）的访问。它内置可观测性，提供缓存命中率分析、token 级指标、工具调用分析和模型效率打分；提供 provider 健康监测与自动故障转移路由；约 0.48ms 开销加 5% 加价；并为企业部署支持 BYOK（Bring Your Own Key）。

什么是 Litefuse？ Litefuse 是开源的 AI Agent 可观测性和评估平台，帮助团队追踪 LLM 调用、监控性能并调试 AI 应用中的问题。

由于 Anannas AI 使用 OpenAI API schema，我们可以利用 Litefuse 与 OpenAI SDK 的原生集成，Python 与 TypeScript 都可用。

快速开始

pip install langfuse openai

import os
 
# Set your Litefuse API keys
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_BASE_URL="https://litefuse.cloud"
 
# Set your Anannas API key
os.environ["ANANNAS_API_KEY"] = "<YOUR_ANANNAS_API_KEY>"

示例 1：简单的 LLM 调用

由于 Anannas AI 提供 OpenAI 兼容 API，我们可以使用 Litefuse 的 OpenAI SDK 包装器自动把 Anannas AI 调用记录为 Litefuse 中的 generation。

base_url 设置为 Anannas AI 的 API 端点。
你可以把 "anthropic/claude-3-5-sonnet" 替换成 Anannas AI 上 500+ 模型中的任意一个。
API Key 从 ANANNAS_API_KEY 环境变量读取。

# Import the Langfuse OpenAI SDK wrapper
from langfuse.openai import openai
 
# Create an OpenAI client with Anannas AI's base URL
client = openai.OpenAI(
    api_key=os.environ["ANANNAS_API_KEY"],
    base_url="https://api.anannas.ai/v1"
)
 
# Make a chat completion request
response = client.chat.completions.create(
    model="anthropic/claude-3-5-sonnet",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a fun fact about pineapples."}
    ],
    name="fun-fact-request"  # Optional: Name of the generation in Litefuse
)
 
# Print the assistant's reply
print(response.choices[0].message.content)

示例 2：嵌套 LLM 调用

通过 @observe() 装饰器，我们可以捕获任意 Python 函数的执行细节，包括嵌套 LLM 调用、输入、输出和执行时间。这样能在极少代码改动下获得深入的可观测性。

@observe() 装饰器捕获函数的输入、输出与执行细节。
嵌套函数 summarize_text 和 analyze_sentiment 也被装饰，从而构成 trace 的层级结构。
函数中每次 LLM 调用都会被记录，提供详细的执行流程 trace。

from langfuse import observe
from langfuse.openai import openai
 
# Create an OpenAI client with Anannas AI's base URL
client = openai.OpenAI(
    api_key=os.environ["ANANNAS_API_KEY"],
    base_url="https://api.anannas.ai/v1"
)
 
@observe()  # This decorator enables tracing of the function
def analyze_text(text: str):
    # First LLM call: Summarize the text
    summary_response = summarize_text(text)
    summary = summary_response.choices[0].message.content
 
    # Second LLM call: Analyze the sentiment of the summary
    sentiment_response = analyze_sentiment(summary)
    sentiment = sentiment_response.choices[0].message.content
 
    return {
        "summary": summary,
        "sentiment": sentiment
    }
 
@observe()  # Nested function to be traced
def summarize_text(text: str):
    return client.chat.completions.create(
        model="openai/gpt-4o",
        messages=[
            {"role": "system", "content": "You summarize texts in a concise manner."},
            {"role": "user", "content": f"Summarize the following text:\n{text}"}
        ],
        name="summarize-text"
    )
 
@observe()  # Nested function to be traced
def analyze_sentiment(summary: str):
    return client.chat.completions.create(
        model="openai/gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You analyze the sentiment of texts."},
            {"role": "user", "content": f"Analyze the sentiment of the following summary:\n{summary}"}
        ],
        name="analyze-sentiment"
    )
 
# Example usage
text_to_analyze = "Anannas AI provides a unified gateway to access hundreds of LLM models with built-in observability and automatic fallback routing."
analyze_text(text_to_analyze)

Example trace in Litefuse

在 Litefuse 中展示的、通过 Anannas AI 网关进行嵌套 LLM 调用的 trace 示例

Interoperability with the Python SDK

You can use this integration together with the Litefuse SDKs to add additional attributes to the observation.

The @observe() decorator provides a convenient way to automatically wrap your instrumented code and add additional attributes to the observation.

from langfuse import observe, propagate_attributes, get_client
 
langfuse = get_client()
 
@observe()
def my_llm_pipeline(input):
    # Add additional attributes (user_id, session_id, metadata, version, tags) to all spans created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-observation"],
        metadata={"email": "user@litefuse.ai"},
        version="1.0.0"
    ):
 
        # YOUR APPLICATION CODE HERE
        result = call_llm(input)
 
        return result
 
# Run the function
my_llm_pipeline("Hi")

Learn more about using the Decorator in the Langfuse SDK instrumentation docs.

Troubleshooting

No observations appearing

First, enable debug mode in the Python SDK:

export LANGFUSE_DEBUG="True"

Then run your application and check the debug logs:

OTel observations appear in the logs: Your application is instrumented correctly but observations are not reaching Litefuse. To resolve this:
1. Call langfuse.flush() at the end of your application to ensure all observations are exported.
2. Verify that you are using the correct API keys and base URL.
No OTel spans in the logs: Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.

Unwanted observations in Litefuse

The Langfuse SDK is based on OpenTelemetry. Other libraries in your application may emit OTel spans that are not relevant to you. These still count toward your billable units, so you should filter them out. See Unwanted spans in Litefuse for details.

Missing attributes

Some attributes may be stored in the metadata object of the observation rather than being mapped to the Litefuse data model. If a mapping or integration does not work as expected, please raise an issue on GitHub.

Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application:

Manage prompts in Litefuse Add evaluation scores Run LLM-as-a-judge Evaluators Create datasets Create custom dashboards Test queries in the Playground

xAI Grok Helicone

这个页面对你有帮助吗？

支持