在 Azure OpenAI 和 Langchain 中使用 Litefuse 的 Trace 与 Prompt Management

本 cookbook 演示如何将 Litefuse 与 Azure OpenAI 和 Langchain 结合，进行 prompt 版本管理和评估。

设置

注意： 本指南使用我们的 Python SDK v2。我们已经推出了基于 OpenTelemetry 的全新改进版 SDK，请查看 SDK v3，体验更强大、更易用的 SDK。

%pip install --quiet "langfuse<3.0.0" langchain langchain-openai --upgrade

import os
 
# get keys for your project from https://litefuse.cloud
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-***"
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-***"
os.environ["LANGFUSE_HOST"] = "https://litefuse.cloud"
 
# your azure openai configuration
os.environ["AZURE_OPENAI_ENDPOINT"] = "your Azure OpenAI endpoint"
os.environ["AZURE_OPENAI_API_KEY"] = "your Azure OpenAI API key"
os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_VERSION"] = "2023-09-01-preview"

我们会使用 Litefuse 对 Langchain 的原生集成。更多内容请见文档。

from langfuse.callback import CallbackHandler
 
langfuse_handler = CallbackHandler()
 
# optional, verify your Litefuse credentials
langfuse_handler.auth_check()

Langchain 相关导入

from langchain_openai import AzureChatOpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.prompts.chat import (
    ChatPromptTemplate,
    SystemMessagePromptTemplate,
    HumanMessagePromptTemplate,
)
from langchain.schema import HumanMessage

简单示例

from langchain_openai import AzureChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langfuse.callback import CallbackHandler
 
langfuse_handler = CallbackHandler()
 
prompt = ChatPromptTemplate.from_template("what is the city {person} is from?")
model = AzureChatOpenAI(
    deployment_name="gpt-4o",
    model_name="gpt-4o",
)
chain = prompt | model
 
chain.invoke({"person": "Satya Nadella"}, config={"callbacks":[langfuse_handler]})

✨ 完成。打开 Litefuse Dashboard 查看本次运行的 trace。

使用 Litefuse Prompt Management 与 Langchain 的示例

关于 Litefuse Prompt Management，请参考文档。

# Initialize the Langfuse Client
from langfuse import Langfuse
langfuse = Langfuse()
 
template = """
You are an AI assistant travel assistant that provides vacation recommendations to users. 
You should also be able to provide information about the weather, local customs, and travel restrictions. 
"""
 
# Push the prompt to Litefuse and immediately promote it to production
langfuse.create_prompt(
    name="travel_consultant",
    prompt=template,
    is_active=True,
)

在生产环境中，你可以拉取该 prompt 的 production 版本。Langfuse 客户端会缓存 prompt 以提升性能，你可以通过自定义 TTL 来配置缓存行为，也可以完全关闭缓存。

# Get the prompt from Litefuse, cache it for 5 minutes
langfuse_prompt = langfuse.get_prompt("travel_consultant", cache_ttl_seconds=300)

我们这里不使用原生的 Litefuse prompt.compile()，而是直接使用 prompt.prompt，由 Langchain 来注入 prompt 变量（如果有的话）。

system_message_prompt = SystemMessagePromptTemplate.from_template(langfuse_prompt.prompt)

llm = AzureChatOpenAI(
    deployment_name="gpt-4o",
    model_name="gpt-4o",
)
 
human_message_prompt = HumanMessagePromptTemplate.from_template("{text}")
chat_prompt = ChatPromptTemplate.from_messages(
    [system_message_prompt, human_message_prompt]
)
chain = LLMChain(llm=llm, prompt=chat_prompt)
result = chain.run(
    f"Where should I go on vaction in Decemember for warm weather and beaches?",
    callbacks=[langfuse_handler],
)
 
print(result)

多次 Langchain 运行写入同一个 Litefuse trace

Langchain 设置

from langchain_openai import AzureChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from operator import itemgetter
 
prompt1 = ChatPromptTemplate.from_template(
    "What {type} is easiest to learn but hardest to master? Give a step by step approach of your thoughts, ending in your answer"
)
prompt2 = ChatPromptTemplate.from_template(
    "How {type} can be learned in 21 days? respond in {language}"
)
model = AzureChatOpenAI(
    deployment_name="gpt-4o",
    model_name="gpt-4o",
)
chain1 = prompt1 | model | StrOutputParser()
chain2 = (
    {"type": chain1, "language": itemgetter("language")}
    | prompt2
    | model
    | StrOutputParser()
)

在同一个 Litefuse trace 内多次运行 chain。

# Create trace using Langfuse Client
langfuse = Langfuse()
trace = langfuse.trace(name="chain_of_thought_example", user_id="user-1234")
 
# Create a handler scoped to this trace
langfuse_handler = trace.get_langchain_handler()
 
# First run
chain2.invoke(
    {"type": "business", "language": "german"}, config={"callbacks": [langfuse_handler]}
)
 
# Second run
chain2.invoke(
    {"type": "business", "language": "english"}, config={"callbacks": [langfuse_handler]}
)

添加 score

在 Litefuse 中评估 LLM 应用的 trace 时，需要给 trace 添加 score。为了演示方便，这里使用一个 mock 的 score。更复杂的 score 类型请参考文档。

获取 trace_id。这里用前面通过 langfuse.trace() 创建的 trace。你也可以通过 langfuse_handler.get_trace_id() 获取 trace_id。

trace_id = trace.id

# Add score to the trace via the Langfuse Python Client
langfuse = Langfuse()
 
trace = langfuse.score(
    trace_id=trace_id,
    name="user-explicit-feedback",
    value=1,
    comment="I like how personalized the response is",
)

Anthropic (Python)Databricks

这个页面对你有帮助吗？

支持