Guides
The guides section is work-in-progress. We want to collect all end-to-end examples of how to use Litefuse here including cookbooks, videos and other external resources.
Introduction to Litefuse
If you are new to Litefuse, watch these walkthroughs to get a thorough introduction to the Litefuse platform.
Introduction to Langfuse
Get an overview of the complete Langfuse platform and learn how it helps teams build better LLM applications through observability, prompt management, and evaluation.
Evaluation Tutorials
Start with the fundamentals and progress through advanced techniques as your application complexity grows. Read our blog post for an overview.
Error Analysis
Review traces to classify issues like hallucinations, irrelevance, and formatting errors. Turn raw logs into actionable insights.
Automated Evaluations
Automated evaluators give you a scalable way to measure and monitor your application’s failure modes.
Testing LLM Applications
Build a testing foundation with deterministic checks and probabilistic ones using LLM judges to prevent regressions.
Synthetic Dataset Generation
Scale your test coverage with LLM-generated synthetic data. Create diverse inputs and adversarial cases without waiting for users.
Experiment Interpretation
Run experiments to compare prompts, models, or pipelines. Measure metrics like accuracy and speed to quantify progress.
RAG Observability & Evals
Specialized metrics for retrieval-augmented generation pipelines including retrieval relevance, answer faithfulness, and context completeness.
Advanced Topics
Extend your evaluation approach for complex, multi-step applications.
Evaluating Multi-Turn Conversations
Evaluate coherence, memory, and resolution in conversational apps that maintain context across multiple turns.
Simulated Multi-Turn Conversations
Generate and test user-AI exchanges safely before production. Build realistic dialogue scenarios for robust evaluation.
Agent Evaluation
Assess end-to-end trajectories for agents with tool use and planning. Evaluate action selection and task completion.
Vibe Coding a Custom Annotation UI
Build a custom annotation UI in minutes using Litefuse as data source
Automatic Prompt Improvement with AI Agent Skills
Use the Litefuse skill to analyze trace feedback and iteratively improve your prompts.
Cookbooks
You can find the ipynb files in the cookbook directory of the repository.
Classification
Evaluation
Examples
Integrations
Prompt Management
Security
Other
Video Tutorials
