Skip to content

Experiment Tracker

As you are building you LLM Application, multiple experiments are run which end up in various version of the application. Experiment tracker allows you to track all your versions along with aggregate evaluations results of the feedback functions defined. Read more about feedback functions here.

Experiment Tracker

Once you enter your project, click on Experiments on the left panel to look at all the applications for which you have ingested traces in experiment config.

LLM Experiment Tracker
click and hold to enlarge

You should see a list of all the app versions ingested in the project along with their evaluation results. You can click on the end of the row of any application to dive deeper and look at all the traces ingested for a particular app version.

Traces List

In this page, you can see meta-data and feedback evaluations per trace (record) for the chosen application in this page. This will help you analyze an app's performance across it's traces and find queries on which the app does not perform as per expectations. To deep dive into a particular trace, click on the end of the row of a particular trace.

Traces Table
click and hold to enlarge

Trace Page

This page shows in-depth details for this particular trace.

  1. User input: The input that the app received
  2. Response: The final response of the app
  3. Prompt: The prompt filled with variables to send to the LLM
  4. Evaluations: Evaluation for each defined feedback function along with the reason when applicable

This page lets you see the breakdown of important steps (spans) in the response generated by your LLM application. You can analyze how each portion of your RAG is performing and take corrective actions as needed.

Trace
click and hold to enlarge

Click Next below to continue.