Skip to content

Integrating Your App

TruEra makes it easy to integrate your app with start streaming app traces. The trulens-eval open source project provides the key primitives for instrumentation and evaluation.

Before we dive deeper into integrations, let's discuss some key conceptual elements:

LLM Application

An LLM application combines one or more of objects like language models, retrievers, data APIs, vector databases to provide rich text capabilities like question answering, summarization, text extraction. Often LLM applications are built using frameworks like Langchain and LlamaIndex, or from scratch in a programming language of your choice.

Instrumentation

Instrumentation is a step where your LLM app and its intermediate steps are recorded in order to provide evaluations and observability. The TruLens package provides native instrumentation for Langchain and LlamaIndex and allows you instrument any custom python class with an @instrument python decorator.

Application Trace

An application trace is the output of running an instrumented app. This is a record of all the inputs and outputs of instrumented steps within your LLM application. For example, for a retrieval augmented generation (RAG) app, these can be the inputs, outputs, and retrieval context.

Feedback and Feedback Function

A piece of feedback on an application trace is a score evaluating the quality of the trace along a specific dimension. This is often either provided by a human judge or an automated evaluation that we call a feedback function. TruLens provides a large variety of feedback functions, and you can also provide your own.

Data Configuration

Depending on the part of the app development lifecycle you're in: experimentation or production. The data can be streamed in two different configurations,

  • Experimental Configuration: This config is useful for when you are building and iterating on your applications and wish to ingest a few traces (records) that are responses to manually triggered queries. For a given App, all traces ingested are appended to the same experiments dataset.

  • Production Configuration: This config is useful for when you have deployed your App in production and want to monitor the application over time by configuring observability dashboards. For a given App, all traces ingested are appended to the same production dataset.

Steps for ingestion

  1. Create a project
  2. Build your LLM application that is compatible with trulens-eval
  3. Define your feedback functions using trulens-eval
  4. Wrap your LLM application with a TruEra wrapper with the appropriate configuration (Experimental/Production)
  5. Send queried to your LLM application

This quickstart walks you through an example of how to integrate your app to TruEra.

Click Next below to continue.