Data Ingestion

Pre-production data is always ingested in batches. Production data is ingested in batches or in real-time, depending on whether your model makes batch or real-time inferences.

First things first

Remember to first create a project before attempting to ingest data. Review the Diagnostics Quickstart or Monitoring Quickstart for an brief overview of the ingestion process using sample data.

Batch Ingestion

A single batch of data is ingested via the add_data() SDK function for pre-production data or add_production_data() for production data.

These functions supports ingesting data batches from the following data sources:

• Pandas dataframes
• Local files
• Remote files in object storage

Real-time ingestion

Production data can be streamed to TruEra in real time using the ingest_events() SDK function. TruEra can also ingest data in near-real time from real-time single model Amazon SageMaker models with Data Capture enabled.

Click Next below to continue.