Data Ingestion

Ingesting data means importing large data files from multiple sources into a single, cloud-based storage medium — a data warehouse, data mart, or database — from which it can be accessed and analyzed by TruEra.

First things first

Remember to first create a project before attempting to ingest data. Review the Quickstart for an brief overview of the ingestion process using sample data.

To ingest project data, TruEra supports the following methods:

• Reading data stored in local files
• Reading data directly from data stores

In terms of task breakout, these comprise, at minimum:

• Pre-deployment
• Feature Development – transforming raw data into features that better represent the underlying problem, resulting in improved model accuracy on unseen data.
• Model Training – fitting the best combination of weights and bias to minimize loss functions over the prediction range.
• Post-deployment
• Logging Inputs and Predictions – classifying whether a particular log event, or set of events, is causing a real incident that requires attention.
• Logging Additional Metrics – score tracking to determine real accuracy and improvement —
  F1

  Measures a model's accuracy by combining its precision and recall scores; computes how many times a model made a correct prediction across the entire dataset.
  ,
  F2

  Weighted harmonic mean of the precision and recall (given a threshold value). Unlike the F1 score, which gives equal weight to precision and recall, the F2 score gives more weight to recall than to precision
  ,
  brier_loss

  Measures the mean squared difference between the predicted probability and the actual outcome.
  ; iteration-level metrics (learning curves); predictions after every epoch; and updated experiment metrics among many others.

What is Feature Development or Feature Engineering?✕

Feature development/engineering is the process of selecting, transforming, extracting, combining, and manipulating raw data to generate the desired variables (features) for improved analysis and predictive modeling — a crucial step in developing an accurate machine learning model.

The five steps described next should help to provide insights for your feature engineering decisions.

Step 1. Data Cleansing

The process of dealing with errors or inconsistencies, data cleansing involves identifying incorrect, missing, duplicated, and/or irrelevant data, then adding, deleting, replacing, or modifying it to remove outliers and incorrect values.

Cleansing is different from transformation (Step 2) in that cleaning removes data that doesn't belong in your dataset, whereas transformation converts the data's format or structure into something more model-usable.

Step 2. Data Transformation

This is the process of transforming the data from one layout to another without changing the meaning of the original data. Transformation processes are also referred to as data wrangling, or data munging, transforming and mapping data from one "raw" data form into another.

The three most effective techniques are:

• Normalize – by applying a mathematical function to every data point to handle highly skewed data.

  

• Standardize – by converting the data to a uniform format as a way of handling data with differing units (e.g., converting US Custom and British Imperial measurements to metric).

  

• Encode – by converting categorical variables into numerical values so that they can be more easily fitted to the model.

  

  ML models require input and output variables to be numeric. Even so, the more simplified the encoding the better; i.e., encoding by shared/common characteristic(s) where feasible and meaningful.

Step 3. Feature Extraction

This is the process of extracting new features from the existing attributes to reduce the number of features in the model.

Feature extraction doesn't have to be complicated; it could simply entail grouping multiple variables into a feature that measures the average of those variables.

Step 4. Feature Selection

This involves selecting the correct subset of features to ensure capturing the closest relationship to the target variable; consists of eliminating features that do not explain the behavior of the target variable. Generally, this is done by ranking features based on a statistical test to identify the most important features.

Note: The third panel (Key important features) under the Focus tab will help you to rank feature importance. You can also use a correlation matrix, eliminating predictor variables that correlate highly with other predictor variables.

Step 5. Feature Iteration

While several techniques and methodologies can work to iterate feature optimization, they all follow a similar framework to the one illustrated below — adding or removing only those features that improve model performance.

Hence, although there is no single correct way of optimizing features for maximizing model performance, the five steps discussed above should surface the necessary decision points leading to that objective; i.e., improved model performance.

Click the ✕ at the top of this popup to close it and continue.

Above all, have a plan for tracking and handling your model's results, both expected results and unexpected, so you can refine and improve your data and model all along your path to ultimate success.

Click Next below to continue.