Machine Learning Index Stage

Table of Contents

Configuration

The Fusion machine learning indexing stage uses a trained machine learning model to analyze a field or fields of a PipelineDocument and stores the results of analysis in a new field of either the PipelineDocument or Context object.

In order to use the Machine Learning Stage, you must train a machine learning model. There are two different ways to train a model:

Use a Fusion AI job that trains a model, like Logistic Regression or Random Forest.
Train a model using Spark’s MLlib API outside of Fusion, and upload this model into Fusion’s blob store. Complete details are available in Machine Learning Models in Fusion. TIP: When specifying field names, multiple field names are supported, in this format: field1:weight,field2:weight,field3:weight

Configuration

Invokes a machine learning model to make a prediction on a document during indexing.

skip - boolean

Set to true to skip this stage.

Default: false

label - string

A unique label for this stage.

<= 255 characters

condition - string

Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.

modelId - stringrequired

The ID of the ML model stored in the Fusion blob store.

>= 1 characters

docFeatureFieldName - string

Name of the field to extract document features from (model input).

>= 1 characters

Default: body_t

predictionFieldName - stringrequired

Name of the field to store the prediction (model output) in the document.

defaultValue - string

Value to provide if a prediction cannot be made for a document.

failOnError - boolean

Flag to indicate if this stage should throw an exception if an error occurs while generating a prediction for a document.

Default: false

storeInContext - boolean

Flag to indicate that the prediction should be set as a context property instead of setting a field on the document.

Default: false

includeRawPredictions - boolean

Flag to indicate that raw predictions and scores, in addition to the best prediction, should be set on the document.

Default: false