Machine Learning Jobs

Table of Contents

Fusion provides these job types to perform machine learning tasks.

Signals analysis

These jobs analyze a collection of signals in order to perform query rewriting, signals aggregation, or experiment analysis.

Ground Truth

Estimate ground truth queries using click signals and query signals, with document relevance per query determined using a click/skip formula.

These jobs produce data that can be used for query rewriting or to inform updates to the synonyms.txt file.

Head/Tail Analysis

Perform head/tail analysis of queries from collections of raw or aggregated signals, to identify underperforming queries and the reasons. This information is valuable for improving overall conversions, Solr configurations, auto-suggest, product catalogs, and SEO/SEM strategies, in order to improve conversion rates.
Phrase Extraction

Identify multi-word phrases in signals.
Synonym and Similar Queries Detection Jobs

Use this job to generate pairs of synonyms and pairs of similar queries. Two words are considered potential synonyms when they are used in a similar context in similar queries.
Token and Phrase Spell Correction

Detect misspellings in queries or documents using the numbers of occurrences of words and phrases.

SQL Aggregation

A Spark SQL aggregation job where user-defined parameters are injected into a built-in SQL template at runtime.

Ranking Metrics

Calculate relevance metrics (nDCG and so on) by replaying ground truth queries against catalog data using variants from an experiment.

These jobs analyze signals and generate matrices used to provide collaborative recommendations.

ALS Recommender

Use this job when you want to compute user recommendations or item similarities using a collaborative filtering recommender. You can also implement a user-to-item recommender in the advanced section of this job’s configuration UI. This job uses SparkML’s Alternating Least Squares (ALS).

Use the BPR Recommender instead.
Query-to-Query Similarity

Train a collaborative filtering matrix decomposition recommender using SparkML’s Alternating Least Squares (ALS) to batch-compute query-query similarities. This can be used for items-for-query recommendations as well as queries-for-query recommendations.

Cluster Labeling

Use this job when you already have clusters or well-defined document categories, and you want to discover and attach keywords to see representative words within those existing clusters. (If you want to create new clusters, use the Document Clustering job.)
Document Clustering

The Document Clustering job uses an unsupervised machine learning algorithm to group documents into clusters based on similarities in their content. You can enable more efficient document exploration by using these clusters as facets, high-level summaries or themes, or to recommend other documents from the same cluster. The job can automatically group similar documents in all kinds of content, such as clinical trials, legal documents, book reviews, blogs, scientific papers, and products.
Logistic Regression Classifier Training

Train a regularized logistic regression model for text classification.
Outlier Detection

Use this job when you want to find outliers from a set of documents and attach labels for each outlier group.
Random Forest Classifier Training

Train a random forest classifier for text classification.
Word2Vec Model Training

Train a shallow neural model, and project each document onto this vector embedding space.

This job is deprecated in Fusion 5.2.0.

Parallel Bulk Loader

The Parallel Bulk Loader (PBL) job enables bulk ingestion of structured and semi-structured data from big data systems, NoSQL databases, and common file formats like Parquet and Avro.