Ranking Metrics Jobs
Calculate relevance metrics (nDCG and so on) by replaying ground truth queries against catalog data using variants from an experiment.
use this job to calculate relevance metrics (nDCG etc..) by replaying ground truth queries (see ground truth job) against catalog data using variants from an experiment.
id - stringrequired
The ID for this Spark job. Used in the API to reference this job. Allowed characters: a-z, A-Z, dash (-) and underscore (_)
<= 128 characters
Match pattern: ^[A-Za-z0-9_\-]+$
groundTruthConfig - Configure ground truth dataset
Configure properties for Ground truth dataset
inputCollection - string
Input collection representing ground truth dataset
>= 1 characters
filterQueries - array[string]
Solr filter queries to apply against Ground truth collection
Default: "aggr_type_s:ground_truth"
queryField - string
Query field in the collection
Default: query_s
docIdField - string
Field containing ranked doc id's
Default: doc_id_s
weightField - string
Field representing the weight of document to the query
Default: weight_d
rankingExperimentConfig - Configure experiment
Configure properties for the experiment
inputCollection - string
Collection to run the experiment on
>= 1 characters
queryPipelines - array[string]
Pipeline variants for experiment
docIdField - string
Doc id field to retrieve values (Must return values that match the ground truth data)
Default: id
experimentId - string
Calculate ranking metrics using variants from experiment
>= 1 characters
experimentObjectiveName - string
Experiment objective name
>= 1 characters
defaultProfile - string
Default query profile to use if not specified in experiment variants
outputCollection - stringrequired
Output collection to save the ranking metrics to
>= 1 characters
rankingPositionK - integer
Ranking position at K for metrics calculation
Default: 10
metricsPerQuery - boolean
Calculate ranking metrics per each query in ground truth set and save them to Solr collection
Default: true
type - stringrequired
Default: ranking_metrics
Allowed values: ranking_metrics