Legacy Product

Fusion 5.10
    Fusion 5.10

    Collection Analysis Jobs

    Use this job when you want to compute basic metrics about your collection, like average word length, phrase percentages, and outlier documents (with very many or very few documents).

    Use this job when you want to compute basic metrics about your collection such as average word length, phrase percentages, and outlier documents (with large or few number of terms)

    id - stringrequired

    The ID for this Spark job. Used in the API to reference this job. Allowed characters: a-z, A-Z, dash (-) and underscore (_)

    <= 128 characters

    Match pattern: ^[A-Za-z0-9_\-]+$

    trainingCollection - string

    Collection you want to analyze.

    >= 1 characters

    fieldToVectorize - string

    Field you want to analyze the lengths of. For example: description fields.

    >= 1 characters

    dataFormat - string

    Spark-compatible format which training data comes in (like 'solr', 'hdfs', 'file', 'parquet' etc)

    Default: solr

    Allowed values: solrhdfsfileparquet

    trainingDataFrameConfigOptions - object

    Additional spark dataframe loading configuration options

    trainingDataFilterQuery - string

    Solr query to use when loading training data

    >= 3 characters

    Default: *:*

    trainingDataSamplingFraction - number

    Fraction of the training data to use

    <= 1

    exclusiveMaximum: false

    Default: 1

    randomSeed - integer

    For any deterministic pseudorandom number generation

    Default: 1234

    outputCollection - string

    Solr Collection to store model-labeled data to

    sourceFields - string

    Solr fields to load (comma-delimited). Leave empty to allow the job to select the required fields to load at runtime.

    numDeviations - integerrequired

    Number of standard deviations away from the mean we deem acceptable for obtaining outlier from this collection. If you want all the documents, set this to a larger number.

    exclusiveMinimum: false

    dateField - string

    The field that corresponds to the date field you will be using

    type - stringrequired

    Default: collection_analysis

    Allowed values: collection_analysis