Legacy Product

Fusion 5.10
    Fusion 5.10

    Encode into Milvus Index Stage

    The Encode into Milvus index stage invokes a machine learning model to encode a field and store it into Milvus.

    Troubleshooting

    Milvus Vector Search Issue

    Issue: Create, Read, Update, and Delete (CRUD) operations for indexing and query objects save current user permissions in thread local variables. Permissions are saved only once, and are never updated for that thread. Calls to a pipeline with a REST call or to Milvus stages use the saved permissions, not the calling user permissions. And the saved permissions are then used to check authorization for Milvus or external http: and https: calls.

    If a thread does not have saved permissions, the call to Milvus is authorized for any user.

    Workaround for Milvus stages:

    1. Add the following permissions to all existing non-admin users:

      GET,POST,PUT:/http-call/milvus/**

    2. Restart all instances of query and indexing services.

    Update a Milvus collection

    Update and delete semantics are not supported on individual items in a Milvus collection. We recommend that you use a blue/green set of collections: reindex to apply updates to an empty Milvus collection, then swap between collections to apply the updates.

    Configuration

    When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.

    Invokes a machine learning model to encode a field and store it into Milvus.

    skip - boolean

    Set to true to skip this stage.

    Default: false

    label - string

    A unique label for this stage.

    <= 255 characters

    condition - string

    Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.

    modelId - stringrequired

    ID of Model used to encode the vector

    modelOutputVector - stringrequired

    The name of the model output vector that stores the encoded data.

    Default: vector

    fieldToEncode - stringrequired

    Name of field to encode and store in Milvus.

    milvusCollection - stringrequired

    The name of the collection in Milvus that will be used to store the model output vector.

    failOnError - boolean

    Flag to indicate if this stage should throw an exception if an error occurs while generating a prediction for a document.

    Default: false