Legacy Product

Fusion 5.10
    Fusion 5.10

    Milvus Query Stage

    The Milvus Query stage performs vectors similarity search in Milvus.

    Troubleshooting

    Milvus Vector Search Issue

    Issue: Create, Read, Update, and Delete (CRUD) operations for indexing and query objects save current user permissions in thread local variables. Permissions are saved only once, and are never updated for that thread. Calls to a pipeline with a REST call or to Milvus stages use the saved permissions, not the calling user permissions. And the saved permissions are then used to check authorization for Milvus or external http: and https: calls.

    If a thread does not have saved permissions, the call to Milvus is authorized for any user.

    Workaround for Milvus stages:

    1. Add the following permissions to all existing non-admin users:

      GET,POST,PUT:/http-call/milvus/**

    2. Restart all instances of query and indexing services.

    Update a Milvus collection

    Update and delete semantics are not supported on individual items in a Milvus collection. We recommend that you use a blue/green set of collections: reindex to apply updates to an empty Milvus collection, then swap between collections to apply the updates.

    Configuration

    When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.

    Perform vectors similarity search in Milvus

    skip - boolean

    Set to true to skip this stage.

    Default: false

    label - string

    A unique label for this stage.

    <= 255 characters

    condition - string

    Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.

    modelId - stringrequired

    ID of Model used to encode the vector

    modelOutputVector - stringrequired

    The name of the model output vector that stores the encoded data.

    Default: vector

    milvusResultsContextKey - stringrequired

    Name of the context key that will be used to store results from Milvus.

    >= 1 characters

    Default: milvus_results

    milvusVectorsCollection - stringrequired

    Name of the vectors collection in Milvus that should be used to search for similar vectors.

    >= 1 characters

    milvusNumResults - integerrequired

    Number of results to request from Milvus

    Default: 10

    milvusSearchParams - array[object]

    Parameters that should be used to search in Milvus. Specific to the chosen IndexType. For example it can be nprobe=128 for IVF indexes or ef=400 for HNSW index.

    object attributes:{param : {
     display name: Milvus Search Param
     type: string
    }    value : {
     display name: Value
     type: integer
    }    }

    failOnError - boolean

    Flag to indicate if this stage should throw an exception if an error occurs.

    Default: false