Legacy Product

Fusion 5.10
    Fusion 5.10

    REST Query Index Stage

    The REST Query index stage performs remote procedure calls to external services, merges the response from that service with a document being processed by the pipeline, and makes calls to the external system for each document. In Fusion version 3.0 and earlier, the stage is called the Query REST Remote Procedure Calls (RPC) Client stage.

    This stage can call external systems in any of the following ways:

    • any HTTP or HTTPS request: <http-protocol>://<path>

    • a Solr request: solr://<collection>/…​

    • a Fusion service request: extservice://<serviceName>/<path>

    The call syntax allows the use of document field variables. For example, variable ${docField} will be replaced with the value of the field named docField for the document being processed.

    Several types of responses are also supported:

    • POJOs

    • JSON

    • XML

    Call Parameters

    In the Fusion UI, select include to display the fields for the call parameters (params in the REST API). Parameters are specified as key-value pairs:

    Endpoint URI / uri

    A fully-qualified service URI. This can be an HTTP call, a Solr request, or a Fusion service call

    * any HTTP or HTTPS request: <http-protocol>://<path>

    * a Solr request: solr://<collection>/…​

    * a Fusion service request: extservice://<serviceName>/<path>

    Call Method / method

    The method to use for the RPC call. Supported methods are GET, POST, PUT, and DELETE.

    Query parameters / queryParams

    Query parameters to be passed with the request.

    Request protocol headers / headers

    Request protocol headers, such as "Content-Type" : "application/json".

    Request entity (as string) / entity

    The request body. This parameter can only be configured via the REST API; see the example below.

    The params can also take variable substitution expressions. Variables are expressed as ${fieldName}, where fieldName is a name of the current document’s field (or id for the document’s id). Only the first value of a multi-valued field is used for substitution, and this value is treated as a string.

    For example, a queryParams could be constructed as follows:

    "params" : {
        "uri" : "solr://collection1/select",
        "method" : "GET",
        "queryParams" : {
          "q" : "{!field f=a_txt v=$prodName}",
          "prodName" : "${prodName}"
    }

    In this example, the variable ${prodName} will be replaced with the string value of the 'prodName' field in the current document being processed. If the current document contains "iPhone 6" as the value of 'prodName', the resulting query in this example will be a fielded search q=a_txt:iPhone 6.

    Mapping Rules

    The mappingRules property takes the following, specified as key-value pairs:

    • path. an XPath expression. It is assumed that this expression always returns a node list, and each returned node is processed separately and its converted value is added to a multi-valued field.

    • target. the name of the target field in the current document where the value(s) will be stored.

    • append. if true, values extracted from the RPC response will be appended to the target multi-valued field. When this is false, the default, existing values in the target field will be discarded and replaced with the values from the RPC response.

    • xml. if true, the extracted DOM nodes will be separately serialized to XML and the resulting XML-formatted text will be added to the target field. When this is false, the default, the extracted DOM nodes will be flattened and converted to a list of fields. Field names in this case will correspond to XML element names, dot-separated, and element attributes will be represented as fields with @attributeName suffix.

    Examples

    Create a REST Query stage to merge results from another Solr system:

    {
      "type" : "indexing-rpc",
      "id" : "BasicSolrCall",
      "mappingRules" : [ {
        "path" : "//result/doc[1]/str[@name='foo_s']/text()",
        "target" : "foo_s",
        "append" : true,
        "xml" : false
      }, {
        "path" : "//result/doc/arr[@name='a_txt']/*",
        "target" : "doc_txt",
        "append" : true,
        "xml" : false
      } ],
      "debug" : true,
      "params" : {
        "uri" : "solr://collection1/select",
        "method" : "GET",
        "queryParams" : {
          "q" : "a_txt:${doc_value}"
        },
        "headers" : { }
      },
      "skip" : false,
      "label" : "indexing-rpc"
    }

    Upload a stopword list:

    {
     "type" : "indexing-rpc",
     "id" : "demo",
     "debug" : true,
     "params" : {
       "uri" : "http://localhost:8764/api/stopwords/movies",
       "method" : "PUT",
       "headers" : { "Content-Type" : "application/json" },
       "entity" : "New stopword list"
     },
     "skip" : false,
     "label" : "Indexing RPC Demo",
     "type" : "indexing-rpc"
    }

    Configuration

    When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.

    Add results of an REST (RPC) call to the document.

    skip - boolean

    Set to true to skip this stage.

    Default: false

    label - string

    A unique label for this stage.

    <= 255 characters

    condition - string

    Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.

    params - Call Parameters

    uri - string

    method - string

    One of GET, POST, PUT, or DELETE

    Allowed values: getputpostdelete

    queryParams - Query parameters

    headers - Request protocol headers

    entity - string

    debug - boolean

    Setting to true will add a number of properties to either the context (in the Query case) or the document (in the indexing case)

    Default: false

    hasNoSideEffects - boolean

    To run this stage in simulation mode, set to 'true'.

    Default: false

    mappingRules - array[object]

    object attributes:{path required : {
     display name: XPath expression
     type: string
    }
    target required : {
     display name: Target field
     type: string
    }
    append : {
     display name: Append to Existing Values in Target Field
     type: boolean
    }
    xml : {
     display name: Add as an XML Fragment
     type: boolean
    }
    }