Legacy Product

Fusion 5.10
    Fusion 5.10

    Field Processors

    Field response processors include ones to perform these operations:

    • Capitalize the display value of the given field names.

    • Add Highlighting to Fields.

    • Tag a document with classifications based on field values.

    • Format a date object, replacing the original date value with another.

    • Parse dates out of field values.

    • Set the value of a field that is missing a value, based on the value of a different field.

    • Parse the String value of the specified fields into an Object representation.

    • Create a multivalued field from a single field value by using a separator.

    • Extract the hostname from URLs and place it in a field named 'site'.

    • Process fully qualified URLs in field values and markup, and add anchor tags for active links in the display values.

    • Statically add metadata to documents that match a given regular expression.

    • Replace field values (actual, display, or both) that are HTML or URL encoded with decoded values.

    • Replace field values (actual, display, or both) that match a given regular expression with a different value.

    • Make Twitter users and hashtags clickable in the display value.

    • Duplicate a field, creating two separate instances.

    • Create a new field by joining multiple existing fields using a pattern expression.

    • Localize the values of a field using a specified bundle.

    twigkit.search.processors.response.CapitaliseFieldValuesProcessor

    Capitalize the display value of the given field names.

    name: twigkit.search.processors.response.CapitaliseFieldValuesProcessor
fields: firstName,lastName

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    twigkit.search.processors.response.HighlightFieldValuesProcessor

    Add Highlighting to Fields. For a more detailed overview, see the highlighting page.

    twigkit.search.processors.response.FieldEntityExtractor

    Tag a document with classifications based on field values. Using the specified fields, look for patterns provided in a properties file and add classifications to a given field if the value matches.

    name: twigkit.search.processors.response.FieldEntityExtractor
fields: issues
classificationField: categorizedIssues
bundle: my-issues

    With:

    #my-issues_en.properties in /resources/conf
Foo
Bar
Bam

    This can be used to pull out keywords from a given field into a new field for display. If the Example above had a field named 'issues' with 'The problem is Foo something else' as value, then the FieldEntityExtractor would match on Foo, and stick 'Foo' into a new categorizedIssues field. This can be used to create new metadata fields, or even filter fields with sensitive data, to just pull out what you want to show.

    bundle (java.lang.String)
    A properties file containing entities to extract and an optional replacement value when found (for example, IBM = International Business Machines where the latter would be stored as the match for IBM). These are expressed as regular expressions (regex) so as well as simple matching based on whether the value was found within a field, entities can be recognized based on a specific pattern within the text of the chosen fields.

    fields (java.lang.String)
    Comma-separated list of fields that should be used to search for matches in order to classify the document.

    classificationField (java.lang.String)
    The field in which to store the classification values. The flexible pattern based approach used allows a document to be tagged with multiple classifications if several matches are found.

    twigkit.search.processors.response.DatePartExtractor

    Format a date object, replacing the original date value with another. Use on fields that are already date objects.

    name: twigkit.search.processors.response.DatePartExtractor
fields: issues
pattern: dd MMM yyyy

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    pattern (java.lang.String)
    Format a Date object according to this Date pattern. Use Java’s SimpleDateFormat syntax. In the examples, for example, you will see that pattern="EEE, MMM d, ''yy" results in Wed, Jul 4, '01.

    twigkit.search.processors.response.FieldDateParser

    Parse dates out of field values. To convert String data into Date objects.

    name: twigkit.search.processors.response.FieldDateParser
fields: issues
pattern: dd MMM yyyy

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    pattern (java.lang.String)
    The pattern to use when parsing the String value to a Date.

    twigkit.search.processors.response.FallbackFieldValue

    Set the value of a field that is missing a value, based on the value of a different field.

    name: twigkit.search.processors.response.FallbackFieldValue
field: phoneNumber
fallback: mainPhoneNumber
pattern:
values: display
decode: false

    field (java.lang.String)
    Field that should be affected by this processor.

    fallback (java.lang.String)
    Field to use for the fallback values.

    pattern (java.lang.String)
    Regex pattern to use to extract a value from the fallback field

    values (java.lang.String)
    Which forms of the value to check for emptiness - 'display', 'actual', or 'either'

    decode (java.lang.Boolean)
    Whether to perform URL decoding on the fallback values

    twigkit.search.processors.response.FieldValueParser

    Parse the String value of the specified fields into an Object representation.

    name: twigkit.search.processors.response.FieldValueParser
fields: amount

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    twigkit.search.processors.response.FieldToMultiValueFieldProcessor

    Create a multivalued field from a single field value by using a separator.

    name: twigkit.search.processors.response.FieldToMultiValueFieldProcessor
field: cities
separator: ,

    field (java.lang.String)
    The name of the field containing the value to be split.

    separator (java.lang.String)
    The separator to use when splitting the value.

    twigkit.search.processors.response.HostNameExtractor

    Extract the hostname from URLs and place it in a field named 'site'.

    name: twigkit.search.processors.response.HostNameExtractor

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    twigkit.search.processors.response.LinkMarkupProcessor

    Process fully qualified URLs in field’s 'actual' value, and markup with anchor tags for active links in display value.

    name: twigkit.search.processors.response.LinkMarkupProcessor
fields: url

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    twigkit.search.processors.response.RegExFieldValueTaggerProcessor

    Statically add metadata to documents that match a given regular expression (in one or more fields).

    name: twigkit.search.processors.response.RegExFieldValueTaggerProcessor
fields: path
classificationField: type
pattern: [\w]+

    The example above looks at a field named path for example, "foo/bar/bam", and breaks all words into a multivalued field named type with multiple values foo, bar and bam using a regex pattern.

    If you want to change data rather than just use it, use a ReplaceFieldValue processor.

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    classificationField (java.lang.String)
    The field that should contain the metadata classificationValue if the pattern matches.

    pattern (java.lang.String)
    The pattern to match to the values in the fields defined with the fields parameter.

    twigkit.search.processors.response.DecodeFieldValueProcessor

    Replace field values (actual, display, or both) that are HTML or URL encoded with decoded values. A use case might be to replace the display value of a URL field. Example usage:

    name: twigkit.search.processors.response.DecodeFieldValueProcessor
fields: url_display
encoding: url
values: display

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    values (java.lang.String)
    Whether to replace 'actual', 'display' or 'both' values.
    Default: 'both'

    encoding (java.lang.String)
    The encoding of the value to be decoded; 'url' or 'html'.
    Default: 'url'

    twigkit.search.processors.response.ReplaceFieldValue

    Replace field values (actual, display, or both) that match a given regular expression with a different value. The replacement value can contain back-references to matches. A common use case for this is to use a CopyFieldProcessor first, then make changes. Example:

    name: twigkit.search.processors.response.ReplaceFieldValue
fields: folder
replace: ^(.*/).*$
with: $1

    The example above strips off a file name from the end of a folder path to leave just the path using capture expressions and back-references regular expressions.

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    replace (java.lang.String)
    The pattern to replace (can contain regular expressions).

    with (java.lang.String)
    The replacement (can contain backreferences to the regular expression pattern).

    values (java.lang.String)
    Whether to replace 'actual', 'display' or 'both' values. Default: 'both'

    ignoreCase (java.lang.Boolean)
    Whether to ignore case during pattern matching. Default: false

    twigkit.search.processors.response.TweetMarkupProcessor

    Make Twitter users and hashtags clickable in the display value.

    name: twigkit.search.processors.response.TweetMarkupProcessor
fields: twitter_user

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    twigkit.search.processors.response.CopyFieldProcessor

    Duplicate a field, creating two separate instances.

    name: twigkit.search.processors.response.CopyFieldProcessor
from: url
to: my_url

    from (java.lang.String)
    Name of field to copy (clone).

    to (java.lang.String)
    Name to assign to the new field.

    twigkit.search.processors.response.ConcatenateFields

    Create a new field by joining multiple existing fields using a pattern expression.

    expression (java.lang.String)
    Concatenated field pattern defined using double curly braces (see below)

    target (java.lang.String)
    Name of the new field to create in the response

    name: twigkit.search.processors.response.ConcatenateFields
expression: {{MemberStreet1}} {{MemberStreet2}} {{MemberCity}} {{MemberState}} {{MemberZipCode}} {{MemberCountry}}
target: compositeAddress

    Or create a new image field that uses a field value as part of an expression:

    name: twigkit.search.processors.response.ConcatenateFields
expression: http://your/custom/path/{{MemberCity}}.jpg
target: image_url

    Then use a <media:image> tag to output the image field in the result output:

    <media:image field-name="image_url" width="156" height="156" ... >

    twigkit.search.processors.response.LocaliseFieldValueProcessor

    Localize the values of a field using a specified bundle. For example:

    name: twigkit.search.processors.response.LocaliseFieldValueProcessor
bundle: languages
locale: en
fields: language

    As an example, add a file named languages_en.properties to your class path (for example, to src/main/resources) and containing these key-value pairs (truncated):

    aa = Afrikaans
ab = Abkhaz
am = Amharic
ar = Arabic
az = Azerbaijani
ba = Bashkir
be = Belarusian
bg = Bulgarian
bm = Bamanankan
bn = Bengali
bo = Tibetan
br = Breton
bs = Bosnian
ca = Catalan
co = Corsican
cr = Cree
cs = Czech
cy = Welsh
etc.

    fields (java.lang.String)
    Comma-separated list of fields that should be affected by this processor.

    values (java.lang.String)
    Whether to replace 'actual', 'display' or 'both' values. Default: both

    bundle (java.lang.String)
    The bundle to use.

    locale(java.lang.String)
    The locale to use.