Twitter Search V1 Connector Configuration Reference
The Twitter Search connector uses Twitter’s search API to query Twitter for tweets that match specific parameters. It allows querying for any keyword, location or other query terms.
|
|
This connector is no longer functional in Fusion 5.9 and later. This incompatibility arises due to changes implemented in the data source version or related APIs.
Although the V1 connector might still be visible within the Fusion UI, it cannot be used effectively. To ensure uninterrupted operation, we strongly recommend switching to a valid V2 alternate connector.
The replacement for this connector is in active development at this time and will be released at a future date.
|
|
|
When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.
|
Twitter Search uses Twitter's search API to query Twitter for tweets that match specific parameters. It allows querying for any keyword, location or other terms.
id - stringrequired
Unique name for this datasource.
>= 1 characters
Match pattern: ^[a-zA-Z0-9_-]+$
pipeline - stringrequired
Name of an existing index pipeline for processing documents.
>= 1 characters
description - string
Optional description for this datasource.
parserId - string
Parser used when parsing raw content. For some connectors, a configuration to 'retry' parsing if an error occurs is available as an advanced setting
properties - Properties
Datasource configuration properties
db - Connector DB
Type and properties for a ConnectorDB implementation to use with this datasource.
type - string
Fully qualified class name of ConnectorDb implementation.
>= 1 characters
Default: com.lucidworks.connectors.db.impl.MapDbConnectorDb
inlinks - boolean
Keep track of incoming links. This negatively impacts performance and size of DB.
Default: false
aliases - boolean
Keep track of original URI-s that resolved to the current URI. This negatively impacts performance and size of DB.
Default: false
inv_aliases - boolean
Keep track of target URI-s that the current URI resolves to. This negatively impacts performance and size of DB.
Default: false
consumer_key - string
The OAuth Consumer Key is provided by Twitter when registering the application.
>= 1 characters
consumer_secret - string
The OAuth Consumer Secret is provided by Twitter when registering the application.
>= 1 characters
access_token - string
The OAuth Access Token is provided by Twitter when registering the application.
>= 1 characters
token_secret - string
The OAuth Token Secret is provided by Twitter when registering the application.
>= 1 characters
max_docs - integer
The maximum number of documents to pull down, as a long. -1 for no limit
Default: -1
sleep - integer
The amount of time, in milliseconds, to sleep when listening so as to not get throttled
Default: 10000
commit_on_finish - boolean
Set to true for a request to be sent to Solr after the last batch has been fetched to commit the documents to the index.
Default: true
initial_mapping - Initial field mapping
Provides mapping of fields before documents are sent to an index pipeline.
skip - boolean
Set to true to skip this stage.
Default: false
label - string
A unique label for this stage.
<= 255 characters
condition - string
Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.
reservedFieldsMappingAllowed - boolean
Default: false
retentionMappings - array[object]
Fields that should be kept or deleted
object attributes:{field required : {
display name: Field
type: string
}operation : {
display name: Operation
type: string
}}
updateMappings - array[object]
Values that should be added to or set on a field. When a value is added, any values previously on the field will be retained. When a value is set, any values previously on the field will be overwritten.
object attributes:{field required : {
display name: Field
type: string
}value required : {
display name: Value
type: string
}operation : {
display name: Operation
type: string
}}
translationMappings - array[object]
Fields that should be moved or copied to another field. When a field is moved, the values from the source field are moved over to the target field and the source field is removed. When a field is copied, the values from the source field are copied over to the target field and the source field is retained.
object attributes:{source required : {
display name: Source Field
type: string
}target required : {
display name: Target Field
type: string
}operation : {
display name: Operation
type: string
}}
unmappedRule - Unmapped Fields
Fields not mapped by the above rules. By default, any remaining fields will be kept on the document.
keep - boolean
Keep all unmapped fields
Default: true
delete - boolean
Delete all unmapped fields
Default: false
fieldToMoveValuesTo - string
Move all unmapped field values to this field
fieldToCopyValuesTo - string
Copy all unmapped field values to this field
valueToAddToUnmappedFields - string
Add this value to all unmapped fields
valueToSetOnUnmappedFields - string
Set this value on all unmapped fields
queries - array[string]
A list of queries to perform
lang - string
Restrict Tweets to the given language
Default: en
Allowed values: aaabaeafakamanarasavayazbabebgbhbibmbnbobrbscacechcocrcscucvcydadedvdzeeeleneoeseteufafffifjfofrfygagdglgngugvhahehihohrhthuhyhziaidieigiiikinioisitiuiwjajijvkakgkikjkkklkmknkokrkskukvkwkylalblglilnloltlulvmgmhmimkmlmnmomrmsmtmynanbndnengnlnnnonrnvnyocojomorospapiplpsptqurmrnrorurwsascsdsesgsiskslsmsnsosqsrssstsusvswtatetgthtitktltntotrtstttwtyugukuruzvevivowawoxhyiyozazhzu