Legacy Product

Fusion 5.4

Develop a Custom Connector

Java SDK configuration

To build a valid connector configuration, you must:

  • Define an interface.

  • Extend ConnectorConfig.

  • Apply a few annotations.

  • Define connector methods and annotations.

All methods that are annotated with @Property are considered to be configuration properties. For example, @Property() String name(); results in a String property called name. This property would then be present in the generated schema.

Here is an example of the most basic configuration, along with required annotations:

@RootSchema(
    title = "My Connector",
    description = "My Connector description",
    category = "My Category"
)
public interface MyConfig extends ConnectorConfig<MyConfig.Properties> {

  @Property(
      title = "Properties",
      required = true
  )
  public Properties properties();

  /**
   * Connector specific settings
   */
  interface Properties extends FetcherProperties {

    @Property(
        title = "My custom property",
        description = "My custom property description"
    )
    public Integer myCustomProperty();

  }

}

The metadata defined by @RootSchema is used by Fusion when showing the list of available connectors. The ConnectorConfig base interface represents common, top-level settings required by all connectors. The type parameter of the ConnectorConfig class indicates the interface to use for custom properties.

Once a connector configuration has been defined, it can be associated with the ConnectorPlugin class. From that point, the framework takes care of providing the configuration instances to your connector. It also generates the schema, and sends it along to Fusion when it connects to Fusion.

Schema metadata can be applied to properties using additional annotations. For example, applying limits to the min/max length of a string, or describing the types of items in an array.

Nested schema metadata can also be applied to a single field by using "stacked" schema based annotations:

interface MySetConfig extends Model {
    @SchemaAnnotations.Property(title = "My Set")
    @SchemaAnnotations.ArraySchema(defaultValue = "[\"a\"]")
    @SchemaAnnotations.StringSchema(defaultValue = "some-set-value", minLength = 1, maxLength = 1)
    Set<String> mySet();
  }

Plugin client

The Fusion connector plugin client provides a wrapper for the Fusion Java plugin-sdk so that plugins do not need to directly talk with gRPC code. Instead, they can use high-level interfaces and base classes, like Connector and Fetcher.

The plugin client also provides a standalone "runner" that can host a plugin that was built from the Fusion Java Connector SDK. It does this by loading the plugin zip file, then calling on the wrapper to provide the framework interactions.

Standalone Connector Plugin Application

The second goal of the plugin-client is to allow Java SDK plugins to run remotely. The instructions for deploying a connector using this method are provided below.

Locating the UberJar

The uberjar is located in this location in the Fusion file system:

$FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar

where $FUSION_HOME is your Fusion installation directory and <version> is your Fusion version number.

Starting the Host

To start the host app, you need a Fusion SDK-based connector, built into the standard packaging format as a .zip file. This zip must contain only one connector plugin.

Here is an example of how to start up using the web connector:

java -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zip

To run the client with remote debugging enabled:

java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5010 -jar $FUSION_HOME/apps/connectors/connectors-rpc/client/connector-plugin-client-<version>-uberjar.jar fusion-connectors/build/plugins/connector-web-4.0.0-SNAPSHOT.zip

Java SDK security

Fusion connectors support SSL/TLS.

Java Plugin Client

The information here is specific to running a Java SDK plugin outside of Fusion.

The Fusion Connector Plugin Client supports several variations of SSL/TLS auth. The examples below show the relevant Java properties.

Example with Mutual TLS auth and private key passwords

-Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.client.certChain=./sslcerts/client.crt
-Dcom.lucidworks.fusion.tls.client.privateKey=./sslcerts/client.pem
-Dcom.lucidworks.fusion.tls.client.privateKeyPassword=password123
-Dcom.lucidworks.fusion.tls.enabled=true</td>

Example without TLS auth and no private key passwords:

-Dcom.lucidworks.apollo.app.hostname=myhost
-Dcom.lucidworks.fusion.tls.trustCertCollection=./sslcerts/ca.crt
-Dcom.lucidworks.fusion.tls.enabled=true

Simple Connector

Connector Description

This connector generates a configurable number of documents, all with random titles and body fields.

Quick start

  1. Clone the repo:

    git clone https://github.com/lucidworks/connectors-sdk-resources.git
cd connectors-sdk-resources/java-sdk/connectors/
./gradlew assemblePlugins

  1. This produces one zip file, named simple-connector.zip, located in the build/plugins directory. This artifact is now ready to be uploaded directly to Fusion as a connector plugin.

  2. See the following instructions on how to build, deploy, and run the plugin.

Connector properties

Random content generator properties

Property Name Property description

Total

The total number of documents to generate

Minimum number of sentences

The minimum number of sentences to generate per document, the random generator will use this value as lower bound to calculate a random number of sentences

maximum number of sentences

The maximum number of sentences to generate per document, the random generator will use this value as upper bound to calculate a random number of sentences

How to use the connector

  • Create a configuration with the properties listed above.

  • After the first job is completed, the connector will index the same number of documents as defined in the Random content generator properties.Total property.