Release NotesFusion Connectors

Fixes Graph security trimming not working with Everyone except external access.

Allows connector to reach maximum referral. The connector will throw the exception but will not stop.

Added the Events Reducer feature, which prevents performing multiple requests to the same SharePoint object and ensures index integrity. Specifically, if multiple change events are made over the same SharePoint object, the connector detects and requests the most relevant event and crawls it correctly.

Fixed Graph Security Trimming when using custom-level permissions.

Fixed an issue to reindex restored lists.

Errors from item/attachment are deleted from crawlDB after being successfully retried.

Fixed a pagination issue that limited the number of records returned.

Added support for pagination of IBM Db2 (version 11 and earlier) by using a template sub-query:

-- add any WHERE, ORDER BY clauses as needed following your_table
SELECT * FROM ( SELECT *, ROW_NUMBER() OVER () AS row_num FROM your_table ) AS subquery WHERE row_num BETWEEN :offset + 1 AND (:offset + :limit)

Fixed an issue where UTF-8 characters stored as CLOB data types didn’t index properly.

LIMIT - specifies the number of rows returned in the results.

OFFSET - dictates the number of rows to skip from the beginning of the returned data before presenting the results.

disableAutomaticPagination - disables automatic pagination to ignore limit and offset fields.

Indexes the all-users acl document when indexing from AzureAD.

Fixes Graph Security Trimming with Everyone except external access in SharePoint Online.

Exposes configuration options for the validation timeout.

Resolves an issue where documents were deleted on incremental crawls when stray delete was enabled.

This adds back in the Kaltura External Media Entry Compare Attribute fields.

The connector supports remote configurations.

Supports remote configurations.

Uses the SDK that correctly processes the NOT-EQUAL phrase in delete-by-query.

Supports unauthenticated proxy.

The connector allows OAuth 2 support for JWT token.

This release expands the types of character large objects (CLOBs) which can be indexed to include IBM CLOBs.

For SharePoint Online connector OAuth authentication and NTLM authentication, an issue with the SharePoint Optimized V2 connector crawl was fixed. Now the crawl fails if the data source is configured with incorrect credentials.

When the moderation status of a published document moves to unpublished, the document is deleted from the index.

The URL that requests folder metadata is now properly encoded and retrieves requested files correctly.

The datasource indexing job will now return an error when invalid access credentials are provided instead of reporting a success.

The Security parameter is now set to enabled by default.

Fixed an issue that occurred when reaching Kaltura’s API limit of 10,000 documents. Indexing over 10,000 documents should now work as expected.

The JDBC V2 connector now supports indexing character large objects (CLOBs).

A NoClassDefFound error in the connector has been fixed.

MapDB tables that are no longer needed after indexing are now automatically removed when the indexing job completes.

The connector no longer uses the index pipeline client when the onlyMetadata property is set to true or when Tika async parsing is enabled. Instead, documents are processed using the Connectors SDK’s .newDocument method.

A bug with the Rewrite URI configuration option that prevented the configuration from being applied during the indexing job has been fixed.

The SAML, Form, and NTML authorization form fields are updated to match those present in the Web V1 connector.

A bug that affected JDBC datasources with large document sizes has been fixed. Previously, the indexing job would fail and produce a Solr error message similar to the following:

Document contains at least one immense term in field="FIELD_NAME"

Added support for the graph security trimming stage for Active Directory in Azure.

A bug that prevented the connector from retrying after a connection error has been fixed.

The statement.setMaxRows() field was added to resolve a timeout error for large queries on a JDBC data source.

A new field, _lw_document_type_s, has been added to differentiate between document types. SharePoint content documents use the value contentDocument. ACL documents use the value aclDocument.

A bug that caused a NullPointerException error when the listDefaultDisplayFormUrl value is null has been fixed.

A bug that prevented the connector from indexing SharePoint Online URLs containing certain special characters, such as +, &, and #, has been fixed.

A bug that stopped the indexing job unexpectedly when the digestValue request fails has been fixed.

Fixed a memory leak that resulted in an OutOfMemoryError runtime error, causing recrawls to quickly fail.

Fixed an issue where sometimes the connector did not index all attachments in full crawls.

Fixed an error related to OAuth authentication where the body would not return the expected value.

Folders with unique permissions no longer inherit list permissions instead of their own.

Fixed a bug that sometimes resulted in an OutOfMemory: Java Heap Space error when the connector was run remotely.

Fixed a dependency issue that sometimes resulted in a Error starting controllers error that caused the job to fail.

Fixed a bug that removed access control lists (ACLs) during incremental crawls in Azure Active Directory. This bug resulted from a deviation from the LDAP ACLs V2 connector’s unique incremental crawl behavior for Azure Active Directory.

Fixed a bug that sometimes deleted ACLs that the indexing job failed to update during its previous run.

Known issue: The job may complete with a "Successful" status even when a network communication error occurred.

Fixed a bug that sometimes generated a "User cannot be found" error statement and prevented the connector from indexing files.

Updated the description of the Item size limit property to clarify its behavior.

Fixed a bug that sometimes prevented the connector from parsing PDF, Word, Excel, and other file types.

Some conflicting dependencies were removed to resolve errors.

A missing dependency, guava-retrying, was added to resolve a NoClassDefFoundError error.

A bug was fixed that resulted in the connector performing full crawls instead of recrawling as expected.

Known issue: JavaScript evaluation is not functional at this time.

Known issue: The counter values in the Datasources > DATASOURCE_NAME > Job History view do not reflect actual values, when the datasource uses the Web V2 connector.

A bug that sometimes prevented files from being deleted as expected when the datasource was recrawled was fixed.

A bug that prevented the connector from indexing data within certain lists was fixed.

When indexing data to the MetaInfo_s field, an error could occur that prevented indexing data when the data was too large. Now, data larger than 32,766 bytes is indexed to the MetaInfo_txt field instead.

The connector now accepts URLs with spaces.