Fusion Microservices
Overview
The table below lists the Fusion microservices deployed by the Helm chart.
| Fusion is a complex distributed application composed of many stateful and stateless services designed to support demanding search-oriented workloads at high scale. |
For Docker image versions associated with microservices, see the list of Docker images and versions for each Fusion release.
| Microservice | Required for Fusion | Protocol | Deployment or StatefulSet | Node Pool Assignment | Autoscaling Supported | Description |
|---|---|---|---|---|---|---|
|
Yes |
REST/HTTP |
Deployment |
system |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Exposes endpoints for admin tasks, such as creating applications and running jobs. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the admin UI. |
|
No |
HTTP |
Deployment |
system |
Yes (CPU or custom metric). |
Orchestrates parallel jobs on Kubernetes. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Stores logs and prior Argo workflow runs. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the login form. |
|
No |
REST/HTTP |
StatefulSet |
analytics or system |
Not supported. |
REST service for supporting non-RPC connector plugins. |
|
No |
REST/HTTP |
Deployment |
analytics or system |
Not required. One pod is enough for most clusters. |
Routes REST API requests to classic-rest-service and connectors-rpc. |
|
No |
gRPC |
Deployment |
analytics or system |
Yes (CPU or custom metric). |
gRPC service for managing SDK-based connector plugins. |
|
No |
HTTP/TCP |
Deployment |
analytics or system |
Yes (CPU or custom metric). |
Deployment for each connector plugin type. Note: There is a base deployment, connector-plugin with 0 replicas. This is used as a deployment template for each connector plugin type. It should not be deleted or scaled. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the DevOps UI. Available in Fusion 5.0 through 5.5. |
|
Yes |
REST/HTTP |
Deployment |
search or analytics depending on write-volume |
Yes (CPU or custom metric). |
Processes indexing requests. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves the App Insights UI |
|
No |
REST/HTTP |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Configures and launches the Spark driver pod for running Spark jobs |
|
No |
REST/HTTP |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Performs admin tasks for creating and running Spark jobs. |
|
No |
HTTP |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Jupyter notebook for ad hoc analytics and visualization. |
|
No |
HTTP |
StatefulSet |
system |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Collects logs from the other microservices and either indexes into system_logs or ships them to an external service like Elastic |
|
No |
REST/HTTP |
Deployment |
analytics or system |
Not required. One pod is enough for most clusters. |
Dense Vector Search Engine for ML models active. |
|
No |
REST/HTTP |
Deployment |
analytics or system |
Not required. One pod is enough for most clusters. |
Handles metadata for Milvus service active. |
|
No |
REST/HTTP and gRPC |
Deployment |
search |
Yes (CPU or custom metric). |
Exposes gRPC endpoints for generating predictions from ML models. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the Predictive Merchandiser app. |
|
Yes |
HTTP |
Deployment |
search |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Performs authentication, authorization, and traffic routing. |
|
Yes |
HTTP |
StatefulSet |
search |
At least 3 nodes in high availability. You need to run 3 or 5 to keep quorum. |
Write Ahead Log (WAL) used for persistent message storage. |
|
Yes |
HTTP and TCP |
Deployment |
search |
At least 3 nodes in high availability. |
Contains REST API for managing administration and dispatcher for handling all message transfers. |
|
Yes |
REST/HTTP |
Deployment |
search |
Yes (CPU or custom metric). |
Processes query requests. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves static Web assets for the Rules UI. |
|
No |
Web |
Deployment |
system |
Not required. Minimum of 1, but 2 pods are recommended for high availability. |
Load balancing and proxy for Seldon Core deployments. |
|
No |
REST/GRPC |
Deployment |
system |
Yes (CPU or custom metric). |
Serves models built in any model building framework. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Maintains Seldon Core deployments for ML model serving active. |
|
Yes |
HTTP |
StatefulSet |
At least 3 nodes in search, 2 in analytics, and 2 in system |
Yes (CPU or custom metric). |
Search engine. |
|
No |
n/a |
single pod per job |
analytics or dedicated Node Pool for Spark jobs |
One per job. |
Launched by the job-launcher to run a Spark job |
|
No |
n/a |
one or more pods launched by the Spark driver for executing job tasks |
analytics or dedicated Node Pool for Spark jobs |
Depends on job configuration. Controlled by the spark.executor.instances setting. |
Executes tasks for a Spark job |
|
No |
REST/HTTP and JDBC |
Deployment |
analytics |
Not required. One pod is enough for most clusters. |
Performs admin tasks for creating and managing SQL catalog assets. Exposes a JDBC endpoint for the SQL service. |
|
No |
Web |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Retrieves and renders Predictive Merchandiser templates. |
|
No |
HTTP |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Is responsible for text extraction using Tika Server Parser |
|
No |
REST/HTTP |
Deployment |
system |
Not required. One pod is enough for most clusters. |
Serves App Studio-based Web apps. |
|
Yes |
TCP |
StatefulSet |
system |
No. You need to run 1, 3, or 5 ZooKeeper pods to keep quorum. Do not use HPA for scaling ZooKeeper. |
Stores centralized configuration and performs distributed coordination tasks. |