IBM Spectrum Conductor is an enterprise class, multitenant solution for Apache Spark and Anaconda/Python. It provides a framework to enable other application integrations, sharing resources dynamically. It specifically enables your organization to deploy Apache Spark and Python-based applications efficiently and effectively, supporting multiple concurrent instances and versions. It can help increase performance and scale, optimize resource usage, and eliminate silos of resources that would otherwise be tied to multiple, separate Apache Spark or Anaconda/Python implementations.
IBM Spectrum Conductor (known previously as IBM Spectrum Conductor with Spark and Platform Conductor for Spark) confidently deploys modern computing frameworks and services for a multitenant enterprise environment, both on-premises and in the cloud.
As an end-to-end solution, IBM Spectrum Conductor provides the following benefits:
- Increases performance and scale through granular and dynamic resource allocation for Spark instance groups that share a resource pool.
- Maximizes usage of resources and eliminates silos of resources that would otherwise each be tied to separate application implementations.
- Integrates with IBM Cloud Private so that you can run an IBM Spectrum Conductor master container in a Kubernetes cluster.
- Integrates with Docker so that you can run Spark instance groups in Docker containers.
- Integrates with IBM Data Science Experience (DSX) Local or Desktop so that you can use notebooks from DSX with Spark instance groups to submit Spark workload.
IBM Spectrum Conductor provides multitenancy through Spark instance groups. You can deploy modern computing frameworks and services, such as Spark and Anaconda efficiently and effectively, supporting multiple versions and instances of each framework and service.
Apache Spark requires a separate resource manager and can interface with various distributed storage models. IBM Spectrum Conductor addresses both of these requirements:
- It incorporates Spectrum Computing’s resource manager, a proven technology for granular and dynamic resource management.
- It supports IBM Spectrum Scale technology for storage management. IBM Spectrum Scale enables high-performance access to a common data to support a scale-out solution and provide a high-availability platform.
IBM Spectrum Conductor also includes a Spark distribution, making it a complete end-to-end solution for organizations looking to deploy Spark both for exploratory projects and in production environments.
Join the IBM Spectrum Conductor technical community to connect with other members and collaborate with experts by accessing blogs, forums, and wikis.
DownloadDownload a free trial
Service requests and PMRs
Request for enhancement
Choose a request type: