An Ansible based installer for Kafka & Confluent Platform on DC/OS
This is a DC/OS Community project and is not supported or warrantied by Mesosphere.
- Deploys Confluent Platform Kafka with full security - Active Directory/Kerberos GSSAPI & SASL_SSL authentication or with TLS & SSL
- Supports deployment of multiple Kafka clusters for multi-tenant environments
- DC/OS group/folder support for organising clusters such as /dev/123456-kafka/
- DC/OS strict mode security out of the box (only strict at this time)
- Deployment of an Active Directory server on AWS for testing
- Dynamic generation of:
- a batch script to add Active Directory users, principals and generate keytabs
- options.json configs for every service
- client-jass.conf for every service
- endpoint dependencies for each service
- Automation of:
- generating and adding binary and text secrets to the DC/OS secret store
- configuring DC/OS security service accounts and ACLs
- Menu system to wrap the Ansible playbooks using GNU make
- Janitor cleanup with one command
- End to end client testing scripts with documented steps
- Easy access to Confluent Control Center
- Archive of each cluster's configuration assets
- CCM support
- Endpoint management
- Configurable resources for JSON options
- Apache Kafka support
- MIT Kerberos support
- Standalone monitoring deployment integrated with dcos-metrics
- Performance benchmarking
- Strict mode only
- Cluster identifier limited to 9 chars due to Active Directory Kerberos naming limitations
- Confluent Connect does not provide security on its REST API endpoint, this is a limitation in the Confluent product
- External (to DC/OS ) service discovery is work in progress
Ansible does the heavy lifting, talking over localhost, to generate configuration files based on templates and talking directly to the DC/OS CLI to manage the deployment.
Really, all this is doing is automating the manual process and myriad of configurations required.
- You're running a multi-tenant / private cloud environment based on DC/OS where you want to deploy multiple Confluent Platform Kafka clusters
- You want to test the full Confluent Platform stack on a DC/OS strict mode cluster with all security features enabled
- You want a convenient testing framework to validate the components are working correctly
- You don't want to run through >50 manual steps