Core Concepts
Stacklet#
The stacklet is the fundamental unit representing one deployed data
platform component — for example, one running Kafka cluster or one running
Airflow instance, created from a custom resource like KafkaCluster or
AirflowCluster.
Roles and role groups#
Every operator organizes its stacklet into roles (for example Kafka's
broker role, or Airflow's webserver, worker, scheduler roles).
Within a role, you define role groups — named subsets of that role with
their own replica count and configuration overrides, so you can, for
instance, run one role group on GPU nodes and another on standard nodes.
Platform overview & multi-platform support#
Covers overall architecture and deployment, plus how the same operator model extends to Stackable Data Platform (SDP) multi-platform deployments.
Product image selection & overrides#
Operators let you choose which container image build to run per stacklet, and expose an overrides mechanism for advanced configuration beyond the CRD's typed fields — documented as something to use cautiously, since it bypasses the operator's normal validation.
Resource management#
CPU, memory and storage are configured per role group. Operators that need object storage also support S3 resource integration for things like backups or data lake access.
Service discovery and exposure#
Each stacklet gets a service discovery ConfigMap that other stacklets and operators read to find its connection details automatically — this is how, for example, a Kafka operator discovers a ZooKeeper stacklet without manual wiring. Separately, service exposition controls how a stacklet is reached from outside the cluster (ListenerClasses, covered per-operator).
Security, operations and observability#
Authentication, OPA-based authorization and TLS verification are documented per-concept. Operational topics — pod disruption budgets, placement strategies, graceful shutdown, logging, labels and telemetry — are covered in their own sections and apply the same way across operators.
Next#
See how these show up in a real operator: Apache Airflow and Apache Kafka.