Operators Development Tools

As it for today, we have three major operator development tools: Kubebuilder, Opeartor SDK and Metacontrooler. They make the life easier for developers to extend K8s using CRDs and custom controllers(sometimes called operators). This post compares all those three tools (inspired by Adrien Trouillaud in his post).

	Kubebuilder	Operator SDK	Metacontroller
Backed by	SIG API Machinery	CoreOS (Red Hat)	Google Cloud Platform
Architecture	Encapsulation	Encapsulation	Framework
Pros	Tests and docs scaffolding; Multiple resources and controllers in one project; Great documentation	Simple API; Part of the Operator Framework	Any language; Higher-level abstractions; JSON (dynamic for fast development); Declarative; Great documentation
Cons	Go only; Could use more abstractions	Go only; Single resource and controller in one project; Reference example doesn’t follow best practices	JSON (no static typing by default); Use case has to be compatible

1. The Kubernetes API can be extended with custom resources at runtime either via CustomResourceDefinitions (CRDs) or aggregated APIs. CRDs are the easiest, if your use case is compatible.

2. The behavior of custom resources (just like Kubernetes resources) is best implemented using the controller pattern, which is declarative, asynchronous, and level-based. client-go provides building blocks for controllers (cache and work queue). Those aren’t yet available in other languages’ client libraries.

3. The building blocks are put together in the workqueue example, sample-controller (for CRDs), and sample-apiserver (for aggregated APIs). The latter two can be forked or copied and modified with the help of code-generator.

4. Better tools have subsequently been released to make developers more productive (see table in TL;DR for a recap of pros and cons):

   • Kubebuilder (following apiserver-builder) facilitates project creation and code generation in Go, and encapsulates client-go’s APIs;

   • The Operator SDK also handles project creation and code generation in Go, butand encapsulates client-go’s APIs.

   • Metacontroller is a framework. It is a black box that runs separately and calls processing functions as webhooks written in any language, accepting and returning JSON.

5. The SIG API Machinery has taken upon itself to further improve the Kubernetes platform development experience. A first step is to extract and standardize the controller pattern. It could still be used in various ways in different tools, or a standard platform SDK may replace the current options; Kubebuilder, which is now owned by the SIG, is a likely contender.