Custom API Server

Introduction

Custom API servers are processes serving API groups, usually built using the generic API server library k8s.io/apiserver . These processes can run inside or outside of the cluster. In the former case, they run inside pods, with a service in front. By implementing Custom API Server, some of the limits of CRDs could be addressed.

Limits of CRDs

What can Custom API Server do

Architecture

Components

kube-aggregator

All requests from kubectl or API clients will reach the kube-apiserver first, and then get proxied by kube-aggregator(inside kube-apiserver process) to the Custom API Server.

Workflow

1. Requests are received by the Kubernetes API server.

2. They pass the handler chain consisting of authentication, audit logging, impersonation, max-in-flight throttling, authorization, and more (the figure is just a sketch and is not complete).

3. As the Kubernetes API server knows the aggregated APIs, it can intercept requests to the HTTP path /apis/ aggregated-API-group-name.

4. The Kubernetes API server forwards the request to the custom API server.

Inner structure of a Custom API Server

• Has the same basic internal structure as the Kubernetes API server

• Has its own handler chain, including authentication, audit, impersonation, max-in-flight throttling, and authorization.

• Has its own resource handler pipeline, including decoding, conversion, admission, REST mapping, and encoding.

• Calls admission webhooks.

• Might write to etcd (it can use a different storage backend, though). The etcd cluster does not have to be the same as the one used by the Kubernetes API server.

• Has its own scheme and registry implementation for custom API groups. The registry implementation might differ and be customized to any degree.

• Does authentication again. It usually does client certificate authentication and token-based authentication, calling back to the Kubernetes API server with a TokenAccessReview request.

• Does its own auditing. This means the Kubernetes API server audits certain fields, but only on the meta level. Object-level auditing is done in the aggregated custom API server.

• Does its own authentication using SubjectAccessReview requests to the Kubernetes API server.