Resource APIs are the core of working with Kubernetes; they provide a simple declarative model that allows you to build complex systems. Using custom resources, you can develop your own APIs to provide new abstractions and automation for reducing complexity and toil.
Kubernetes has been a leading force in the container revolution, in no small part because of its simple, declarative API architecture.
There is a trend toward developing custom resource APIs outside of the Kubernetes project. People develop these APIs as extensions they can install in Kubernetes clusters, resulting in APIs that look and feel just like the built-in types. For example, you could create a Guestbook resource type, allowing users to declare Guestbooks directly as Kubernetes objects, rather than declaring deployments and services that implement a Guestbook.
Unlike traditional APIs, which are modeled as imperative operations, Kubernetes APIs are structured as requests on objects that contain some desired state that you declare. These objects are called resources and are the foundation for working with Kubernetes (e.g. deployment, service, and ConfigMap.)
Here's why Kubernetes resource APIs are special, and how your team can go under the hood to leverage them.
Why invest in resource APIs?
What’s driving this trend? There are many things. One big factor is that you can simplify the processes and tooling used to manage applications and containerized workloads. Resource APIs reduce complexity in two ways: by providing abstractions, and by building complex or routine management tasks directly into the API.
Abstractions
Resource APIs provide simple abstractions for building complex applications through the composition and configuration of lower-level resources.
For example, they may:
Create collections of resources
Wire resources together
Configure lower-level resources—containers, init-containers, sidecar-containers, health/readiness checks, labels/selectors, etc.
Rather than declaring an application as a collection of deployments, services, ConfigMaps, etc., users might declare their application a single Guestbook object. Additionally, Guestbook API authors may enforce opinions and best practices as to how to configure lower-level resources by choosing what to expose in the Guestbook API.
Management
Resource APIs can encapsulate complex or routine management tasks by building them into the API.
Automated management can perform operations such as scheduled tasks (e.g., backups); respond to user changes such as manual scaling or CPU resource tuning; do self-healing when, say, a replica fails; perform self-tuning when processes are CPU-throttled; and respond to external events such as when a new release becomes available.
Notably, when management logic is built directly into the API, no external tooling or orchestration processes are necessary for management.
What types of projects should you develop as resource APIs?
There are many types of solutions being developed as Kubernetes resource APIs. Here are just a few categories.
Operators
These are APIs for specific applications. They encode human operational knowledge for configuring and managing that application into the API itself—automating tasks and reducing the need for orchestration tools.
Examples of operators include the Spark Operator and the Airflow Operator.
Operators may provide cloud-native capabilities for non-cloud-native applications by embedding the cloud-native logic in the Kubernetes API.
Decorators
Rather than managing collections of resources, these APIs add functionality to other APIs. Decorators provide generalized common logic that you can apply to many different APIs.
Examples of decorators include the Horizontal Pod Autoscaler and the Vertical Pod Autoscaler.
New container-based APIs
New container-based abstractions are being built where the primary logic is built directly into the API, rather than into a separate, "wrapped" application.
Examples include Knative for serverless and Tekton for pipelines.
These APIs target specific problems and provide high-level, cloud-native abstractions for running containers.
The Kubernetes resource API architecture
How are API extensions developed? There are a few atomic building blocks you can put together to build APIs, including:
Resources: Declare the API and define concerns such as the API endpoints, schema, versions, etc.
Controllers: Actuate the API and implement level-based, asynchronous reconciliation loops.
Webhooks: Admit mutation requests and perform validation, defaulting and conversion.
Resources: State definition
Resources store the desired state declared by a user, as well as observed status. You can implement them as built-in resources (deployments, services) or as user-defined custom resource definitions.
Resources have a standard schema and endpoint structure, which may be discovered using tools through the Kubernetes Discovery API and OpenAPI. This allows commands such as kubectl apply and kubectl get to "just work" regardless of how the resource is implemented.
Resources typically have the following structure:
Here's how those structural elements break down.
Metadata:
Resource: name, namespace, labels, annotations
API: kind, version, group
Spec:
Desired state written by users and the control plane, actuated by controllers
Examples: replicas, pod template
Status:
Observed state published by controllers, read by users
Examples: readyReplicas, availableReplicas
Because resources provide only storage for structured data, they do not contain any business logic or actuation. Instead, such functionality is implemented by a loosely coupled component: controllers.
Controllers: Implementation
Controllers are reconciliation loops that watch for changes to a resource (data), and then make changes to the cluster or external state in response. Controllers are:
Asynchronous: Reconciliation happens after the create/update/delete resource operation completes.
Level-based: While reconciliations are triggered by events, upon execution they are reading current state rather than looking at the contents of the event (for instance, they should be triggerable by a cron task, not just events).
Most controllers are designed to control only a single resource (for example, deployment), but generate (create) other resources that they own. For example, deployments generate ReplicaSets.
Controllers make changes based on the resource spec and communicate back to the user through the resource status.
Webhooks: Admission
Webhooks provide custom admission logic for resources and may be used to perform validation, defaulting, or converting resources between API versions (e.g., a v1beta1 object to a v1 object).
Webhooks may be used independently of resources and controllers. This lets administrators enforce advanced policy, set domain-specific defaults (e.g., "all pods should have the standard sidecar for logging"), or provide similar custom logic.
Tools for developing custom APIs
Writing a well-structured and -designed Kubernetes resource API can be challenging. Fortunately, if you are comfortable with the Go programming language, several libraries and tools are available to make the task easier.
Libraries: Controller-runtime
Controller-runtime is a set of libraries that provide high-level abstractions and simple defaults for writing Kubernetes controllers. They are written and maintained by Kubernetes project maintainers, and are published as a Kubernetes sub-project.
Frameworks: Kubebuilder and Operator SDK
Kubebuilder and Operator SDK are both frameworks, built on top of the controller-runtime libraries, that reduce the toil of building APIs. The two frameworks share most of their opinions, and continue to converge with one another. The most notable differences between them are that Operator SDK has a stronger operator focus and integrates with other operator-framework projects, where Kubebuilder's focus is more generalized for supporting Kubernetes subprojects.
Build cloud-native abstractions that just work
Resource APIs provide a way to build simple, cloud-native abstractions that drop into existing Kubernetes workflows and just work with Kubernetes tooling. You can develop your own resource APIs to reduce complexity and toil by introducing new abstractions and automating common tasks. Head over to this free e-book to get started building the APIs that you always wished you could use.
Come to Phillip Wittrock’s talk, "Kubernetes APIs under the hood," at the O'Reilly Velocity Conference on June 12 to get a close look at the mechanics of resource APIs and everything you need to know to build one yourself.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
