This blog post is based on a discussion with Carlos Santana from AWS, focusing on Kubernetes Resource Orchestrator (KRO), an open-source project designed to simplify and enhance platform engineering.
The promise of Kubernetes is simple: a platform for automating deployment, scaling, and managing containerized applications. Yet, the reality for many organizations, especially large enterprises, is far more complex. Developers often find themselves wrestling with intricate YAML configurations, cloud provider-specific services, and the underlying infrastructure, drawing their focus away from feature development. This complexity leads to operational overhead and inefficiencies, highlighting the need for higher-level abstractions. Kubernetes Resource Orchestrator (KRO) is an open-source project designed to address these challenges by simplifying the creation and management of Kubernetes APIs.
The Kubernetes Conundrum in Large Enterprises
In sprawling enterprise environments, the path to deploying an application can be fraught with disjointed processes. Consider a scenario where a developer needs to deploy an application to Amazon EKS with a database on Amazon RDS. The developer may have to navigate multiple teams and systems:
Submitting a request for a namespace through one ticketing system.
Coordinating the provisioning of a database with a separate infrastructure team.
Configuring network access and security by engaging with the networking and security teams.
Ensuring compliance with organizational policies, which may involve yet another set of tools and approvals.
This intricate web of dependencies and handoffs not only slows down the deployment process but also introduces potential bottlenecks and inconsistencies. Without a centralized platform, developers spend valuable time on infrastructure-related tasks rather than focusing on innovation and feature delivery.
KRO: Simplifying API Creation
KRO offers a streamlined approach to building Kubernetes APIs by allowing users to define schemas that translate into Custom Resource Definitions (CRDs). Unlike traditional methods that require extensive coding in Go and deep knowledge of Kubernetes internals, KRO simplifies the process with a user-friendly interface.
Example KRO Schema:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: webapps.stable.example.com
spec:
group: stable.example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
name:
type: string
image:
type: string
scope: Namespaced
This simplified schema automates the creation of CRDs, significantly reducing the manual effort required. By abstracting away the complexities of YAML configurations and Go programming, KRO empowers platform engineers to create APIs more efficiently.
Output Wiring and Common Expression Language (CEL)
A key feature of KRO is its ability to wire outputs from one resource to another using the Common Expression Language (CEL). This is particularly useful when dealing with cloud resources where certain parameters, such as ARNs or unique identifiers, are not known until the resource is created.
Example CEL Expression:
resource.status.bucketArn
In this example, the CEL expression extracts the bucketArn from the status of the S3 bucket resource and injects it into the IAM policy. This automated wiring eliminates manual steps, reduces the risk of errors, and ensures that resources are properly configured with the correct dependencies.
KRO vs. Other Tools
When it comes to creating APIs and managing Kubernetes resources, several tools are available, each with its strengths and weaknesses. KRO offers a unique balance of simplicity and power, making it a compelling choice for platform engineers.
Feature | KRO | Helm | Custom Controllers | Crossplane |
API Creation | Yes, simplified schema definition | No | Yes, but requires extensive coding | Yes, through composition functions |
Resource Ordering | Yes | Limited | Yes, full control through coding | Yes, through composition functions |
Output Wiring | Yes, using CEL | No | Yes, full control through coding | Yes, through composition functions |
Coding Required | Minimal | Templating | Extensive (Go, Kubernetes SDK) | Moderate to Extensive (Go, Python) |
Multi-Cloud Support | Yes, collaboration with cloud providers | Limited | Yes, but requires custom implementation | Yes, built-in, but can be complex |
Complexity | Moderate | Low | High | High |
Learning Curve | Moderate | Low | High | High |
As the table illustrates, KRO strikes a balance between the simplicity of Helm and the flexibility of custom controllers, while offering a more streamlined approach than Crossplane.
Use Case: Web Application Deployment
To illustrate the practical benefits of KRO, consider the use case of deploying a web application with an S3 bucket for storage.
First, define the KRO definition for creating an S3 bucket:
apiVersion: k8s.aws/v1alpha1
kind: S3Bucket
metadata:
name: my-bucket
spec:
name: "my-bucket-name"
region: "us-west-2"
Next, create an IAM policy that uses the bucket ARN:
apiVersion: iam.aws/v1alpha1
kind: IAMPolicy
metadata:
name: my-bucket-policy
spec:
policyDocument: |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::<KRO_S3Bucket_my-bucket.status.bucketArn>/*"
}
]
}
KRO automatically replaces the <KRO_S3Bucket_my-bucket.status.bucketArn> placeholder with the actual ARN of the created bucket, ensuring that the IAM policy is correctly configured.
Finally, create a pod identity association:
apiVersion: iam.aws/v1alpha1
kind: PodIdentityAssociation
metadata:
name: my-pod-identity
spec:
clusterName: test-cluster-32
namespace: default
podIdentityName: my-pod-identity
roleARN: arn:aws:iam::123456789012:role/my-role
serviceAccountName: my-service-account
By composing these resources using KRO, platform engineers can create a simplified API for developers to deploy web applications with S3 storage, without needing to understand the underlying complexities.
Community and Collaboration
KRO is an open-source project fostered by collaboration between AWS, Google, and Azure. To get involved and contribute to the project, consider joining the community:
Slack Channel: The link is posted in the chat during the live stream.
GitHub Repository: https://github.com/kro-on/kro
By participating in discussions, contributing code, and sharing use cases, you can help shape the future of KRO and Kubernetes abstractions.
Roadmap and Future Directions
The KRO team is actively working on new features and enhancements to further simplify Kubernetes abstractions. Some of the key roadmap items include:
Collections: Simplifying the creation of multiple resources, such as subnets or load balancers.
CLI and OCI Distribution: Enabling better versioning and sharing of KRO definitions.
External Configuration: Sharing an existing network, and the ability to put a cluster into an existing cluster and create a new cluster.
By contributing your ideas and feedback, you can help prioritize these features and ensure that KRO meets the evolving needs of the Kubernetes community.
Comments