Www.itsportsbetDocsCloud Computing
Related
10 Key Insights into Kubernetes v1.36’s Fine-Grained Kubelet AuthorizationKubernetes v1.36 Beta: Dynamic Resource Tuning for Suspended Jobs10 Key Insights: How Kubernetes Became the Backbone of AIAmazon ECS Managed Daemons Unlock Independent Agent Lifecycle for Infrastructure TeamsUnderstanding Ingress-NGINX Quirks: What You Need Before MigrationKubernetes v1.36 Memory QoS: Smarter Memory Protection for Your Pods5 Essential Facts About Microsoft's Sovereign Private Cloud and Azure Local ScalingMastering Amazon S3 Files: Transforming S3 Buckets into High-Performance File Systems

The Silent Upgrade: How Kubernetes Image Promotion Got a Modern Makeover

Last updated: 2026-05-01 14:36:04 · Cloud Computing

Every container image you pull from registry.k8s.io arrives thanks to kpromo, the Kubernetes image promoter. This tool copies images from staging registries to production, signs them with cosign, replicates signatures across over 20 regional mirrors, and generates SLSA provenance attestations. If kpromo breaks, no Kubernetes release ships. Recently, the team rewrote its core from scratch, deleted 20% of the codebase, made it dramatically faster, and nobody noticed. That was the goal.

A Brief History

The image promoter began in late 2018 as an internal Google project by Linus Arver. The aim was to replace manual, Googler-gated image copying into k8s.gcr.io with a community-owned GitOps workflow. Push to a staging registry, open a PR with a YAML manifest, get reviewed and merged, and automation handles the rest. KEP-1734 formalized this.

The Silent Upgrade: How Kubernetes Image Promotion Got a Modern Makeover

In early 2019, the code moved to kubernetes-sigs/k8s-container-image-promoter and grew quickly. Stephen Augustus consolidated multiple tools (cip, gh2gcs, krel promote-images, promobot-files) into a single CLI called kpromo. The repo was renamed to promo-tools. Adolfo García Veytia (Puerco) added cosign signing and SBOM support. Tyler Ferrara built vulnerability scanning. Carlos Panato maintained the project. 42 contributors made ~3,500 commits across 60+ releases. It worked, but by 2025 the codebase carried seven years of incremental additions. The README noted duplication, multiple techniques, and many TODOs.

Problems to Solve

Production promotion jobs for Kubernetes core images regularly took over 30 minutes and often failed with rate limit errors. The core promotion logic became a monolith hard to extend and test, making new features painful. On the SIG Release roadmap, two work items lingered: "Rewrite artifact promoter" and "Make artifact validation more robust." Discussions at meetings and KubeCons led to open research spikes on project board #171 capturing eight questions.

The Phased Rewrite

In February 2026, the team opened issue #1701 to rewrite the artifact promoter pipeline. All eight spikes were answered in a single tracking issue. The rewrite was phased so each step could be reviewed, merged, and validated independently.

Phase 1: Rate Limiting (#1702)

Rewrote rate limiting to properly throttle all registry operations with adaptive backoff.

Phase 2: Interfaces (#1704)

Put registry and auth operations behind clean interfaces for independent swapping and testing.

Phase 3: Pipeline Restructuring

The pipeline was restructured into composable stages, allowing easier extension for provenance and vulnerability scanning. This modular approach reduced coupling and simplified testing.

Conclusion: Invisible Improvements

The rewrite made kpromo faster, more reliable, and easier to maintain. The 20% code deletion and new architecture ensure future features can be added without breaking existing workflows. The team succeeded in making the improvements invisible to users — exactly as intended.