Microsoft’s Azure Kubernetes Application Network launched at KubeCon EU 2026 in Amsterdam with a deliberate strategic choice embedded in its name: the words “service mesh” appear nowhere in it. The product is built entirely on Istio’s ambient mode, one of the most significant service mesh architectures available, and its lead architect, Microsoft principal software engineer Mitch Connors, defined success for the project in a single sentence: “Most people not knowing what a service mesh is, even though they’re using one.“
That goal statement is not modest. It is a direct response to a decade of self-inflicted complexity that drove 60% of Kubernetes clusters away from service mesh adoption entirely.
Why Service Mesh Earned Its Reputation for Being Complicated
Service mesh complexity originates directly from the sidecar model, the first-generation architecture that injected a proxy container into every application pod running in a Kubernetes cluster. Sidecar-based service meshes provided granular security and traffic control but extracted a severe operational price. Every mesh upgrade required restarting every application across the cluster. Platform teams became de facto proxy management specialists.
CNCF Annual Survey data confirmed the damage: sidecar-based mesh adoption dropped from 50% of organizations to 42%, declining precisely as cloud-native adoption was accelerating.
Organizations did not stop wanting zero-trust security. They stopped wanting to pay the operational cost that the sidecar model charged for it.
What Istio Ambient Mode Actually Changed
Istio ambient mode is a structural architectural shift, not a feature update. Ambient mode moves encryption functions into a lightweight per-node Rust proxy operating at the infrastructure layer, entirely separate from application pods. Layer 7 traffic management features shift into independently upgradeable waypoint proxies. The result eliminates the defining pain point of the sidecar era: applications no longer need to restart when the mesh upgrades.
The problem Connors identified at KubeCon is that ambient mode alone has not solved the operational discipline gap. Approximately 85% of ambient mode installations are not keeping current with CVE security patches, leaving known vulnerabilities unaddressed across the majority of deployments. Azure Kubernetes Application Network addresses this directly by delivering ambient mode as a fully managed service with mTLS encryption enabled by default across all clusters, removing the patching responsibility from platform teams entirely.
The AI Workload Problem Service Mesh Was Not Built For
Traditional HTTP routing operates on a foundational assumption that AI workloads invalidate completely: that every request costs roughly the same to serve. Connors identified the gap at KubeCon directly. An LLM receiving a one-word greeting and an LLM generating a detailed technical explanation represent orders-of-magnitude differences in compute cost, response time, and token consumption. Standard Kubernetes routing treats both identically.
Azure Kubernetes Application Network addresses AI traffic through 2 mechanisms. The Gateway API inference extension runs a lightweight LLM as a token complexity estimator, scoring incoming requests before routing. Token usage in response payloads feeds rate limiting that Istio distributes across the cluster automatically.
Azure Kubernetes Application Network addresses AI traffic through 2 mechanisms.
1. Inference-Aware Request Routing: The Gateway API inference extension runs a lightweight LLM as a token complexity estimator, scoring incoming requests before routing. Token usage in response payloads feeds rate limiting that Istio distributes across the cluster automatically, ensuring compute-heavy LLM requests receive appropriate routing treatment rather than being processed identically to lightweight queries.
2. Agent Gateway Integration for Agentic Traffic: Agent Gateway, a Linux Foundation project designed specifically for agentic traffic, including MCP and A2A protocols, integrates as the bleeding-edge layer for teams building on AI agent infrastructure. Anthropic’s explicit acknowledgment applies here: these APIs carry alpha-level stability commitments, meaning teams adopting this layer are working with protocols that may change shape within 2 years.
Anthropic’s own enterprise AI deployment strategy reflects the same governance-first philosophy Microsoft is applying to App Net. Claude Cowork’s general availability launch was built around organizational access controls, audit infrastructure, and identity provider integration, specifically because enterprise IT approval requires governance architecture before capability claims.
The AI governance dimension adds a third requirement: enforcement. Platform teams can designate approved LLM endpoints for organizational use, but network-layer enforcement of those designations requires the mesh to inspect request bodies and validate routing destinations. App Net ships with that capability built in.
Conclusion
Invisible infrastructure is the only infrastructure that gets adopted.
Microsoft’s framing of Azure Kubernetes Application Network as a product that never says “service mesh” is the correct commercial decision and the correct technical philosophy simultaneously. The service mesh category earned its complexity reputation honestly. The sidecar model was genuinely difficult to operate at scale, and organizations that walked away 5 years ago made a rational decision with the tooling available at the time.
Ambient mode changes the underlying architecture. Managed delivery removes the operational burden. mTLS by default eliminates the configuration step that most organizations skipped. The result is a security infrastructure layer that platform teams can approve without becoming Istio specialists, which is precisely the condition under which the 60% of Kubernetes clusters currently running with no mesh will finally adopt one.
The infrastructure layer these clusters run on is itself expanding at a pace that makes mesh adoption urgency concrete. AI data center construction is accelerating globally, and the workload density, token throughput, and east-west traffic volumes that these facilities generate are precisely the conditions under which unmanaged Kubernetes clusters without zero-trust encryption create the most concentrated security exposure.
Good infrastructure disappears. Microsoft is betting the Azure Kubernetes Application Network succeeds by disappearing completely.
Cloud infrastructure, Kubernetes architecture, and the network-layer changes that AI workloads are forcing on enterprise platforms are covered at The IT Horizon. Subscribe to our newsletter. We translate the infrastructure decisions happening at KubeCon and beyond into what they actually mean for your technology strategy.
