EnvoyCon North America 2021

Modern apps are increasingly relying on using external 3rd party services (such as Twilio for e.g) and shared cloud services (such as S3 for e.g.). External interactions are important not only for security but for the app's continuity and resiliency as well. The use of Envoy as an egress proxy for external interactions has been limited though, mostly as the external interactions are TLS protected and Envoy is not able to decrypt the external TLS sessions. This session demos a solution that enables Envoy as an egress proxy for external access. It builds upon a combined approach of deploying Envoy as a transparent egress sidecar proxy along with the SSLproxy (github.com/sonertari/SSLproxy). In this approach, SSLproxy acts as a transparent TLS interception proxy and Envoy provides traffic management & security on the decrypted traffic. We dive into the traffic stitching mechanism and a new Envoy listener filter that acts as the glue between Envoy and SSLproxy and extends Envoy for the integrated solution.

The Emissary-ingress (née Ambassador API Gateway) project got its start in 2017, with the stated goal of making it easy for developers to harness the power and flexibility of Envoy, in Kubernetes, without needing to be experts on either Envoy or Kubernetes. Realizing that goal has been quite a journey: to bring Envoy to non-experts, the Emissary-ingress team has needed to learn an enormous amount about how Envoy, from the basics of how it works and how to configure it to best effect, to how to build it, test it, extend it, and debug it. The experience has been by turns challenging, surprising, frustrating, delightful, and ultimately extremely rewarding: after four years, Emissary-ingress is a CNCF project running in thousands of installations around the world, bringing Envoy into production use for organizations that wouldn't otherwise be able to take advantage of it. Join Flynn from Ambassador Labs to dive further into the challenges we've seen, the many lessons we've learned along the way, and the things we think anyone working with Envoy should know.

Health checks are an essential part of distributed systems of today that allow one to operate services in a reliable manner. Without health checks, operating a large latency-sensitive system becomes impossible. However, as the composition of environments differ, it can become a real burden to support health checks. Additionally, as the system scales, users tend to face the dreaded N-square problem, and then hit a tipping point, and traditional solutions don't seem to work. In this talk, we will discuss the benefits of health checks in Envoy, some problems we have encountered at scale at Stripe, and options to mitigate such issues.

Running multitenant Kubernetes clusters is challenging, particularly when different tenants require their own dedicated infrastructure. At Adobe Experience Manager Cloud Service we built solutions to provide customers with their own dedicated infrastructure, such as ips, DNS, VPN connectivity,... while running most services in multitenant Kubernetes clusters. We make extensive use of Envoy to run networking tunnels between Kubernetes pods and customer dedicated infrastructure, enforcing encryption and mutual authentication using certificates. This allows, for example, different pods to have their own dedicated egress ip instead of the cluster's, or connections from pods to multiple customer on-premise services using VPN. The solution is provisioned automatically using Terraform, Kubernetes operators and other services. We have previously shown the architecture we built to run this service, and we will provide updates on what worked well, what didn't and the lessons we learned running it in production.

With HTTP/3 support going alpha in the Envoy 1.19.0 release, the team that launched HTTP/3 is excited to explain what HTTP/3 is, and talk about Envoy’s HTTP/3 integration: what is supported today, what is upcoming, and what best practices there are for testing and deploying HTTP/3 for your own Envoy instances, client side or server side.

Envoy has a number of different load shedding mechanisms available out-of-the-box. Circuit breaking, rate limiting, admission control, adaptive concurrency-- how do they work? When should you use them? How can they fail spectacularly? Let's find out!

This one's a deep-dive into the different load shedding mechanisms available in Envoy. Come watch Tony gush about how each one is implemented, the queuing/scheduling concepts behind them, and how they can fail to protect us from noisy-neighbor problems. He'll conclude with some experimental results based on his previous work on FBThrift and how we (as a community) can apply those ideas to improve Envoy.

If you’ve ever wondered what WebAssembly (Wasm) is and how it works with Envoy, this session is for you. At the heart of Envoy lies a variety of filters that provide features such as network routing, observability, and security. Did you know that you can also write your own filters to extend Envoy functionality? In this session, you will learn about Envoy extensibility and the details of extending Envoy with Wasm.

Tinder adopted an Envoy based service mesh for their Kubernetes based infrastructure a few years ago. Our implementation is custom to Tinder's architecture but the lessons we picked up along the way are universal. Whether it's how we store and represent our routing configuration, building out a robust monitoring pipeline for all our Envoy metrics, utilizing Envoy's global rate limiting capabilities to protect our infrastructure, meshing our database and microservice requests, retry and timeout do’s and don’ts, and more, there is plenty of knowledge to share. We will explore our bespoke features like deadline propagation, multi-cluster Envoy metric monitoring, and how we implemented our own xDS control plane. From the early struggles of our service mesh migration, to cautionary tales and best practices, we're excited to walk through the process of building out an Envoy topology that powers Tinder's infrastructure worldwide.

Each day, Twitter responds to hundreds of billions of requests from users around the world. Today, Envoy is the point of entry for 100% of these requests. This hasn't always been the case. Not long ago, requests to Twitter passed through an ancient and highly bespoke edge proxy that was created internally and predated the open-source Envoy. In this talk, Ryland will describe how Twitter runs Envoy at the edge at scale, some of the unique benefits that Envoy provides in Twitter's edge architecture, and highlight features that Twitter has contributed to Envoy to support the edge use-case. He will outline how Twitter's edge architecture has evolved over time as the number of users and services has grown, what the next steps are for Twitter's edge, and the role Envoy will play in the future.

xDS is the way config is distributed to Envoy. Most of the ways the xDS API is implemented today (and in Istio) is through the state-of-the-world design. If one out of a thousand clusters changes and Envoy needs to know about the change, most control planes (including Istio) will send all of the thousand clusters to reflect the configuration change. On top of that, every configuration type is converged into one stream with Istio (ADS), which does not do the network any favors. This is the “quick and dirty” way, when logically, there should be no reason to send configuration when it hasn’t changed. This is what the delta xDS API aims to solve. Delta (or incremental) xDS is a variant of ADS/xDS, which has a different interface. If one configuration changes, that is the only configuration that will be sent. Istio is having quite a journey implementing delta xDS, sending only the “deltas” in configuration changes. Especially in a service mesh like Istio, which is the largest and most in-use mesh, there are quite a few caveats that need to be covered. This talk will be about the journey of Istio in implementing delta xDS, along with the expected benefits & apparent struggles that we had, along with guidance for future implementers of this amazing API.