Announcing Backyards 1.3

Today we've launched the 1.3 release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud's production ready Istio distribution. Along with some performance improvements and bug fixes, the 1.3 release is centered around three main topics:

• a brand new gateway management feature,
• a new declarative installation and configuration method,
• and support for Istio 1.6.

If you're not familiar with Backyards (now Cisco Service Mesh Manager), and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

---

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we've already blogged about.

---

tl;dr

Now lets see some of the new significant features in Backyards 1.3:

• Lightweight API gateway management purely based on Istio resources
• A new UI dashboard for managing and displaying gateway port and host configurations
• Configuration of routing and traffic management rules like timeouts, retries, or fault injection for upstream services
• Configuration of TLS settings through Automated Certificate Management Environments (ACME), like Let’s Encrypt
• Automatic banzaicloud.io domain names for Istio gateways
• Out-of-the box monitoring for Istio gateways
• The Backyards operator: a declarative installation and configuration method for Backyards components
• Backyards is keeping up with the fast moving Istio project and already supports Istio 1.6 that was released on May 21, 2020
• Stability and performance fixes

What's new

Backyards (now Cisco Service Mesh Manager) is moving to a release schedule that closely follows Istio's quarterly releases. We're planning to support new Istio releases in our distribution as soon as they come out, and as usual for our customers, with seamless upgrades. The 1.3 release is the first that is aligned with the upstream release cycle, as it already supports Istio 1.6 that was released a few days back. We're also adding new features on top of the Istio release: the main highlight of this release is a brand new gateway management feature and dashboard, that is the first step towards a fully Istio-based, lightweight API Gateway management platform. The second main item is the addition of the Backyards operator, a new, declarative way of installing and configuring all of the managed components. Now let's dig deeper, and see these things in detail.

Gateway management

The Backyards (now Cisco Service Mesh Manager) dashboard already covers the most important Istio observability and traffic management features for east-west traffic in the mesh, but so far it was missing the representation of north-south traffic, in other words gateway management and observability. The 1.3 release adds a new page to the dashboard, called Gateways, which:

• lists all the Istio gateways in the cluster,
• adds out-of-the-box monitoring for them, and
• provides the ability to configure ports, hosts, certificates, routes and other traffic management features for incoming traffic.

You can think of it as a lightweight API gateway management UI, built purely on Istio primitives. It doesn't bring convenience features like JWT authentication or rate limiting for now, but with the help of Envoy WASM extensions, it remains fully customizable, and we're already working on some of these features to be included in the near future. If you want to avoid having yet another product along Istio to handle north-south traffic in your cluster, but are afraid of the complexity and the lack of a management and overview dashboard in Istio, Backyards (now Cisco Service Mesh Manager) is a great fit. Here comes a highlight of the dashboard capabilities.

Monitoring of upstream traffic

Backyards collects upstream metrics like latencies, throughput, RPS, or error rate from Prometheus, and provides a summary for each gateway. It also sets up a Grafana dashboard and displays appropriate charts in-place.

Managing port and host configurations

Backyards (now Cisco Service Mesh Manager) understands Istio's Gateway CRs and the gateway's service configuration in Kubernetes (with the help of the MeshGateway CR), so it can display information about ports, hosts and protocols that are configured on a specific gateway. If you'd like, you can also set up a new entry point, and Backyards (now Cisco Service Mesh Manager) will translate your configuration to declarative custom resources.

Note: the open source Banzai Cloud Istio operator has a concept called MeshGateway, a declarative representation of Istio ingress and egress gateway services and deployments. With the help of MeshGateways, it's easy to set up multiple gateways in a cluster used for different purposes.

Note: the Backyards UI has a refreshed YAML viewer. To display MeshGateways, Gateways, and VirtualServices, just click the icon next to their name:

Routing and traffic management

One of the main reasons to use Istio gateways instead of native Kubernetes ingress is to configure the routing of incoming traffic just like in-mesh, using VirtualServices. Istio concepts like redirects, rewrites, timeouts, retries, or fault injection can be applied to incoming requests. Backyards (now Cisco Service Mesh Manager) displays routes and their related configuration on the gateway management page, and gives you the ability to configure routing. As with port configurations, it translates the inputs to Istio CRs (mainly VirtualServices), then validates and applies them to the cluster.

TLS configuration

When setting up a service on a gateway with TLS, you need to configure a certificate for the host(s). You can do that by bringing your own certificate, putting it down in a Kubernetes secret, and configuring it for a gateway server. This works for simple use cases, but involves lots of manual steps when obtaining or renewing a certificate. Automated Certificate Management Environments (ACME) automates these kinds of interactions with the certificate provider. ACME is most widely used with Let's Encrypt and - when in a Kubernetes environment - cert-manager. Backyards (now Cisco Service Mesh Manager) helps you set up cert-manager, and you can quickly obtain a valid Let's Encrypt certificate through the dashboard with a few clicks - even with a banzaicloud.io domain automatically if you'd like!

Note: Backyards (now Cisco Service Mesh Manager)uses and extensively automates many certificate management features. There is a community requirement toSupport VirtualService resources for HTTP01 solving for better Istio supportin cert-manager, and based on our work done in this new Backyards release we will be pushing it upstream.

Istio 1.6 support

While Istio's 1.6 release didn't bring that many architectural changes as 1.5, it continued on that path. It further simplified architecture and operational experience. Many people, including us have blogged about istiod a lot, and how Istio moved towards a more monolithic approach. It shouldn't be a surprise now that Istio continues on that path, and has removed legacy control plane components completely (Citadel, Galley, Pilot, Sidecar Injector). Maybe the speed of that transition is a bit of a surprise, as those components were completely killed off in two minor releases.

As an [Istio distribution] vendor and someone who’s been running production Istio clusters with seamless version upgrades, it’s been definitely a challenge to follow up the changes (new and renovated CRs, complete architectural shifts, etc.) in the 1.5 and 1.6 releases. The good news is that we have done this with Backyards (now Cisco Service Mesh Manager) without breaking SLOs.

When Telemetry V2 was first released, it was lacking behind Telemetry V1 feature-wise. While it may still be the case, the gap is closing. The Istio community put a lot of effort in enhancing telemetry V2 in this release. There is now better support for customizing metrics, a new experimental feature called request classification filters was added, and a bunch of smaller issues were fixed or enhanced. You can read the full set of changes in the announcement. WebAssembly plugins for Envoy also continue to gain traction. A range of blog posts and community projects emerged in the last few months. We wrote about [Envoy WASM filters)]({{< relref "/blog/envoy-wasm-filter.md" >}}) a while ago, and how we use them to enable native [Kubernetes RBAC for Kafka]({{< relref "/blog/kafka-acl-with-istio-mtls.md" >}}). Another high impact change is the removal of Security alpha API. It included custom resources like Policy that was used to manage mTLS settings in the mesh, so you may need to migrate your resources to the new beta API and PeerAuthentication CRs. Backyards (now Cisco Service Mesh Manager) 1.3 comes with Istio 1.6 as the default installation option. Because Backyards doesn't add an abstraction layer on top of Istio, it's kept up-to-date with the above changes, and the new features are all handled properly. To learn more about the new Istio release, read more in our own [blog post]({{< relref "/blog/istio-1.6/index.md" >}}), or in the official announcement.

The Backyards operator

As many of our Banzai Cloud products, Backyards (now Cisco Service Mesh Manager) is moving towards a declarative installation. You can read more about it in our Design choices for a declarative installer blog post, but in a nutshell all these projects can act as a CLI tool, a local reconciler, and an operator at the same time, built in a single binary. When using the default quickstart method of installing Backyards, this change should be transparent. The operator is not installed on the cluster, but the controller code runs from the CLI on the client side.

Stability and performance fixes

Here are some of the most noteworthy bug fixes and enhancements:

• The code behind the topology view that builds up the graph representation of the services was completely rewritten to handle edge cases better and to improve performance with larger number of services
• The Istio configuration validator now handles gateway configurations better
• The topology view displays service entries properly even with Telemetry V2
• Fixed VirtualService filtering for Kubernetes services
• Fixed app label handling on the topology view
• Fixed a caching issue in the Backyards dashboard that caused stale information to be presented in some cases

Wrap-up

The Istio project is moving quicker than ever in the last few months, with a clear focus of simplifying the project architecturally and in terms of usability. The goal is to help the adoption of the service mesh, and we think that Backyards (now Cisco Service Mesh Manager) can greatly benefit from that. When we started to build this project over a year ago, our goal was something similar: to make work with Istio smooth and simple. That's why we've created an Istio operator and a management dashboard: to help you understand and manage Istio. We believe that we have already achieved that, and now we can finally shift to more high-level use cases. The gateway management feature we've introduced today is a good example for that, but expect even more exciting things in the next releases, so stay tuned!

---

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we've already blogged about.

---