January 27, 2022 | Episode 197

Expanding Apache Kafka Multi-Tenancy for Cloud-Native Systems ft. Anna Povzner and Anastasia Vela

  • Transcript
  • Notes

In an effort to make Apache Kafka® cloud native, Anna Povzener (Principal Engineer, Confluent) and Anastasia Vela (Software Engineer I, Confluent) have been working to expand multi-tenancy to cloud-native systems with automated capacity planning and scaling in Confluent Cloud. They explain how cloud-native data systems are different from legacy databases and share the technical requirements needed to create multi-tenancy for managed Kafka as a service. 

As a distributed system, Kafka is designed to support multi-tenant systems by: 

  • Isolating data with authentication, authorization, and encryption
  • Isolating user namespaces
  • Isolating performance with quotas

Traditionally, Kafka’s multi-tenant capabilities are used in on-premises data centers to make data available and accessible across the company—a single company would run a multi-tenant Kafka cluster with all its workloads to stream data across organizations. 

Some processes behind setting up multi-tenant Kafka clusters are manual with the requirement to over-provision resources and capacity in order to protect the cluster from unplanned demand increases. When Kafka is on cloud instances, you have the ability to scale cloud resources on the fly for any unplanned workloads to meet expectations instantaneously. 

To shift multi-tenancy to the cloud, Anna and Anastasia identify the following as essential for the architectural design: 

  • Abstractions: requires minimal operational complexity of a cloud service
  • Pay-per-use model: requires the system to use only the minimum required resources until additional is necessary 
  • Uptime and performance SLA/SLO: requires support for unknown and unpredictable workloads with minimal operational workload while protecting the cluster from distributed denial-of-service (DDoS) attacks
  • Cost-efficiency: requires a lower cost of ownership 

You can also read more about the shift from on-premises to cloud-native, multi-tenant services in Anna and Anastasia’s publication on the Confluent blog. 

Continue Listening

Episode 198February 1, 2022 | 30 min

Intro to Event Sourcing with Apache Kafka ft. Anna McDonald

What is event sourcing, and how does it work? Event sourcing is often used interchangeably with event-driven architecture and event stream processing. However, Anna McDonald (Principal Customer Success Technical Architect, Confluent) explains it's a specific category of its own—an event streaming pattern.

Episode 199February 3, 2022 | 6 min

On to the Next Chapter ft. Tim Berglund

After nearly 200 podcast episodes of Streaming Audio, Tim Berglund bids farewell in his last episode as host of the show. Tim reflects on the many great memories with guests who have appeared on the segment—and each for its own reasons. He has covered a wide variety of topics, ranging from Apache Kafka fundamentals, microservices, event stream processing, use cases, to cloud-native Kafka, data mesh, and more.

Episode 200February 16, 2022 | 2 min

What’s Next for the Streaming Audio Podcast ft. Kris Jenkins

Meet your new host of the Streaming Audio podcast: Kris Jenkins (Senior Developer Advocate, Confluent)! In this preview, Kris shares a few highlights from forthcoming episodes to look forward to, spanning topics from data mesh, cloud-native technologies, and serverless Apache Kafka, to data modeling.

Got questions?

If there's something you want to know about Apache Kafka, Confluent or event streaming, please send us an email with your question and we'll hope to answer it on the next episode of Ask Confluent.

Email Us

Never miss an episode!

Confluent Cloud is a fully managed Apache Kafka service available on all three major clouds. Try it for free today.

Try it for free