Introduction to Hybrid and Multicloud Architecture
Dan Weston
Senior Curriculum Developer
Hybrid and Multicloud Architecture with Apache Kafka
About This Course
Most organizations today are a mix of cloud and on-premise systems. Many have come about piecemeal as cloud providers have been added as different departments move to the cloud, retain their on-premise legacy systems, or as new companies are acquired and their infrastructure is integrated. Streaming data provides a mechanism to join systems asynchronously so they can handle long distances, high latency, or unpredictable connections with relative ease. These two transitions, while relatively new, have the potential to be some of the most impactful and important shifts in the industry in a long time.
Customers expect our systems to work unilaterally - wherever the servers are located. Data should be accessible to everyone that needs it no matter where they or the systems are located. In this course, we’ll cover the things you need to be successful in building real-time multicloud and hybrid-cloud systems. Systems that span continents and clouds. There are three tools that can achieve this: the open-source MirrorMaker 2, Confluent Replicator, and Cluster Linking.
We'll talk about the benefits and considerations that need to be made with each one and then configure a local instance of MirrorMaker 2 and a cloud cluster using Cluster Linking.
Intended Audience
- You are familiar with the basic concepts of a publish/subscribe system such as:
- Topics
- Producers
- Consumers
- You may have never used Kafka
- Have a need or desire to replicate your data
Course Outline
- Introduction to Hybrid and Multicloud Architecture
- Setup Confluent Cloud - Exercise
- Understanding Hybrid and Multicloud Architectures
- Cluster Linking
- MirrorMaker 2
- Confluent Replicator
- Use Cases for Hybrid and Multicloud Architectures
- High Availability and Disaster Recovery with Cluster Linking
- Configuring Cluster Linking - Exercise
- Configuring MirrorMaker 2 - Exercise
- Configuring a Global Hybrid Architecture - Challenge
- Additional Resources
Prerequisites
- Required Setup
- A machine that can run a Docker container for the MirrorMaker 2 hands-on
Length
- Approximately 2 hours
Staff
Dan Weston (Course Author)
Dan Weston is a senior curriculum developer at Confluent and has been building technical training for over 15 years. He previously worked as a senior manager of production at LinkedIn Learning, focusing on their technology library. He enjoys spending time with his family, especially when it involves cooking delicious food.
- Previous
- Next
Use the promo code HYBRCLOUD101 & CONFLUENTDEV1 to get $25 of free Confluent Cloud usage and skip credit card entry.
Be the first to get updates and new content
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.
Introduction to Hybrid and Multicloud Architecture
Some of you might remember a world where your network architecture looked a little like this.
Not that it was just one round, perfectly shaped ball, but more like it was gathered in one location, planned, and shaped the way that worked for your company and customers.
Most likely your network architecture consisted of systems and services that lived in a data center onsite and didn't rely on external services.
When you needed to add a new database or service for a department it was relatively easy and you could do it in a way where everything was neatly tied together.
Then along came cloud services and we all added them because of how easy they are to scale, add additional remote locations where we don’t have to have physical space, and of course, the added features and functionality that would have taken our internal teams months or years to add.
But this created a new problem, as we added all these systems, getting them to talk to each other, and share data, has become more difficult.
So we end up with a system that looks like this.
To get data from one component to another we have to add steps and more complexity.
All of this adds overhead and slows things down.
This is where a hybrid and multi-cloud architecture comes in.
We know it is impossible to go back to something like this, and in reality, we don’t want to as it lacks many of the features and tools our business needs.
Instead, we need a way to tie our on-premise systems, disparate cloud providers, and multiple locations together.
Giving our architecture greater flexibility, supporting our compliance and local data regulations, and providing a manageable workload for both on-premise and cloud-based systems.
Ideally, tying everything together so it’s fast while maintaining the benefits of this.
This course will explore three options: Confluent Cluster Linking, the open-source MirrorMaker2, and Confluent Replicator.
With the goal of helping you understand your options and how to configure your systems to take advantage of the cloud while having you feel in control of a system more like this.
If you aren't already on Confluent Developer, head there now using the link in the video description to access the rest of this course, the hands-on exercises, and additional resources.