Get Started Free
course: Apache Kafka® for Python Developers

Hands On: Setup the Exercise Environment for Confluent Cloud and Python


Dave Klein

Developer Advocate

Confluent Cloud Setup

Throughout this course, we’ll introduce you to developing Apache Kafka event streaming apps with Python through hands-on exercises that will have you produce data to and consume data from Confluent Cloud. This exercise will get you set up for the exercises that follow.

Sign up for Confluent Cloud

Note: If you already have a Confluent Cloud account, you can skip this section and proceed to step 8.

If you haven’t already signed up for Confluent Cloud, sign up now so when your first exercise asks you to log in, you are ready to do so.

  1. Browse to the sign-up page:
  2. Fill in your contact information and a password. Be sure to remember these details as you will need them to log in to Confluent Cloud.
  3. Click the Start Free button and wait for a verification email.
  4. The link in the email will lead you to a screen where you will be prompted to complete a short survey. After doing so, click Continue or alternatively just click Skip.
  5. When the information dialog appears that informs you of the promotional credit you can use to become familiar with Confluent Cloud, click Get started to continue with these set up steps.
  6. When the next screen appears that prompts you to set up your first cluster, click I'll do it later.
  7. Continue to step 9.

Login to the Confluent Cloud Console

  1. Open URL and log in to the Confluent Cloud console.
  2. Navigate to the environments page and click Add cloud environment.
  3. Name the new environment learn-kafka-python and click Create.

Enable Streams Governance

Next, you will be prompted to enable one of the available Streams Governance Packages. You need to do so since the course exercises utilize Schema Registry which these packages include.

  1. Click Begin configuration for the Essentials package.
  2. Select a provider and region and click Enable.

Create a Cluster in the learn-kafka-python Environment

  1. Click Create cluster on my own.
  2. Click Begin configuration for the Basic cluster.
  3. Select a provider and region and click Continue.
  4. Assign a name of kafka-python and click Launch cluster.

Create and Download Python Client Configuration Properties

Next, you will create the configuration properties needed to connect to your cluster.

  1. Navigate to the Cluster Overview Dashboard and for the kafka-python cluster.

  2. Click on the Python tile in the Connect to your systems section.

  3. Create a cluster API key and secret by clicking Create Kafka cluster API key.

  4. Review the key and the secret in the pop-up and click Download and continue.

    This will download the key and secret to the local machine Downloads directory.

  5. To create the Schema Registry API key and secret, repeat the process by clicking Create Schema Registry API key.

  6. Review the sample Python client configuration settings. The API keys and secrets should be populated in the properties in the middle of the page. If they are not visible, select the Show API keys option.

  7. Click the Copy button.

Setup Project

Create Python Dictionary

  1. Create a file named located in the ~/.confluent directory.

    Note: The .confluent directory is the default location for Confluent Platform related configuration files.

  2. Edit and paste the previously copied Python client Confluent Cloud properties.

  3. Save but leave it open in your IDE.

  4. Create a config directory named kafka-python for the course exercises and cd into it.

  5. Create a file named that will contain a Python dictionary.

  6. Add the following lines to

    config = {
         'bootstrap.servers': '<bootstrap-server-endpoint>',     
         'security.protocol': 'SASL_SSL',
         'sasl.mechanisms': 'PLAIN',
         'sasl.username': '<CLUSTER_API_KEY>', 
         'sasl.password': '<CLUSTER_API_SECRET>'}
  7. Update the values for bootstrap.servers, sasl.username, and sasl.password to the values contained in

    We’ll add more to in later exercises.

The course exercises depend upon Python 3.X and several related packages being installed. The steps in this prerequisites section satisfy this dependency. Complete the steps as needed on your local machine.

Note: These steps correspond to installing these packages on Ubuntu 20.04. Adjust the steps as needed to suit your local environment.

  1. First update all the packages in your local machine:

    sudo apt update
  2. If any of the system packages on your local machine need to be upgraded, run the following command to do so:

    sudo apt upgrade
  3. Install python3:

    sudo apt install python3
  4. Install pip3:

    sudo apt install python3-pip
  5. Identify default python version:

    python --version

    The version shown in the command response should be similar to Python 3.X.X. If a non-Python 3.X version is returned, you will need to either update your alias for the python command or explicitly run the python3 and pip3 commands during the exercises for this course.

  6. Install virtualenv:

    pip install virtualenv

Create a New Virtual Environment with virtualenv

  1. Create a virtual environment:

    virtualenv kafka-env
  2. Activate the virtual environment:

    source kafka-env/bin/activate

Install Confluent Kafka

  1. Run the following command:

    pip install confluent-kafka

You’re now ready for the upcoming exercises.

Use the promo code PYTHONKAFKA101 to get $25 of free Confluent Cloud usage

Be the first to get updates and new content

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.