Get Started Free
dan-weston

Dan Weston

Senior Curriculum Developer

Kafka Authentication Basics

One way that Kafka provides security is through built-in authentication. Similar to checking an ID, authentication is the act of verifying the identity of an entity requesting access to a system component. In a Kafka-based system, there are many different interactions that begin with participants authenticating the components with which they are communicating.

For example, when a connection is established between a client (a user, application, or service) and a broker, each side of the connection will usually wish to verify the other. The same holds true when two brokers connect—each may verify the other. A final authentication scenario is a broker accessing ZooKeeper, whereby the broker may be required to authenticate before being allowed to access sensitive cluster metadata.

kafka-authentication

KafkaPrincipal

Internally in Kafka, a client's identity is represented using a KafkaPrincipal object, or principal. So, for example, if you connect to Kafka and authenticate with a username and password, the principal associated with the connection will represent your username.

Your principal is assigned your authorizations in the target system, as we will learn in the Authorization module, and it is also used to log details of any permissible operation you perform—as we will learn in the Audit Logs module.

kafkaprincipal

Note that all requests are assigned a principal, even if authentication hasn't been enabled for a connection. In this case, the principal associated with the connection would be ANONYMOUS, and granting access to this type of user, particularly in a production environment, should generally be avoided.

Configuring Authentication: Listeners and Security Protocols

You can configure authentication for client-broker communication and broker-broker communication independently, but the available authentication mechanisms are the same. Essentially, when configuring the broker, you specify listeners, which determine the hostname or IP address and port that can be used to reach the broker. Each listener also specifies the security protocol needed to authenticate, whether SSL or SASL_SSL. Note that a broker can be configured with more than one listener, i.e., it can use various address/port/security protocol combinations.

listener-security

The following broker configuration snippet specifies three listeners for a broker: an external network listener, an internal listener, and an inter-broker listener (the inter-broker and internal listeners are configured to use SSL, the external listener SASL_SSL):

listeners=EXTERNAL://:9092,INTERNAL://10.0.0.2:9093,BROKER://10.0.0.2:9094
advertised.listeners=EXTERNAL://broker1.example.com:9092,INTERNAL:// broker1.local:9093,BROKER://broker1.local:9094
listener.security.protocol.map=EXTERNAL:SASL_SSL,INTERNAL:SSL,BROKER:SSL 
inter.broker.listener.name=BROKER

Relatedly, here is a config snippet for a client, specifying that SASL_SSL should be used to communicate with the listed bootstrap servers:

security.protocol=SASL_SSL
bootstrap.servers=broker1.example.com:9092,broker2.example.com:9092 

Use the promo code 101SECURITY to get $25 of free Confluent Cloud usage

Be the first to get updates and new content

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.