Authenticating Users and Applications

How many times do you think you use authentication on a daily basis? I bet most of us don't even think, or realize how often it happens. Authentication is the process of verifying you are who you say you are. So every time you log into your phone, check your email, enter a password, or perform a biometric scan to work on your computer, these are all different types of authentication. In its simplest form, we present something that identifies us. For low-security situations, that could be something as simple as stating our name. In a high-security situation, that could be some form of ID, password, and a biometric identifier. When we talk about Confluent Cloud, there are two distinct interactions that need authentication, authenticating your users, and authenticating your applications, or services. For your users, the simplest way to access Confluent Cloud is via email address. While this method works great for a small set of users, most organizations will want to use Single Sign-On, or SSO. SSO is the process of combining multiple login screens into one. From a user perspective, they enter one username and password, and are automatically logged into any of the applications they have access to. You may have seen this using identity providers, such as Okta, Active Directory, or Google accounts for authentication. The benefits of using SSO is that it enables a central and consistent policy layer for multifactor authentication, password enforcement, and terminating user access. The only requirement for this method is that your identity provider must use Security Assertion Markup Language, or SAML. To authenticate applications, or services, you have two options, using API keys, or connecting your applications via OAuth. Let's talk about API keys first. You can get an API key either via the Confluent CLI, or via the web interface. Each API key consists of a key and a secret. There are two types of API keys you can use, resource-specific keys, and cloud API keys. Resource-specific API keys give access to individual resources in a particular environment. An environment is a collection of Kafka clusters and deployed components, such as Connect, ksqlDB, and Schema Registry. An organization can contain multiple environments. You might have different environments for different departments, or teams, for example, all belonging to the same organization. The second type of API key are cloud API keys, which allow you to grant access to manage the entire organization's Confluent Cloud instance and environments, or just a single environment, depending on how you configure it with RBAC. If you plan on using API keys, there are a couple things you should keep in mind. First, only use service accounts in production. API keys for Confluent Cloud can be created with user and service accounts. A service account is intended to provide an identity for an application, or service that needs to perform programmatic operations within Confluent Cloud. When moving to production, ensure that only service account API keys are used. Avoid user account API keys, except for development and testing. If a user leaves, changes departments, or their account is deleted, all API keys created with that user account are deleted, and will cause your applications to stop working. For this reason, we recommend creating a policy of only using service accounts for your applications and services, even in development and testing stages. Number two, manage and delete unneeded keys and service accounts. As a standard practice of your security strategy, you should regularly review and clean up your existing API keys and service accounts. To better understand which API keys are being used, you can review and monitor authorization and authentication events in your audit logs. To list the API keys that you currently have, run the Confluent api-key list command. Use the --service-account option to list only the API keys associated with the specific service account. Number three, rotate your API keys regularly. Access to your Confluent Cloud resources is controlled by API keys associated with service accounts, which have access controls determining what the service account has access to. API keys can be created and destroyed without affecting the service account ACLs and RBAC row bindings. Rotating API keys is a good security practice that limits the potential impact of an API key that is leaked. When you rotate API keys, it is recommended you perform the steps in the following order. First, create a new API key. Then update the resource, or application to use your newly created API key. Verify the connection via your audit logs. Last, delete the old API key. Number four, last, but not least, make sure you are using audit logs to track the usage of API keys. As was mentioned earlier, OAuth is the other way to provide access to your application and services. OAuth allows you to access your resources and data, without having to share, or store user credentials. It's built off cryptographically signed access tokens that allow your applications, or services to authenticate. There are few advantages to using OAuth for authentication. OAuth is a cloud-native authentication solution. Most companies already employ an OAuth solution for other cloud resources they use for authentication and identity management. OAuth provides centralized identity management. System administrators have only one place to manage authentication for all their systems, rather than having to create and manage credentials, or set up a sync system for each independent service in their organization. OAuth is an industry standard for providing authentication. If you are are an organization with a large number of service accounts, or applications accessing Confluent Cloud, you might run out of API keys. OAuth allows you to scale to many thousands of identities and credentials. While we'll talk about authorization later in this course, I wanted to briefly touch on how OAuth ties in to make sure your applications only have the access they're supposed to have. OAuth uses identity pools to map groups of identities to your role-based access control, or access control policies. For example, let's say you have a group of applications that all need access to the same cluster, or Kafka topic. Rather than give each application individual access, and controlling access on a per-application basis, you can create an identity pool that uses an identity pool filter to map each application identity to this identity pool. Then using role-based access control, or access control list policies, you can control permissions for the pool. There are a few things you should keep in mind if you plan on using OAuth. First, you can use OAuth with Apache 3.2.1, Confluent Platform 7.2.1 and 7.1.3, and later, and librdkafka 1.9.2, or later. Second, you'll need to upgrade any older, or legacy clients that don't support OAuth. While most applications, or services have support built-in, you may have some legacy systems that will need to be upgraded, or which may require you to use API keys. Third, group clients into identity pools when possible. If you come from using API keys, your first inclination might be to create an identity pool for each service account. While this is possible, take some time to really think about your clients, and see if there are groups that you can combine and control with one RBAC, or ACL policy. Last, but not least, whether you use API keys, or OAuth, we highly recommend using RBAC for authorization, something that we'll look at later in this course. API keys allow you to control access on a per-application and service basis, but you are limited in terms of the number of keys. OAuth allows you to group identities into identity pools and manage access at the pool level. OAuth also allows you to manage authentication across your entire organization for all applications and services, whereas API keys are specific to Confluent Cloud, and can't be used for other applications and services. At the end of the day, both will allow your applications and services to get access to your Confluent Cloud cluster. You may find that as you start to test and develop your Confluent Cloud cluster that API keys meet your needs. However, in most cases, the time and effort required to implement OAuth at the beginning pays off as your cluster grows, and saves you from having to convert your processes and systems over at a later date.