Client-side Field-level Encryption Demo

Creating environment and cluster setup on Azure Hello, this is Peter Moskowitz from Confluent, and in this video, we'll walk through an end-to-end scenario demonstrating how client-side field-level encryption works with fully managed connectors in Confluent Cloud. ♪♪ I'm logged into Confluent Cloud, and the first step is to create an environment. Let's call it csfle-demo-env. For this demo, we need data governance enabled. However, for experimentation, we'll start with a data governance essentials environment, and we'll see the consequences later on. Next, we create a cluster hosted by Microsoft Azure, selecting East US as our region. Since our key encryption key will be managed by Azure, knowing our cluster's region is crucial. Let's name the cluster csfle-cluster. To ensure proper permissions, we'll configure our service account. Click the hamburger menu, the three lines in the top right, Configuring service account permissions and then navigate to accounts and access. Under service accounts, select your service account, Peter SA in our case, then assign the necessary roles. Schema objects, all schema objects, developer right. And then encryption keys, all encryption keys, resource owner. With permissions in place, let's create our first connector, Creating and configuring HTTP source V2 connector an HTTP source V2 connector, which will simulate credit card transactions from an HTTP endpoint. Upload the Swagger YAML file defining the API. We'll select Peter SA as the service account with required ACLs enabled. Let's review the HTTP endpoint in the Swagger file. Under paths, select the get method. This is the HTTP API that will serve us credit card transactions. Create a topic called csfle-demo-topic. Click test to verify the endpoint. Here are our records. Skip single message transforms and launch the connector. Now, switching to our topic, we see messages being produced. Tagging fields for encryption and creating encryption rules However, sensitive fields like credit card numbers and CVC codes remain unencrypted. To encrypt these fields, we'll take two steps. One, tag fields for encryption in the data contracts tab. We evolve our schema and add a PII, personally identifiable information tag, to the credit card number field. And then we define an encryption rule. Since this requires advanced governance, this is the point where we are prompted to upgrade our environment. As part of this step, we also have to head over to Azure Setting up key encryption key in Azure Key Vault and Confluent Cloud to create a key encryption key in Azure Key Vault. I navigate to Azure portal, and here we create a new key. Confluent CSFLE key. We go ahead and copy its key ID. In Confluent Cloud, we navigate to encryption keys and add a new key. And here we paste the key ID. The Azure tenant ID, and the client ID. The scripts, down below this form, walk you through the steps you need to take to set things up in Azure. With encryption configured, we return to data contracts and create a PII encryption rule. This is a key encryption rule, We return to data contracts and create a PII encryption rule. Category, data encryption rule. Rule name, CSFLE PII rule. Tags, PII. Encryption key, CSFLE demo key. Back in the HTTP source connector settings, Enabling client-side field-level encryption on the connector we navigate to settings, advanced configuration. We enable client-side field level encryption and select my service account, Peter SA. Navigating back to our message viewer, we can see that the credit card number field is now encrypted. Wouldn't it be nice if we could encrypt the CVC field as well? The process is very simple. All we need to do is tag the CVC field with the PII tag. Let's go ahead, restart the connector, and verify the encryption. Success! Now, let's consume these encrypted records Consuming encrypted records with HTTP sync V2 connector using an HTTP sync V2 connector. We'll upload the swagger file that defines the HTTP endpoint. We use Peter SA as the service account. Select a post method under paths and choose the CSFLE demo topic. Let's go ahead and test the setup with a dummy credit card transaction. The HTTP service we connected our sync connector to inserts records into Airtable. We can leave the defaults in the remaining connector definition screens and go ahead and launch our connector. And voila! The records are flowing. Once our Airtable is caught up to the previously published records on the topic, we will need to restart the source connector we paused earlier. Our final step is to configure our sync connector to decrypt the credit card number and the CVC fields. Decrypting records at the sync connector and verifying data privacy Let's navigate to the sync connector, advanced configuration, advanced configuration, enable client-side field lever encryption for this connector, select Peter SA, and now records are decrypted before reaching Airtable. We verify this by checking a transaction encrypted in Confluent Cloud and decrypted in Airtable. By encrypting at the source and decrypting at the sync, we ensure data privacy and compliance. Thank you for joining me.