How to encrypt data and manage encryption keys using Key Management Service (KMS)

Check out the data

1. We will store the data in Cloud Storage Bucket.


2. Run the following and pick a name for it, such as encrypted_data_imp.
BUCKET_NAME=encrypted_data_imp


3. Copy the data to the bucket or copy from other buckets
gsutil cp gs://bucket_copy_from/filename .


4. Before using KMS you need to enable it in your project. This can be done using the gcloud command-line utility. Run the following command in your shell:
gcloud services enable cloudkms.googleapis.com

Create a Keyring and Cryptokey

1. In order to encrypt the data, you need to create a KeyRing and a CryptoKey. KeyRings are useful for grouping keys. Keys can be grouped by environment (like test, staging, and prod) or by some other conceptual grouping. Let's define a KeyRing called my_cloud_key_ring and your CryptoKey will be called my_cloud_key.
KEYRING_NAME=test CRYPTOKEY_NAME=codelab


2. Execute the gcloud command to create the KeyRing. The location as global, but it could also be a specific region.
gcloud kms keyrings create $KEYRING_NAME --location global


3. Next, using the new KeyRing, create a CryptoKey named my_cloud_key.
gcloud kms keys create $CRYPTOKEY_NAME --location global --keyring $KEYRING_NAME --purpose encryptionNote: You can't delete CryptoKeys or KeyRings in Cloud KMS! You've created a KeyRing and CryptoKey! Open Encryption Keys through the Console. Go to Products & services > IAM & Admin > Encryption Keys
   The Key Management Web UI allows you to view and manage your CryptoKeys and KeyRings. You will use this UI later when you manage permissions.

Encrypt Data

1. Next, try to encrypt some data! Take the contents and base64 encode it by running the following:
PLAINTEXT=$(cat FILENAME | base64 -w0) Note: Base-64 encoding allows binary data to be sent to the API as plaintext. This command works for images, videos, or any other kind of binary data.


2. Using the encrypt endpoint, you can send the base64-encoded text you want to encrypt to the specified key. Run the following:
curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:encrypt" \
-d "{\"plaintext\":\"$PLAINTEXT\"}" \
-H "Authorization:Bearer $(gcloud auth application-default print-access-token)"\
-H "Content-Type: application/json
"Note: The encrypt action will return a different result each time even when using same text and key.


3. The response will be a JSON payload containing the encrypted text in the attribute ciphertext. Now that your data is encrypted, you can save it to a file and upload it to your Cloud Storage bucket. To grab the encrypted text from the JSON response and save it to a file, use the command-line utility jq. The response from the previous call can be piped into jq, which can parse out the ciphertext property to the file 1.encrypted. Please note that jq is a lightweight and flexible command-line JSON processor. Run the following:
curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:encrypt" \
-d "{\"plaintext\":\"$PLAINTEXT\"}" \
-H "Authorization:Bearer $(gcloud auth application-default print-access-token)"\
-H "Content-Type:application/json" | jq .ciphertext -r > FILENAME.encrypted



4. To verify the encrypted data can be decrypted, call the decrypt endpoint to verify the decrypted text matches the original email. The encrypted data has information on which CryptoKey version was used to encrypt it, so the specific version is never supplied to the decrypt endpoint. Run the following:
curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:decrypt" \
-d "{\"ciphertext\":\"$(cat 1.encrypted)\"}" \
-H "Authorization:Bearer $(gcloud auth application-default print-access-token)"\
-H "Content-Type:application/json" | jq .plaintext -r | base64 -d
Note: Usually decryption is performed at the application layer. For a walkthrough on how to encrypt and decrypt data in multiple programming languages, read the Cloud KMS Quickstart.


5. Now that you have verified the text has been encrypted successfully, upload the encrypted file to your Cloud Storage bucket.
gsutil cp 1.encrypted gs://${BUCKET_NAME}

Configure IAM Permissions

In KMS, there are two major permissions to focus on. One permissions allows a user or service account to manage KMS resources, the other allows a user or service account to use keys to encrypt and decrypt data. The permission to manage keys is cloudkms.admin, and allows anyone with the permission to create KeyRings and create, modify, disable, and destroy CryptoKeys. The permission to encrypt and decrypt iscloudkms.cryptoKeyEncrypterDecrypter, and is used to call the encrypt and decrypt API endpoints.

1. Let's use the current authorized user to assign IAM permissions. To get the current authorized user, run the command below.
USER_EMAIL=$(gcloud auth list --limit=1 2>/dev/null | grep '@' | awk '{print $2}')


2. Next, assign that user the ability to manage KMS resources. Run the following gcloud command to assign the IAM permission to manage the KeyRing you just created:
gcloud kms keyrings add-iam-policy-binding $KEYRING_NAME \
--location global \
--member user:$USER_EMAIL \
--role roles/cloudkms.admin
Since CryptoKeys belong to KeyRings, and KeyRings belong to Projects, a user with a specific role or permission at a higher level in that hierarchy inherits the same permissions on the child resources. For example, a user who has the role of Owner on a Project is also an Owner on all the KeyRings and CryptoKeys in that project. Similarly, if a user is granted the cloudkms.admin role on a KeyRing, they have the associated permissions on the CryptoKeys in that KeyRing.


3. Without the cloudkms.cryptoKeyEncrypterDecrypter permission, the authorized user will not be able to use the keys to encrypt or decrypt data. Run the following gcloud command to assign the IAM permission to encrypt and decrypt data for any CryptoKey under the KeyRing you created:
gcloud kms keyrings add-iam-policy-binding $KEYRING_NAME \
--location global \
--member user:$USER_EMAIL \
--role roles/cloudkms.cryptoKeyEncrypterDecrypter
Now you can view the assigned permissions in Key Management in the Console. Click the three vertical dots by the KeyRing and select Edit Permissions.

Back up data on the Command Line

Now that you have an understanding of how to encrypt a single file, and have permission to do so, you can run a script to backup all files in a directory. For this example, copy all emails for allen-p, encrypt them, and upload them to a Cloud Storage bucket.

1. First, Copy the Folder to the bucket or copy from other buckets:
gsutil -m cp -r gs://bucket_copy_from/FOLDERNAME .


2. Now copy and paste the following into the Cloud Shell terminal to backup and encrypt all the files in the allen-p directory to your Cloud Storage bucket:
MYDIR=allen-p
FILES=$(find $MYDIR -type f -not -name "*.encrypted")
for file in $FILES; do
PLAINTEXT=$(cat $file | base64 -w0)
curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:encrypt" \
-d "{\"plaintext\":\"$PLAINTEXT\""'}'"" \
-H "Authorization:Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type:application/json" \
| jq .ciphertext -r > $file.encrypted
done
gsutil -m cp allen-p/inbox/*.encrypted gs://${BUCKET_NAME}/allen-p/inbox
This script loops over all the files in a given directory, encrypts them using the KMS API, and uploads them to Google Cloud Storage. Note: Cloud Storage supports Server Side Encryption, which supports key rotation of your data and is the recommended way to encrypt data in Cloud Storage.