Cryptography and security

Version 9.0 of Data Management introduced new password storage and encryption technologies with enhanced encryption algorithms, keys, and access control. These changes support regulatory compliance and best practices for security-conscious customers.

Secrets management

Data Management transmits and stores a variety of digital authentication credentials, or secrets. These secrets are used to:

• Authenticate user credentials with the operating system.

• Run processes (such as projects and automations) on behalf of logged-in users.

• Access external resources (like databases and cloud storage).

• Store "run as" credentials for Web Services, Live Automations, and Schedules defined in the Data Management repository.

• Store internally managed user passwords when Advanced security mode is not enabled.

These secrets may be transmitted from client to server or between server components. They may also be stored in repository projects, in repository settings, or in files. In version 9.0 and higher, Data Management employs modern standards-compliant cryptographic methods to ensure that stored or transmitted secrets cannot be leaked or stolen. These include:

• FIPS 140-2 compliant security modules.

• Best-practice encryption algorithms: 2048-bit RSA keys, 256-bit AES ciphers, and password-protected key stores.

• Key access provided on an as-needed basis.

• Each secret is encrypted using a hybrid RSA/AES-GCM algorithm and encoded into a self-describing ASN.1 envelope.

• Access to private keys for decryption and signing is controlled via password-protected key stores.

• Access to public keys for encryption and signature validation is available to all users and clients.

Inter-service authentication

Data Management's own service components have special privileges in the Data Management system. Data Management employs a signed timestamp strategy (similar to signed JWTs) to ensure that Data Management services cannot be spoofed by bad actors on the network:

• Data Management has a dedicated RSA key pair for inter-service authentication.

• Only the user running the Data Management services has access to the keystore containing this key pair.

• The Site service performs all session authentication, including inter-service authentication.

• The Execution and WSP services authenticate by sending a request to the Site service with a signed timestamp

• The Site service verifies that the signature was created using the system key, and that the timestamp is within a narrow window.

• The Site service replies with a session token.

Keys and keystores

On startup, the Data Management Site service checks for the existence of encryption keys and generates them if they are missing, storing keys in a password-protected keystore file and passwords in a separate password file. It also sets appropriate file permissions on keystores and password files to prevent unauthorized access. Keystores are located under the crypto_keys folder in the Data Management installation location. Within that folder, Data Management creates the following sub-folders, which each contain a keystore and associated password:

The following sub-folders may also be present:

Upgrade secrets from versions prior to 9.0

If you upgrade from a version of Data Management prior to 9.0, any stored secrets will still be in their previous unsecured form. You should update these stored secrets to take advantage of Data Management’s enhanced cryptography.

Update secrets stored in the repository

Data Management will automatically convert any secrets stored in repository objects like settings, projects, and macros when those objects are updated and saved by a user. However, the services will not automatically migrate all stored secrets.

To update secrets stored in the repository

1. Log on to the Data Management client as Administrator.

2. Right-click the repository object containing secrets you want to update, and then select Advanced Repository Actions.

3. Go to the Security update tab.

4. Optionally, clear All subdirectories and their contents to limit the action to the current folder.

5. Select the desired action:

   • Re-encrypt all items with current keys will re-encrypt all secrets.

   • Re-encrypt only pre-v.9.0 legacy items will update only those objects that have not been previously converted.

6. Select Preview to see a list of items that will be updated.

7. Select Start to perform the update.

Update secrets stored in disk files

Data Management includes a command-line utility that will read a project or automation file stored on disk, upgrade all of its stored secrets, and write the upgraded file. This utility is available on any host running Data Management services.

To update secrets stored in disk files

• Run Data Management's command-line utility
  rpdm_convert [-input=filenamein] [-output=filenameout] [-address=host:port] [-reencrypt]

where…

Replacing outdated keyfiles

Any keyfiles created using in versions of Data Management prior to 9.0 are invalid, and you must recreate them manually.

Key rotation

If your organization is security-conscious or is required to conform to PCI or a similar security standard, you should rotate encryption keys. This does the following:

• Creates a new set of encryption key pairs.

• Designates the new encryption key pairs as primary so that new stored secrets will be encrypted using the new keys.

• Designates the old encryption key pairs as retired. They will continue to be available for decryption of legacy stored secrets.

To rotate encryption keys

1. Log into the host running the Data Management Site service.

2. Stop the Data Management services.

3. Log on as the user running the Data Management services, and then run Data Management's command-line utility rpdm_rotate_keys.

4. If there are additional Data Management hosts in the cluster (Execution or Web Service Proxy workers), for each host:

   1. Stop Data Management services on that host.

   2. Copy the crypto_keys folder from the Site service host's Data Management installation location to the equivalent folder on each worker host.

   3. Ensure that the copied folders are owned by the user running the Data Management services.

   4. Ensure that the copied folders and files are writable only by the user running the Data Management services.

   5. Ensure that the "password" files within the copied folders/files are readable only by the user running the Data Management services.

5. When complete, start the Data Management services on all hosts.

• Key rotation does not automatically re-encrypt stored secrets using the new keys. See below.

• If you are promoting items between sites (for example, Development to QA to Production), you will want to copy some or all crypto_keys to the target sites (see Inter-site compatibility).

• Key rotation limits the number of retired keys. Once that limit is reached, Data Management will recycle older keys. The retired keys limit defaults to 4, but this can be changed by adding or editing an item in CoreCfg.properties:
  retired_keys_limit=4

To re-encrypt secrets stored in the repository with new keys

1. Log on to the Data Management client as Administrator.

2. Right-click the repository root, and then select Advanced Repository Actions.

3. Select the Security update tab.

4. Select Re-encrypt all items with current keys.

5. Optionally, select Preview to see a list of items that will be updated.

6. Select Start to perform the update.

This operation re-encrypts secrets in all historic versions of projects, macros, and automations as well as the current version.

To re-encrypt secrets stored in disk files with new keys

See Update secrets stored in disk files, and use the -reencrypt argument.

Purging old keys

For maximum security, you may purge the "old" crypto keys that result from key rotation. To avoid losing access to the encrypted secrets, do this only after re-encrypting stored secrets.

To purge old crypto keys

1. Stop the Data Management services.

2. Go to Data Management installation location and remove all crypto_keys/retired_* folders.

3. Start the Data Management services.

You should retain copies of these old key folders until you have verified that nothing has been lost.

Inter-site compatibility

Because each Data Management Site has its own unique set of keys, secrets stored on one Site cannot be read on another. If you need to share or promote projects, macros, automations, or settings that contain secrets between multiple Sites, you have two options:

• Share all keys between sites.

• Install external keys on a target system.

Shared crypto keys

The simplest way to share secrets between multiple Data Management sites is to share the crypto keys.

To share secrets between sites

1. Update or install the primary site. This site will generate the keys.

2. For each secondary site:

   1. Install or update Data Management on that site.

   2. Stop the Data Management services.

   3. Go to the Data Management installation location and delete the crypto_keys folder.

   4. Copy the crypto_keys folder from the primary site to the secondary site.

   5. Set permissions on the copied folders as follows:

      • All files and folders are owned by the user running the Data Management services.

      • All keystore files are readable by everyone, but writable only by the user running the Data Management services.

      • All password files are readable and writable only by the user running the Data Management services.

3. Start all Data Management services.

If Advanced security is OFF, the Administrator password will be reset to Administrator.

External crypto keys

If you already have two Data Management installations running version 9.0 or later, and they have both been used to store secrets, you cannot share the crypto keys. However, you can still access secrets created on other sites. For example, if a partner sends you a project file or archive containing stored secrets, you can access those stored secrets as long as you have access to the crypto_keys/general folder on the source site.

To install external keys on a target system

1. On the source Data Management site (from which the project file or archive was copied), go to the Data Management installation location and then copy the crypto_keys/general folder to a temporary location accessible by both source and target systems (such as a network share or USB drive).

2. Rename the temporary folder from general to external_general.

3. On the target Data Management site (where you want to open the foreign project), stop the Data Management services.

4. On the target Data Management site, copy the external_general folder from the temporary location to the crypto_keys folder.

5. Set permissions on the copied folders as follows:

   • All files and folders are owned by the user running the Data Management services.

   • All keystore files are readable by everyone, but writable only by the user running the Data Management services.

   • All password files are readable and writable only by the user running the Data Management services.

6. On the target Data Management site, start the Data Management services.

You can copy keys from multiple source systems by naming their target folders uniquely with names like external_general_from_system1, external_general_from_system2. Each folder name must start with external_general.

You can also copy the services folder in a similar manner. You should do this only if you want to promote web service, schedule, and live automation definitions between systems. However, a better strategy is to keep separate definitions of these items with their own credentials.

Third-party credential vaults

Data Management supports several third-party credential vaults, including AWS Secrets Manager, Azure Key Vault, and Google Secret Manager. With Data Management credential vault integration, you have the option of storing secrets such as passwords as either encrypted values (the default in Data Management version 9.0 and later) or as key vault references.

To enter a Key Vault reference

1. Select the key/lock icon, and then select Key Vault reference.

2. Enter the name of the secret stored in the Key vault (get this from your system administrator).

AWS Secrets Manager integration

AWS Secrets Manager is a cloud service that enables you to easily rotate, manage, and retrieve database credentials and other secrets throughout their lifecycle. Data Management supports optional integration with AWS Secrets Manager to retrieve secrets indirectly, instead of encrypting and storing them. You must already have provisioned a AWS Secrets Manager instance and have access credentials.

To enable AWS Secrets Manager integration

1. In the Repository, navigate to Site Settings and select the Security tab.

2. In the Secrets management support section, select Provider and choose AWS Secrets Manager.

3. Specify the Region selection method:

   • Choose Specify region to explicitly define a Region.

   • Choose Default provider chain to get the Region value from the AWS environment.

4. Specify the Authentication method to use when authenticating to Secrets Manager:

   • Choose Default provider chain to get credentials from the AWS environment.

   • Choose Specify credentials to manually enter the Access key ID and Secret access key.

5. Restart the services.

Azure Key Vault integration

Azure Key Vault is a cloud service for securely storing and accessing secrets like keys, certificates, and passwords. With Data Management integration with Azure Key Vault enabled, you have the option of storing secrets such as passwords as either encrypted values (the default in Data Management version 9.0 and later) or as Key Vault references. Note that Data Management only accesses secrets stored in Azure Key Vault. You can access keys or certificates, but they must be stored as secrets, not as keys. You must already have provisioned an Azure Key Vault instance and have access credentials.

To enable Azure Key Vault integration

1. In the Repository, navigate to Site Settings and select the Security tab.

2. In the Secrets management support section, select Provider and choose Azure Key Vault.

3. Enter the URL of the key vault. This is usually of the form https://unique_name.vault.azure.net/.

4. Specify the Authentication method to use when authenticating to Azure Key Vault:

   • Choose Managed identity to use an Azure Active Directory managed identity.

   • Choose Service credentials to enter a service principal's credentials:

5. Restart the services.

Google Secret Manager integration

Google Secret Manager is a cloud service providing a secure and convenient storage system for secrets like keys, passwords, certificates, and other sensitive data. Data Management supports optional integration with Google Secret Manager to retrieve secrets indirectly, instead of encrypting and storing them. Your organization must already have provisioned a Google Secret Manager instance and have access credentials.

To enable Google Secret Manager integration

1. In the Repository, navigate to Site Settings and select the Security tab.

2. In the Key vault support section, select Provider and choose Google Secret Manager.

3. Specify the Authentication method to use when authenticating to Secret Manager:

   • Choose Application default credentials to use the service account credentials for a specified Project ID.

   • Choose Credentials file to specify the absolute path to a JSON Credentials file that contains the credentials.

4. Restart the services.