Version: 7.8

HashiCorp Vault Integration

Overview

Resolve Actions Pro integration with Vault provides you with the added security of storing passwords and other secrets in HashiCorp Vault. To check which passwords are stored in Vault for your version of Actions Pro, check the version-specific information in the Compatibility Matrix section.

Benefits

Without Vault integration, Actions Pro stores the various kinds of secrets that are used by Runbooks, Gateway Filters, Action Tasks, and so on in the app database and in .properties and .xml files in encrypted form. Compared to it, the Vault approach adds the following benefits:

Role-based access—Role-based access for individual Actions Pro components is used to provide an extra level of protection to customer secrets. For example, only RSRemote can read gateway passwords and only RSMgmt can write them, while previously RSControl and other Actions Pro components also had the complete ability to decrypt and read a password.
Reduced risk—With the use of Vault, any secrets stored in Vault are masked with asterisks across the Actions Pro application memory, file systems, and .properties files. This greatly reduces the risk of unencrypted passwords being written to some crash log or memory cache. It also ensures that nobody has access to encrypted passwords, reducing the chance of a data breach if a major encryption vulnerability is discovered.
Consolidation of secrets—Actions Pro shares data and credentials with many third-party systems. Due to the HashiCorp Vault popularity, the integration enables use cases where secrets between Resolve Actions Pro, Resolve Insights, and third-party systems can be centralized in a single Vault cluster.
State of the art security—Due to the specialization and popularity of HashiCorp Vault, it can implement new encryption schemes and patch security vulnerabilities much faster.

Vault Workflow

When Vault integration is enabled, Actions Pro can pick up gateway passwords entered in blueprint.properties or in the Actions Pro UI and store them in Vault. After Vault stores them, RSRemote is the Actions Pro component that retrieves secrets from Vault.

The following high-level diagram illustrates the various components implementing the Vault Integration with Actions Pro. It depicts both use cases.

On the diagram, Network with Actions Pro is the network that contains the main Actions Pro installation and Remote Network contains standalone RSRemote and RSmgmt components.

In the first scenario, the user enters a secret in the UI, such as on the Gateway Filter Configuration page available on Example HTTPGateway, Builder Gateway, and others. When the HTTP request from the UI reaches RSView, RSView removes the secret from the request and sends it to RSMgmt (KeyServiceController). In turn, RSMgmt sends the secret to Vault for storing.

In the second scenario, the users enters a secret by editing a blueprint.properties file. An Actions Pro configuration script then removes the secret from the file system and calls RSMgmt to store the secret in Vault.

After a secret is stored in Vault, eligible Actions Pro components can read it directly from Vault after authenticating against Vault.

Vault Concepts

Vault supports a wide array of alternative technologies. The following sections explain the technology choices made by Actions Pro for the Vault integration.

Auto Unseal

When the Vault service starts, it starts in a sealed state and it does not know how to decrypt data. Before it can provide secrets to Actions Pro, it must unseal. Unsealing is the process of constructing the master key necessary to decrypt the data encryption key.

Actions Pro utilizes the Vault auto-unseal feature to unseal the service whenever needed. This reduces the manual intervention otherwise required to unseal Vault which involves providing key-shards (parts of a key) that Vault uses to decrypt the master key and unseal.

With auto-unseal, the Vault service can be automatically sealed or unsealed using Transit Secrets Engine which represents another trusted Vault service called RSVault. This is how RSVault differs from RSKeyservice:

RSKeyservice is the main Vault service which stores the secrets. It must be accessible from all RSRemote instances to allow them to read secrets and talk to third-party services using gateways. It must also be accessible from other Actions Pro cluster nodes to allow RSKeyservice instances on other Actions Pro cluster nodes to sync with each other.
RSVault is a local auxiliary Vault service that only runs on local loopback and is primarily responsible for auto-unsealing the local RSKeyservice vault service. To make it a trusted Vault service, you will be asked to password protect it during the initial Vault setup.

Every time the RSKeyservice service restarts, RSVault automatically unseals it without user intervention, but only if the RSVault service is running.

In production, it is recommended to always keep RSVault running in the background as а daemon, even when stopping or restarting other Actions Pro services, including RSKeyservice.

note

Actions Pro does not officially support using your own Vault instance as a trusted Vault instance instead of RSVault. Use one at your own risk.

Recovery (Master) Password

The recovery (or master) password is a passphrase that you use to secure the RSVault installation. You will need to enter it whenever you restart the RSVault server, perform planned maintenance or migration, or for recovery purposes if applications on the server crash.

Store the recovery key some place safe. Do not store it on the same machine where RSVault runs.

You only need the recovery key when restarting RSVault. You don’t need it when restarting other Actions Pro components or RSKeyservice.

Secret Engine and Path

From the selection of Secret Engines and paths supported by Vault, Actions Pro uses the K/V v1 engine. To check what secrets Actions Pro stores in Vault and at what paths, check the the Compatibility Matrix section for your Actions Pro version.

Authentication and Authorization

From the selection of authentication and authorization methods supported by Vault, Actions Pro uses both token-based authentication and AppRole authentication. All tokens generated are either encrypted using RSA keys or ephemeral and expire after a few minutes.

Vault Policies & Roles

Actions Pro internally creates policies and roles in Vault. See Vault Policies and Roles for details.

Compatibility Matrix

The following table shows which Actions Pro features can store secrets in Vault.

Feature	Description	Action
Vault integration	Vault services: RSVault, RSKeyservice	Disabled by default. When disabled, passwords use Actions Pro encryption.
Gateway passwords	See full list in Gateway Passwords Stored in Vault	Vault
EWS Address	EWS password on Filter UI	Vault
Gateway Builder Encrypted Fields	Encrypted fields on Gateway Builder UI, such as password fields	Vault
Database Gateway Connection Pool password	DB Connection Pool password	Vault
Email Gateway, Email Address password	Email password on Filter UI	Vault
HTTP Gateway	Basic Auth password	Vault
Action Task Parameters	Encrypt-type Action Task Parameters	Encrypted
Action Task Encrypted Properties	Encrypt-type Action Task properties	Encrypted
Database, RSMQ, Elastic Search passwords	Other passwords stored in the blueprint.properties file	Encrypted

Gateway Passwords Stored in Vault

With Vault integration, service account credentials for gateways are stored in Vault as opposed to the blueprint. The value of any blueprint.properties property named rsremote.receive.<gateway-name>.<property-type> is stored in Vault, where gateway name is the name of the gateway and property type is a property name containing the phrase pass. This applies to the following gateways:

Arcsight
AMQP
MCAFEE
ASSURENOW
SCOM
ServiceDesk
Solarwinds
Jira
Ibmmq
CA Spectrum
XMPP
HPSM
Remedy
SplunkPull
Servicenow
Email
Exchange
HTTP
Moogsoft
Splunk
Weblogic

Gateway Filter Secrets Stored in Vault

Gateways can have secrets that are specific to a filter. For example, you can have two EWS Addresses entries for EWS Gateway, each having its own password. Such secrets are stored in Vault, including:

Exchange User accounts for EWS Gateway
Database User accounts for DB Gateway
Basic Auth credentials for HTTP Gateway, Moogsoft Gateway
Dynamic secret key-value pairs configured in Gateway Builder

Making a Fresh Vault Installation

Actions Pro comes prepackaged with a Vault distribution that you can install and configure in the following cases:

You are already running Actions Pro and would like to start using Vault as your secrets storage.
You are making a new Actions Pro deployment and would like to store secrets in Vault from the get go.

If you have a pre-existing Vault deployment that you manage, see Using an Existing Vault Installation.

Installing Vault requires the following procedures:

Installing Vault Locally
Configuring Remote RSRemote and RSMgmt Instances

Installing Vault Locally

First, you need to configure and install Vault on the machine or machines that run Actions Pro. Actions Pro provides scripts to help automate the Vault pre-configuration and the installation.

Perform the procedure on every node in your Actions Pro cluster.

Prerequisites

A working Actions Pro deployment.
All Actions Pro cluster nodes must be in the same domain. Standalone RSRemote instances can be in a different domain.
Ensure that the Linux user that you are logged in with can read, write, and execute files located in the following directories:
- <actions-pro-home>/rsvault
- <actions-pro-home>/rskeyservice
Ensure that you have a Vault distribution available in your Actions Pro installation directory by listing its directory content and looking for the directories listed below. If any of the directories are missing, create them.
```
./rsvault
./rsvault/bin/
./rsvault/config/
./rsvault/config/certs/
./rsvault/data/
./rsvault/logs/
./rskeyservice
./rskeyservice/bin/
./rskeyservice/config/
./rskeyservice/data/
./rskeyservice/logs/
```
Take note of the Fully Qualified Domain Name (FQDN) of the machine that will be running Vault.
Ensure that the FQDN can be resolved to an IP from the machine.

Procedure

note

All commands below assume that Actions Pro is installed under /opt/resolve. Change the commands accordingly if your installations uses another directory.

Log in to the Actions Pro machine as the resolve user that you created during Actions Pro installation.
Stop all locally-installed Actions Pro services.
You can keep any remote Actions Pro services such as standalone RSRemote and RSMgmt instances running.
```
 /opt/resolve/bin/stop.sh all -k
```
Change directory to the Actions Pro installation directory.
```
cd /opt/resolve
```

Open ./rsmgmt/config/blueprint.properties for editing and make the following changes:

Set the following properties to true.

resolve.rsvault = true
resolve.rskeyservice = true

Modify or add the following properties as shown.
- RSKEYSERVICE_HOST—Set to the FQDN of the machine that will be running Vault (RSKeyservice).
- RSKEYSERVICE_PORT— Set to the Vault port number for reading and writing. The default-installation Vault port is 8200.
- RSKEYSERVICE_CLUSTER_PORT— Set to the Vault port number for synchronizing clustered Vault instances. The default-installation Vault port is 8201.
- RSKEYSERVICE_LOG— Set the Vault log verbosity level that you want. Possible values: OFF, INFO, DEBUG, TRACE. Recommended level for production: WARN or INFO.
- RSVAULT_PORT— Set to the Vault port used for internal operations. The default-installation Vault port is 8300.

Ensure the following properties are present. If not, add them. Do not edit the properties unless a custom setup is needed.

#RSVAULT
rsvault.hcl.storage=file
rsvault.hcl.storage.path=${DIST}/rsvault/data
rsvault.hcl.storage.node_id=rsvault
rsvault.hcl.listener=tcp
rsvault.hcl.listener.address=127.0.0.1:${RSVAULT_PORT}
rsvault.hcl.listener.tls_disable=true
rsvault.hcl.nostanza.disable_mlock=true
rsvault.hcl.nostanza.api_addr=http://127.0.0.1:${RSVAULT_PORT}
rsvault.hcl.nostanza.ui=false

#RSKEYSERVICE
rskeyservice.hcl.storage=raft
rskeyservice.hcl.storage.path=${DIST}/rskeyservice/data
rskeyservice.hcl.storage.node_id=rskeyservice
rskeyservice.hcl.listener=tcp
rskeyservice.hcl.listener.address=${RSKEYSERVICE_HOST}:${RSKEYSERVICE_PORT}
rskeyservice.hcl.listener.tls_cert_file=${DIST}/rsvault/config/certs/rsvault.pem
rskeyservice.hcl.listener.tls_key_file=${DIST}/rsvault/config/certs/private.pem
rskeyservice.hcl.seal=transit
rskeyservice.hcl.seal.address=http://127.0.0.1:${RSVAULT_PORT}
rskeyservice.hcl.seal.disable_renewal=false
rskeyservice.hcl.seal.key_name=autounseal
rskeyservice.hcl.seal.mount_path=transit/
rskeyservice.hcl.nostanza.disable_mlock=true
rskeyservice.hcl.nostanza.api_addr=https://${RSKEYSERVICE_HOST}:${RSKEYSERVICE_PORT}
rskeyservice.hcl.nostanza.cluster_addr=https://${RSKEYSERVICE_HOST}:${RSKEYSERVICE_CLUSTER_PORT}
rskeyservice.hcl.nostanza.ui=false

Apply the changes to blueprint.properties.
```
 /opt/resolve/bin/config.sh
```

In the config.sh output, ensure that the following three lines are present.

Configuring Vault
Configured File: /opt/resolve/rsvault/config/rsvault.hcl
Configured File: /opt/resolve/rskeyservice/config/rskeyservice.hcl

Execute the following command to install the RSVault and RSKeyservice services:
```
rsvault/bin/run.sh -p -d example.com -t 30m
```
Where:
- The -p flag ensures that you will be prompted to set a master (recovery) password.
- The -d flag allows you to specify the domain name of the machine where Vault will be running. It is used to generate private/public keys for securing the Vault TLS transport.
- The -t flag is the time for which an ephemeral token used by config.sh script will be valid. This time will apply to future config.sh executions.

When prompted, enter a master (recovery) password and press Enter.
The password must be at least six characters long.

--------------------Start Vault Setup--------------------   

>  Recovery Setup                                                           
Enter a secure passphrase that will be used to
 encrypt/decrypt the master key:

Verify that the command output does not contain ERROR outcomes.
If you see any errors, see the Troubleshooting section for possible resolutions.
Start your local Actions Pro services.
```
bin/run.sh all
```
Migrate your existing secrets to Vault to ensure you don’t lose any data.

Next steps

If you are running remote standalone RSRemote or RSMgmt servers, take the steps in the Configuring Remote RSRemote and RSMgmt Instances section.

Configuring Remote RSRemote and RSMgmt Instances

Unlike local instances of RSMgmt and RSRemote, which are autoconfigured to work with Vault, you need to manually configure these services if you run them as remote standalone components.

Perform the following steps to configure remote services:

Copy the local version of the following three files to the standalone RSRemote machine, overwriting the remote files with the same name.
- <actions-pro-home>/rsremote/config/application.properties
- <actions-pro-home>/rsmgmt/config/application-rest.properties
- <actions-pro-home>/rsmgmt/config/application-config.properties
Open each file and verify that all settings look correct.

Using an Existing Vault Installation

You can configure Actions Pro to utilize a pre-existing Vault installation that you have in your environment and that you manage. To do that, you need to take configuration steps on both the Actions Pro and the Vault sides.

caution

For the integration with your existing Vault instance to work, you need to create a dedicated path for Actions Pro with read, write, and admin permissions in place.

Configuring Vault

To set up your existing Vault deployment to work with Actions Pro, perform the following general steps. For detailed steps, refer to the official Hashicorp Vault documentation.

Create Actions Pro backend - Mount a KV version 1 secrets engine at path resolve/action.
Enable AppRole authentication.

Create the following Actions Pro policies:

resolve-admin

path "auth/approle/role/rsremote/*" {
    capabilities = ["read", "update","create", "list"]
}

path "resolve/action/*" {
    capabilities = ["read", "update", "delete", "create", "list"]
}

resolve-user

path "resolve/action/*" {
    capabilities = ["read", "list"]
}

resolve-initial

path "auth/approle/role/rsmgmt/*" {
    capabilities = ["read", "update","create", "list"]
}

Create the following Actions Pro roles:
- rsmgmt—based on the resolve-admin policy.
- rsremote—based on resolve-user policy.

Configuring Actions Pro

To set up Actions Pro to work with your Vault deployment, you need to configure your RSRemote and RSMgmt instances with the Vault details.

You must take the configuration steps in the following order:

Stop RSRemote.
1. Log in to the RSRemote instance with Actions Pro user that you created during Actions Pro installation.
2. Run the stop script.
```
<rsremote-install-dir>/rsremote/bin/stop.sh
```
Stop RSMgmt.
1. Log in to the RSMgmt instance with the resolve user that you created during Actions Pro installation.
2. Run the stop script.
```
<rsmgmt-install-dir>/rsmgmt/bin/stop.sh
```
Configure RSRemote.
Configure RSMgmt.
Start RSMgmt.
1. Log in to the RSMgmt instance with the resolve user that you created during Actions Pro installation.
2. Run the start script.
```
<rsmgmt-install-dir>/rsmgmt/bin/run.sh
```
Start RSRemote.
1. Log in to the RSRemote instance with Actions Pro user that you created during Actions Pro installation.
2. Run the start script.
```
<rsremote-install-dir>/rsremote/bin/run.sh
```

Configuring RSRemote

To set up RSRemote to work with your Vault deployment, perform the following steps for each of its local and remote instances.

Ensure that RSRemote and RSMgmt are stopped.
Log in to the RSRemote instance with Actions Pro user that you created during Actions Pro installation.
In Vault, generate a Role ID for RSRemote and set the vault.app-role.role-id property with it in <rsremote-install-dir>/rsremote/config/application.properties.
For example:
```
vault.app-role.role-id=98995263-71f9-fa53-373a-0bd4318b6bf3
```
If you are using a local CA authority to sign Vault Server certificates, take these steps:
- Generate a truststore and add the root certificate in it.
- In <rsremote-install-dir>/rsremote/config/application.properties, add the following properties, specifying the truststore path relative to rsremote/lib/ and the truststore password.
  tip
  It is recommended that you place the trust-store inside the rsremote/lib/ folder to keep things simple.
```
vault.ssl.trust-store=classpath:<trust-store-name or path>
vault.ssl.trust-store-password=<trust-store password>
```

Comment out the vault.token property.

#vault.token=00000000-0000-0000-0000-000000000000

Keep RSRemote and RSMgmt stopped.

The following is a sample application.properties file, showing a truststore named ca.cert.jks located under rsremote/lib.

vault.authentication=APPROLE
vault.app-role=rsremote
vault.uri=https://test-server.example.com:8200
vault.ssl.trust-store=classpath:ca.cert.jks
vault.ssl.trust-store-password=resolve
#vault.token=00000000-0000-0000-0000-000000000000

Configuring RSMgmt

To set up RSMgmt to work with your Vault deployment, perform the following steps for each of its instances.

Ensure that RSRemote and RSMgmt are stopped.
In Vault, generate a short-lived (around 30 min) token using the resolve-initial policy.
Log in to the RSMgmt instance with the resolve user that you created during Actions Pro installation and set the token in the following files:
1. In <rsmgmt-install-dir>/rsmgmt/config/application-config.properties, set the vault.token property with the token that you generated.
```
vault.token=<generated token goes here>
```
2. In <rsmgmt-install-dir>/rsmgmt/config/application-rest.properties, set the vault.token property with the token that you generated.
```
vault.token=<generated token goes here>
```
If you are using a local CA authority to sign Vault Server certificates, take these steps:
- Generate a truststore and add the root certificate in it.
- In <rsmgmt-install-dir>/rsmgmt/config/application-config.properties, add the following properties, specifying the truststore path relative to rsmgmt/lib/ and the truststore password.
  tip
  To keep things simple, place the trust-store inside the rsmgmt/lib/ folder.
```
# Use custom trust-store
vault.ssl.trust-store=classpath:<trust-store-name or path>
vault.ssl.trust-store-password=<trust-store password>
```
- In <rsmgmt-install-dir>/rsmgmt/config/application-rest.properties, add the following properties, specifying the truststore path relative to rsmgmt/lib/ and the truststore password.
  tip
  To keep things simple, place the trust-store inside the rsmgmt/lib/ folder.
```
# Use custom trust-store
vault.ssl.trust-store=classpath:<trust-store-name or path>
vault.ssl.trust-store-password=<trust-store password>
```
Make the following changes to the <rsmgmt-install-dir>/rsmgmt/config/application-config.properties file:
1. Add the vault.authentication and set it to to APPROLE.
```
vault.authentication=APPROLE
```
2. Add the vault.app-role property and set it to rsmgmt.
```
vault.app-role=rsmgmt
```
3. Add the vault.uri property and set it to the FQDN of the Vault machine, including port number.
```
vault.uri=https://customvault.example.com:8200
```
Make the following changes to the <rsmgmt-install-dir>/rsmgmt/config/application-rest.properties file.
1. Add the following properties and set them as shown:
```
vault.authentication=APPROLE
vault.app-role=rsmgmt
```
2. Add the vault.uri property and set it to the FQDN of the Vault machine, including port number.
```
vault.uri=https://customvault.example.com:8200
```
Migrate your existing secrets to Vault, if any.

Example of <rsmgmt-install-dir>/rsmgmt/config/application-config.properties:

vault.authentication=APPROLE
vault.app-role=rsmgmt
vault.token=<generated token goes here>
vault.uri=<enter vault uri here>
#Use custom trust-store
vault.ssl.trust-store=classpath:<trust-store-name>
vault.ssl.trust-store-password=<trust-store-password>

Example of <rsmgmt-install-dir>/rsmgmt/config/application-rest.properties:

vault.authentication=APPROLE
vault.app-role=rsmgmt
vault.token=<generated token goes here>
vault.uri=<enter vault uri here>
#Use custom trust-store
vault.ssl.trust-store=classpath:ca.cert.jks
vault.ssl.trust-store-password=resolve

Migrating Existing Secrets

If you had been running Actions Pro for a while before you enabled Vault, chances are you have secrets stored in the Actions Pro database and in .properties files. It is highly recommended to migrate these secrets to Vault as soon as you complete the Vault installation to avoid data loss.

Only perform migration after you’ve successfully completed the Vault installation and verified that it stores Actions Pro secrets properly.

Migrating Secrets Stored on the Filesystem

All secrets stored in .properties files will be automatically migrated on the next config.sh run, which can be as soon as you create a secret in a blueprint.

To manually trigger the migration, run config.sh:

<actions-pro-home>/bin/config.sh

Migrating Secrets Stored in the Database

For migrating secrets stored in the Actions Pro database, such as secrets entered in the Actions Pro UI, Actions Pro provides a migration script.

To migrate all database secrets to RSKeyservice, run the following command:

<actions-pro-home>/rsmgmt/bin/run.sh -f mgmt/MigrateSecrets

You can expect to see output similar to the one shown below on successful migration. The output depends on what secrets are present in your database. In this example, an EWS Address password was migrated to Vault.

Vault instance RSkeyservice is Running
>  Generate initial token DONE                                                  
>  Configure RSMgmt DONE                                                        

Attempting to Execute Script MigrateSecrets.groovy
Migrating secrets
Authenticated with keyservice
Password for ews address 'ews_test@example.com' migrated to vault

Troubleshooting

This section contains actions and tips that can help you troubleshoot an unsuccessful Vault installation. This information applies only to the Vault distribution prepackaged with Actions Pro. To troubleshoot client-maintained Vault deployments, refer to the official Vault documentation.

Running the Installation Script in Debug Mode

Re-running the Vault installation in debug mode can help you identify the step at which the installation fails.

Stop any running instances of Vault.
These commands might not run successfully if the service was not properly installed but they will still clear any lock files that might have been created.
```
<actions-pro-home>/rskeyservice/bin/stop.sh
<actions-pro-home>/rsvault/bin/stop.sh
```
Ensure no Vault instances are running locally.
1. Find out the process' or processes' PID.
```
ps -ef | grep 'vault'
```
2. Kill the process or processes.
```
kill -9 <PID from ps command>
```
Run the install script in debug mode and check the output for error messages.
note
Debug mode results in verbose output printed out in the terminal and in files in plain-text. It is a good idea to clear up after identifying the issue and before re-attempting the installation.
```
bash -x <actions-pro-home>/rsvault/bin/run.sh
```

The error messages you might see fall in one of the following categories:

Unresolvable address—If the error relates to the server name not being resolved, try the following remediation steps:
- Ensure that you have entered the full FQDN of the server and that it is resolvable through DNS or /etc/hosts.
- Double-check the following <actions-pro-home>/rsmgmt/config/blueprint.properties values:
```
RSKEYSERVICE_HOST
rskeyservice.hcl.nostanza.api_addr=https://${RSKEYSERVICE_HOST}:${RSKEYSERVICE_PORT}
rskeyservice.hcl.nostanza.cluster_addr=https://${RSKEYSERVICE_HOST}:${RSKEYSERVICE_CLUSTER_PORT}
rskeyservice.hcl.listener.address=${RSKEYSERVICE_HOST}:${RSKEYSERVICE_PORT}
```
- Double-check that the hostname and the port number are set correctly in <actions-pro-home>/rskeyservice/config/rskeyservice.hcl as api_addr, cluster_addr, and in the listener stanza.
Port already in use—If the error relates to a port number already being in use, try the following remediation steps:
- Change the port to an available one by modifying the respective <actions-pro-home>/rsmgmt/config/blueprint.properties property.
```
RSKEYSERVICE_PORT
RSVAULT_PORT
```
- Ensure that the following address properties in the same file are either using a dollar reference to RSKEYSERVICE_PORT and RSVAULT_PORT or an explicit correct port value.
```
rsvault.hcl.listener.address
rsvault.hcl.nostanza.api_addr
rskeyservice.hcl.nostanza.api_addr
rskeyservice.hcl.nostanza.cluster_addr
rskeyservice.hcl.listener.address
```
Directory issues—If the error relates to a directory missing or missing permissions, try the following remediation steps:
- Double-check the directory list in Installing Vault Locally: Prerequisites. Ensure that all subdirectories are present and that all shell scripts inside them provide read, write, and execute permission to your user.
Unknown issue—If the error does not relate to the aforementioned categories, contact Actions Pro support.

Clearing Up the Installation

Running bash with the -x flag prints the output in plain-text in the terminal and in files, which might include sensitive information. Thus, it is recommended to do a clean-up after identifying the issue and before reattempting the Vault installation.

If you already have secrets stored in Vault, run the following command to clear the recovery (master) password and the public and private keys.

# Clear Master password and public/private keys
cd <actions-pro-home>/
rm -rf rsvault/config/certs/*

If you don’t have any important data stored in Vault, run these three commands to clear all data.

cd <actions-pro-home>/
rm -rf rsvault/data/*
rm -rf rskeyservice/data/*
rm -rf rsvault/config/certs/*

In both cases, it is recommended to clear up your terminal window and scroll buffer.

clear

Vault Administration

Starting Vault After Reboot

Normally, RSVault does not need to be stopped, even when restarting Actions Pro components. If, however, RSVault stops unexpectedly, such as when having insufficient disk space, out of memory issues, or when rebooting the server machine, you need to provide the recovery (master) password to start and unseal it. This ensures that a malicious user cannot unseal RSKeyservice without authentication.

Always follow this succession when restarting RSVault or RSKeyservice will remain in sealed state:

Ensure all Actions Pro and Vault services are stopped.
Start RSVault, entering the recovery (master) password when prompted.
Start RSKeyservice.
With RSVault running in the background, RSKeyservice will auto-unseal.
Start all Actions Pro services.

See Vault Administration for instructions on using start and stop commands.

Start or Set Up Vault

Name

rsvault/bin/run.sh - start, unseal, or set up Vault

Synopsis

rsvault/bin/run.sh [-p][-d domain-name][-t time-to-live]

Description

The run.sh script is used to:

Start and unseal RSVault and RSKeyservice, in this order, as well as regenerate any expired tokens.
When run for the first time, set up and install RSVault and RSkeyservice, including creating a recovery (master) password.

Options

-p

Prompts for password during Vault installation. Use this flag only when running the script for the first time. Without the option, the script autogenerates a random password.

-d <domain-name>

Specifies the domain name of the server as an argument. This flag is required when running the script for the first time. It is not required after RSVault is already installed.

-t <time-to-live>

Sets the time that the short-lived token generated during auto-configuration will be valid for. This flag is optional. Without it, the time is set to 30 minutes. The duration must be short to boost security. The token is written to the rsmgmt/config/application-*.properties files. Format: <number><time unit> where time unit is m for minutes (Example: 30m).

Stop RSVault

Name

rsvault/bin/stop.sh - stop the RSVault instance of Vault

Synopsis

rsvault/bin/stop.sh

Description

The stop.sh script is used to stop the RSVault instance of Vault and other shell programs used to auto-unseal RSKeyservice.

RSVault does not need to be stopped when restarting Actions Pro components. Run this command when performing server machine shutdown or other planned maintenance. In those cases, always stop RSKeyservice before stopping RSVault.

Stop RSKeyservice

Name

rskeyservice/bin/stop.sh - stop the RSKeyservice instance of Vault

Synopsis

rskeyservice/bin/stop.sh

Description

The stop.sh script is used to stop the RSKeyservice instance of Vault. Always stop RSKeyservice before stopping RSVault.

Start RSKeyservice

Name

rskeyservice/bin/run.sh - start the RSKeyservice instance of Vault

Synopsis

rskeyservice/bin/run.sh

Description

The run.sh script is used to start the RSKeyservice instance of Vault.

Run this command to start RSKeyservice while restarting other Actions Pro components or if RSKeyservice crashes. If RSVault is running in the background, RSKeyservice will be auto-unsealed on start. If RSVault is not running, RSKeyservice will remain sealed.

Check Vault Status

Name

rskeyservice/bin/status.sh - check the initialization and seal status of RSKeyservice

Synopsis

rskeyservice/bin/status.sh

Description

The status.sh script returns the initialization and seal status of RSKeyservice. Run this command to check if Vault is up and running.

Example

rskeyservice/bin/status.sh

Checking init status of RSKeyservice:Initialized DONE                        
Checking seal status of RSKeyservice:Unsealed DONE

Vault Configuration Files

The configuration files described in this section only apply to the prepackaged Vault distribution coming with Actions Pro.

The properties in the next table are available in the <actions-pro-home>/rsmgmt/config/blueprint.properties file.

Property Name	Default Value	Description
resolve.rsvault	false	When set to true, it generates the `<actions-pro-home>/rsvault/config/rsvault.hcl` file based on the `rsvault.hcl.*` blueprint properties where you can make advanced Vault settings. See the official Vault documentation for HCL file options.
resolve.rskeyservice	false	When set to true, it generates the `<actions-pro-home>/rskeyservice/config/rskeyservice.hcl` file based on the `rskeyservice.hcl.*` blueprint properties where you can make advanced Vault settings. See the official Vault documentation for HCL file options.
RSKEYSERVICE_HOST	empty	The FQDN of the Vault machine.
RSKEYSERVICE_PORT	8200	Port used by Actions Pro components to access Vault. When using remote components, this port must be opened for network traffic.
RSKEYSERVICE_CLUSTER_PORT	8201	When using a cluster of Action nodes, this port is used by the Vault instance to communicate with each other.
RSKEYSERVICE_LOG	OFF	Log verbosity level for Vault logging within RSRemote and RSMgmt log files. Possible values: OFF, INFO, DEBUG, TRACE. Recommended level for production: WARN or INFO.
RSVAULT_PORT	8300	Port used for local loopback communication between RSVault and RSKeyservice.

The properties in the next table are available in each of the following files:

<actions-pro-home>/rsmgmt/config/application-config.properties
<actions-pro-home>/rsmgmt/config/application-rest.properties
<actions-pro-home>/rsremote/config/application.properties

The properties are autoconfigured if the RSRemote and RSMgmt services are on the same server as the RSVault and RSKeyservice services. In the case of standalone RSRemote and RSMgmt instances, you must manually replicate the values across all of them.

Property Name	Default Value	Description
vault.authentication	APPROLE	Sets the Vault authentication engine. Ensure that this value is always set to APPROLE. No other authentication type is supported.
vault.app-role	rsremote or rsmgmt, depending on the file	Vault role name for the component. It can be either `rsremote` or `rsmgmt`.
vault.app-role.role-id	empty	Only for RSRemote Vault Role ID for the RSRemote role.
vault.uri	empty	RSKeyservice API URL
vault.ssl.trust-store	classpath:ca.cert.jks	Do not modify
vault.ssl.trust-store-password	resolve	Do not modify
vault.token	short-lived token value	Only for RSRemote Replace with the short-lived Vault token generated using the RSMgmt role. Refresh every time RSMgmt is stopped or the token expires (whichever comes first).
server.port	8081	Only in application-rest.properties Port used for local web traffic between RSView and RSMgmt. Does not need to be opened to external web traffic.

Vault Policies and Roles

Actions Pro internally creates three policies and two roles in Vault as described below.

Policy	Roles	Capabilities
resolve-admin	rsmgmt	The role can read and write secrets and create Role ID and Secret ID for RSRemote.
resolve-user	rsremote	The role can read secrets.
resolve-initial		The role cab read and write secrets and Role ID and Secret ID for RSRemote.

Overview​

Benefits​

Vault Workflow​

Vault Concepts​

Auto Unseal​

Recovery (Master) Password​

Secret Engine and Path​

Authentication and Authorization​

Vault Policies & Roles​

Compatibility Matrix​

Gateway Passwords Stored in Vault​

Gateway Filter Secrets Stored in Vault​

Making a Fresh Vault Installation​

Installing Vault Locally​

Prerequisites​

Procedure​

Next steps​

Configuring Remote RSRemote and RSMgmt Instances​

Using an Existing Vault Installation​

Configuring Vault​

Configuring Actions Pro​

Configuring RSRemote​

Configuring RSMgmt​

Migrating Existing Secrets​

Migrating Secrets Stored on the Filesystem​

Migrating Secrets Stored in the Database​

Troubleshooting​

Running the Installation Script in Debug Mode​

Clearing Up the Installation​

Vault Administration​

Starting Vault After Reboot​

Start or Set Up Vault​

Name​

Synopsis​

Description​

Options​

Stop RSVault​

Name​

Synopsis​

Description​

Stop RSKeyservice​

Name​

Synopsis​

Description​

Start RSKeyservice​

Name​

Synopsis​

Description​

Check Vault Status​

Name​

Synopsis​

Description​

Example​

Vault Configuration Files​

Vault Policies and Roles​

Overview

Benefits

Vault Workflow

Vault Concepts

Auto Unseal

Recovery (Master) Password

Secret Engine and Path

Authentication and Authorization

Vault Policies & Roles

Compatibility Matrix

Gateway Passwords Stored in Vault

Gateway Filter Secrets Stored in Vault

Making a Fresh Vault Installation

Installing Vault Locally

Prerequisites

Procedure

Next steps

Configuring Remote RSRemote and RSMgmt Instances

Using an Existing Vault Installation

Configuring Vault

Configuring Actions Pro

Configuring RSRemote

Configuring RSMgmt

Migrating Existing Secrets

Migrating Secrets Stored on the Filesystem

Migrating Secrets Stored in the Database

Troubleshooting

Running the Installation Script in Debug Mode

Clearing Up the Installation

Vault Administration

Starting Vault After Reboot

Start or Set Up Vault

Name

Synopsis

Description

Options

Stop RSVault

Name

Synopsis

Description

Stop RSKeyservice

Name

Synopsis

Description

Start RSKeyservice

Name

Synopsis

Description

Check Vault Status

Name

Synopsis

Description

Example

Vault Configuration Files

Vault Policies and Roles