Certificate management in Zowe API Mediation Layer
Review details of certificate management in Zowe API Mediation Layer including running on localhost, Zowe runtime on z/OS. This topic also addressing key information about the API ML truststore and keystore, and API ML SAF Keyring.
- Running on localhost
- Zowe runtime on z/OS
- API ML truststore and keystore
- API ML SAF Keyring
Running on localhost
How to start API ML on localhost with full HTTPS
The https://github.com/zowe/api-layer repository contains pre-generated certificates that can be used to start API ML with HTTPS on your computer. The certificates are not trusted by your browser so you can either ignore the security warning or generate your own certificates and add them to the truststore of your browser or system.
For more information about certificates, see TLS Certificates for localhost.
Note: When running on localhost, only the combination of using a keystore and truststore is supported.
Certificate management script
Zowe API Mediation Layer provides a script that can be used on Windows, Mac, Linux, and z/OS to generate a certificate and keystore for the local CA, API Mediation Layer, and services.
This script is stored in
zowe/zowe-install-packaging repository bin/apiml_cm.sh.
It is a UNIX shell script that can be executed by Bash or z/OS Shell.
Generate certificates for localhost
Use the following procedure to generate certificates for localhost.
Follow these steps:
- Clone the
zowe-install-packagingrepository to your local machine.
- Place the
bin/apiml_cm.shscript into the
scriptsdirectory in your API Mediation Layer repository folder
- Use the following script in the root of the
api-layerrepository to generate certificates for localhost:
scripts/apiml_cm.sh --action setup
This script creates the certificates and keystore for the API Mediation Layer in your current workspace.
Generate a certificate for a new service on localhost
To generate a certificate for a new service on localhost, see Generating certificate for a new service on localhost.
Add a service with an existing certificate to API ML on localhost
For more information about adding a service with an existing certificate to API ML on localhost, see Trust certificates of other services.
Service registration to Discovery Service on localhost
To register a new service to the Discovery Service using HTTPS, provide a valid client certificate that is trusted by the Discovery Service.
Zowe runtime on z/OS
Certificates for the API ML local CA and API ML service are managed by installing the Zowe runtime on z/OS. Follow the instructions in Installing the Zowe runtime on z/OS.
There are two ways to set up certificates on a z/OS machine:
- Certificates in SAF keyring
- Certificates in UNIX files (keystore and truststore)
The Configuring PKCS12 certificates and Configuring JCERACFS certificates in a key ring contain instructions about how to set up certificates during installation. Follow the procedure in the applicable section described in this article during installation.
Import the local CA certificate to your browser
Trust in the API ML server is a necessary precondition for secure communication between Browser or API Client application. Ensure this trust through the installation of a Certificate Authority (CA) public certificate. By default, API ML creates a local CA. Import the CA public certificate to the truststore for REST API clients and to your browser. You can also import the certificate to your root certificate store.
If a SAF keyring is being used and set up with
ZWEKRINGJCL, the procedure to obtain the certificate does not apply. It is recommended that you work with your security system administrator to obtain the certificate. Start the procedure at step 2.
The public certificate in the PEM format is stored at
<KEYSTORE_DIRECTORY>is defined in a customized
<RUNTIME_DIR>/bin/zowe-setup-certificates.envfile during the installation step that generates Zowe certificates. The certificate is stored in UTF-8 encoding so you need to transfer it as a binary file. Since this is the certificate to be trusted by your browser, it is recommended to use a secure connection for transfer.
Windows currently does not recognize the PEM format. For Windows, use the P12 version of the local_cer.
Follow these steps:
Download the local CA certificate to your computer. Use one of the following methods to download the local CA certificate to your computer:
Use Zowe CLI (Recommended)
Issue the following command:
zowe zos-files download uss-file --binary $KEYSTORE_DIRECTORY/local_ca/localca.cer
Issue the following command:
To verify that the file has been transferred correctly, open the file. The following heading and closing should appear:
Import the certificate to your root certificate store and trust it.
For Windows, run the following command:
certutil -enterprise -f -v -AddStore "Root" localca.cer
Note: Ensure that you open the terminal as administrator. This will install the certificate to the Trusted Root Certification Authorities.
For macOS, run the following command:
$ sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain localca.cer
For Firefox, manually import your root certificate via the Firefox settings, or force Firefox to use the Windows truststore.
Note: Firefox uses its own certificate truststore.
C:\Program Files (x86)\Mozilla Firefox\defaults\prefwith the following content:
/* Enable experimental Windows truststore support */
Generate a keystore and truststore for a new service on z/OS
You can generate a keystore and truststore for a new service by calling the
apiml_cm.sh script in the directory with API Mediation Layer.
Note: This procedure applies to UNIX file keystore and truststore only. For the SAF keyring option, it is recommended that you perform the actions manually using your security system commands.
apiml_cm.sh script in the directory with the API Mediation Layer as in the following example.
bin/apiml_cm.sh --action new-service --service-alias <alias> --service-ext <ext> \
--service-keystore <keystore_path> --service-truststore <truststore_path> \
--service-dname <dname> --service-password <password> --service-validity <days> \
is a unique string to identify the key entry. All keystore entries (key and trusted certificate entries) are accessed via unique aliases. Since the keystore has only one certificate, you can omit this parameter and use the default value
specifies repository of security certificates plus corresponding private keys. The
<keystore_path>is the path excluding the extension to the keystore that is generated. It can be an absolute path or a path relative to the current working directory. The key store is generated in PKCS12 format with the
.p12extension. Ensure that the path is in an existing directory where your service expects the keystore.
contains certificates from other parties that you expect to communicate with, or from Certificate Authorities that you trust to identify other parties. The
<truststore_path>is the path excluding the extension to the trust store that is generated. It can be an absolute path or a path relative to the current working directory. The truststore is generated in PKCS12 format.
specifies the X.509 extension that should be the Subject Alternate Name (SAN). The SAN contains host names that are used to access the service. You need to specify the same hostname that is used by the service during API Mediation Layer registration.
Note: For more information about SAN, see SAN or SubjectAlternativeName at Java Keytool - Common Options.
specifies the X.509 Distinguished Name and is used to identify entities, such as those which are named by the subject and issuer (signer) fields of X.509 certificates.
"CN=Zowe Service, OU=API Mediation Layer, O=Zowe Sample, L=Prague, S=Prague, C=CZ"
specifies the number of days until the certificate expires.
specifies the keystore password. The purpose of the password is the integrity check. The access protection for the keystore and keystore need to be achieved by making them accessible only by the ZOVESVR user ID and the system administrator. The
local-ca-filenameis the path to the keystore that is used to sign your new certificate with the local CA private key. It should point to the
$KEYSTORE_DIRECTORYis defined in a customized
$ZOWE_ROOT_DIR/bin/zowe-setup-certificates.envfile during the installation step that generates Zowe certificates.
Add a service with an existing certificate to API ML on z/OS
The API Mediation Layer requires validation of the certificate of each service that it accessed by the API Mediation Layer. The API Mediation Layer requires validation of the full certificate chain.
Note: This procedure applies only to UNIX file keystore/truststore. For the SAF keyring option, we recommend to perform the actions manually using your security system commands.
Import the public certificate of the root CA that has signed the certificate of the service to the APIML truststore.
Note: Validation fails if a service does not provide an intermediate CA certificates to the API ML. This can be circumvented by importing the intermediate CA certificates to the API ML truststore.
The following path is an example of how to import a public certificate to the API ML truststore by calling in the directory with API Mediation Layer:
bin/apiml_cm.sh --action trust --certificate <path-to-certificate-in-PEM-format> --alias <alias>
Procedure if the service is not trusted
If your service is not trusted, you may receive a response with the HTTP status code 502 Bad Gateway and a JSON response in the standardized format for error messages. The following request is an example of when this error response may occur.
http --verify=$KEYSTORE_DIRECTORY/local_ca/localca.cer GET https://<gatewayHost>:<port>/<untrustedService>/api/v1/greeting
In this example, you receive a similar response:
"messageContent": "The certificate of the service accessed by HTTPS using URI '/<untrustedService>/api/v1/greeting' is not trusted by the API Gateway: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
The message has the key
apiml.common.tlsError, and message number
AML0105. The content explains details about the message.
If you receive this message, import the certificate of your service or the CA that signed it to the truststore of the API Mediation Layer as described previously.
API ML truststore and keystore
A keystore is a repository of security certificates consisting of either authorization certificates or public key certificates with corresponding private keys (PK), used in TLS encryption. A keystore can be stored in Java specific format (JKS) or use the standard format (PKCS12). The Zowe API ML uses PKCS12 to enable the keystores to be used by other technologies in Zowe (Node.js).
API ML SAF Keyring
As an alternative to using a keystore and truststore, API ML can read certificates from a SAF keyring. The user running the API ML must have rights to access the keyring. From the java perspective, the keyring behaves as the
JCERACFKS keystore. The path to the keyring is specified as
safkeyring:////user_id/key_ring_id. The content of SAF keyring is equivalent to the combined contents of the keystore and the truststore.
Note: When using JCEFACFKS as the keystore type, ensure that you define the class to handle the RACF keyring using the
-D options to specify the
The elements in the following list, which apply to the API ML SAF Keyring, have these corresponding characteristics:
The API ML local certificate authority (CA)
- The API ML local CA contains a local CA certificate and a private key that needs to be securely stored.
- The API ML local certificate authority is used to sign certificates of services.
- The API ML local CA certificate is trusted by API services and clients.
The API ML keystore or API ML SAF Keyring
- Server certificate of the Gateway (with PK). This can be signed by the local CA or an external CA.
- Server certificate of the Discovery Service (with PK). This can be signed by the local CA.
- Server certificate of the Catalog (with PK). This can be signed by the local CA.
- Private asymmetric key for the JWT token, alias
jwtsecret. The public key is exported to the
- The API ML keystore is used by API ML services.
The API ML truststore or API ML SAF Keyring
- Local CA public certificate.
- External CA public certificate (optional).
- Can contain self-signed certificates of API Services that are not signed by the local or external CA.
- Used by API ML services.
Zowe core services
- Services can use the same keystore and truststore or the same keyring as APIML for simpler installation and management.
- When using a keystore and truststore, services have to have rights to access and read them on the filesystem.
- When using a keyring, the user of the service must have authorization to read the keyring from the security system.
- Alternatively, services can have individual stores for higher security.
API service keystore or SAF keyring (for each service)
- The API service keystore contains a server and client certificate signed by the local CA.
API service truststore or SAF keyring (for each service)
- (Optional) The API service truststore contains a local CA and external CA certificates.
- A client certificate is a certificate that is used for validation of the HTTPS client. The client certificate of a Discovery Service client can be the same certificate as the server certificate of the services which the Discovery Service client uses.