Sunday, April 5, 2015

High Availability for PKI in 8 Simple Steps

PKI Appliance takes HA Clustering to the next level

Historically, setting up and maintaining High Availability (HA) setups for PKI, required a lot of knowledge about the database in use and how it would interact with HA.

Lately, the packaged PKI product PrimeKey PKI Appliance (running EJBCA and/or SignServer), manages to hide the HA complexity and makes HA easy to setup, robust during operation and simple to handle during recovery. Clustering a PKI has never been easier!

On the PKI Appliance we define the meaning of High Availability as being able to keep services running non-stop, with full data integrity for all included applications.

High Availability Fail Scenarios

Real High Availability requires a cluster setup of three nodes. The reasoning here is that in the case of a node failure, the system should continue to operate on its own, without any administrator measures needed.

Hot standby with manual fail-over is a two nodes cluster setup, which to a lesser extent reduces administration efforts for running systems. Here, the assumption is that depending on which node eventually fails, the system either continues to operate or needs to have a button clicked.

A main site hosting two PKI Appliances and a DR site hosting one PKI Appliance.
With its handy instructions and simple web GUI configuration, PrimeKey PKI Appliance makes it possible to quickly set up a cluster for Disaster Recovery (includes a main site and a remote site).

Technical details

High Availability with the PKI Appliance is implemented using database replication between cluster nodes connected within a network. Using active-active cluster technology means that all the nodes can be fully utilized in the cluster setup. The implementation uses regular network connectivity over the application interface for all cluster communication, which means that cluster nodes don’t have to be placed physically close, as long as they have good network connectivity.

The drawback of most cluster technologies, is that a node cannot distinguish between a failure of another node (such as a broken power supply) and disrupted network connectivity (such as a broken network switch). So in order to avoid a situation where cluster nodes start to operate independently and get diverging data sets, a so called split brain, the cluster nodes take a vote and cease to operate unless they are part of the majority of connected nodes. This way there is only a single data set that can be updated at a specific time.

In case of a temporary network failure, disconnected nodes automatically synchronize their data as soon as the network is back, then start to operate again as if nothing had happened.

Howto: Setting up a 2 node cluster (Hot standby with fail-over)

Using the Appliance Web Configurator it's easy to set up a two node cluster in a few easy steps. Setting up a three node PKI cluster (real High Availability) works the same way, and clustering a PKI has never been easier! 
  1. Make an installation according to the standard installation procedure on the initial Appliance node.
  2. On the initial node, go to the cluster tab in the Appliance Web Configurator. To start with it shows the Appliance running as a single instance.
    The PrimeKey PKI Appliance Web Configurator showing cluster configuration of a single instance.
  3. Still on the initial node, add a connection (IP address) to where the second node’s application interface will be. The Appliance will now configure the cluster settings and produce a setup bundle for the second node.
  4. Adding a second cluster node IP in the PKI Appliance.
    PKI Appliance cluster configuration going on.
    PKI Appliance cluster configuration completed.

  5. Download the setup bundle for the second node by clicking Create and Download.
    From the first node, downloading the configuration package for the second cluster node.
  6. Factory reset the second node and connect to the Web Configurator.
    PrimeKey PKI Appliance initial screen after Factory Reset.

  7. On the second node, select the Connect to cluster option and upload the setup bundle.
  8. Uploading cluster setup bundle on the PKI Appliance.

    When processing an uploaded cluster setup bundle, a domain master secret must be specified.

    Date and time settings when connecting PKI Appliance cluster node.

    PrimeKey PKI Appliance Cluster configuration preview on second node.

    Ready to begin installation of second Appliance cluster node.
  9. After the installation has been completed, you should be able to manage the new node using the same login credentials as with the first one.
  10. The two nodes are now installed and can be used. Status of the second cluster node shows up as Connected.
  11. PrimeKey PKI Appliance Web Configuration cluster status with two nodes.

More information
Basic information on EJBCA Enterprise PKI and PKI Appliance is available here.
EJBCA is a registered trademark of PrimeKey Solutions AB in the EU, the United States, Japan and certain other countries.

Monday, February 9, 2015

Peer Systems in EJBCA Enterprise 6.3.0 - introduction and tutorial

We hope that you'll get as excited as we are about our latest new EJBCA feature: Peer Systems. We'd like to take this blog post to tell you all a little about what this means and how EJBCA Peer Systems will improve your PKI, as well as give a simple tutorial on how it works.

Peer Systems is a protocol for communicating between EJBCA nodes securely. Its purpose is to allow propagation of data and information in a synchronous fashion, without requiring a safe connection or an isolated network. In previous iterations of EJBCA the only means of communication were either the WebService API or direct database publishing. Web services is on one hand too slow to use for data heavy applications like very high speed certificate publishing, while database publishing needs database specific security measures or a closed network, seldom an option for OCSP.

Two requirements have prompted Primekey to move towards developing a brand new feature set:
  1. The need from the PKI Appliance platform to be able to set up OCSP responders without redeploying the application, requiring steps to be performed via the CLI or set up special database rules.
  2. The value of enabling our users to spend less time digging into the server configuration, setting up database connections. VA machines can often be physically or geographically inaccessible from their respective CAs, and Peer Systems is a step in the right direction making things easier to set up using only the GUI.
Note that this feature is for EJBCA Enterprise customers only. If you'd like to find out how to enroll, please contact us.

Initial Concepts

This tutorial will cover an installation involving two machines in a PKI setup, a Certificate Authority (CA) and a Validation Authority (VA) serving as an OCSP responder for the CA. We will be building on concepts introduced in earlier versions of EJBCA 6, so if you're not familiar with the creation of End Entities or OCSP Key Bindings, feel free to check our User Guide.

Establishing Trust

For this tutorial, all communication will be initiated on the CA, so our connection will be one way. The first thing we need to do is to establish trust between the two machines (CA and VA), and we will do so by creating an Authentication Keybinding on the CA.
  1. Click on the Internal Key Bindings item in the menu and choose the AuthenticationKeyBinding tab.
  2. Create an authentication keybinding using the crypto token of your choice. The crypto token chosen will be the one providing keys for the TLS connection.
  3. Return to the AuthenticationKeyBinding overview page. Notice that your newly created keybinding isn't enabled, and doesn't have a certificate. Let's fix that.
  4. Click on the CSR button in the Action column.
  5. On the CA machine create an End Entity and issue the keybinding certificate. Make sure that Key Usage: Digital Signature and Key Encipherment and Extended Key Usage: Client Authentication are set in the Certificate Profile.
  6. Return to the AuthenticationKeyBinding overview page on the CA machine. Import the certificate you got in the previous step to your authentication keybinding by clicking the Update button.
  7. Enable your keybinding.
A properly set up Authentication Keybinding

Setting up the Outgoing Peer on the CA

Now we'll talk about how to set up our CA and VA as Peers.
  1. Incoming connections are disabled by default, so first of all on the VA, click on Peer Systems in the admin menu and then click on the Allow incoming connections checkbox.
  2. Now go to the CA, and enter the Peer Systems page here as well.
  3. Click on the Add button under Outgoing Peer Connectors.
  4. Create new Peer, making sure to enter the VA's address (matching the VA's server side TLS certificate) and the selected authentication keybinding.
A newly created Outgoing Peer on the CA

Setting up the Incoming Peer on the VA

  1. Click the Ping button for the Outgoing Peer (on the CA) and you'll get the reply "Unable to connect to peer. Unauthorized." Not to worry, you simply have to add rights on the VA.
  2. Import the CA certificate of the CA (used to issue the certificate for the End Entity on the CA in the steps above) as an External CA.
  3. Go to the VA, and under the Incoming Connections section you should see the incoming connection from the CA. To the right you'll see the button Create Authorization (see screen shot for required options).
  4. Either add the incoming peer to an existing role or go right ahead and create a new one.
  5. In the next screen, choose the required rights for the incoming connection and click on Create Authorization.
  6. Returning to the Peer Systems screen, there should be a newly created role under the First matching role column of your created peer.
  7. Returning to the CA, you should be able to ping the VA as before, now receiving a response.
  8. Congratulations, you've established trust.
Authorizing an Incoming Peer

Setting up a Validation Authority Peer Publisher

The last step is to set up a Validation Authority Peer Publisher, which happily works just like the old VA Publisher.
  1. On the CA, go to Publishers and create a new publisher.
  2. Choose Validation Authority Peer Publisher as a type and choose the newly created peer as a Peer System.
  3. Click Save and Test Connection and you're done!
You've now set up VA Publishing between CA and VA, without having to grant any database rights or allowing direct database access over network to your VA. Cheers!

More information
Basic information on EJBCA Enterprise PKI is available here.
EJBCA is a registered trademark of PrimeKey Solutions AB in the EU, the United States, Japan and certain other countries.

Friday, December 19, 2014

Improved crypto token configuration and re-keying in SignServer 3.6

SignServer 3.6 brings great improvements in both crypto token configuration and management. There is now the advantage of single crypto token configuration and, related to this, re-keying has become far more convenient.

Single crypto token configuration

As crypto token configuration in SignServer 3.6 is now done in a single spot, management has become significantly simpler.

In previous versions, the necessity to configure keystores (soft or PKCS#11) in each worker resulted in duplicated configuration. The repetitive steps couldn't be avoided even when all workers shared the same keystore or HSM slot.

From version 3.6 there is separation between worker and crypto token configuration, which eliminates duplicate configuration entirely. You simply set up a single crypto worker (a special type of worker). Other workers, using key-pairs in the same keystore or HSM slot, will then point to the singly configured crypto worker.

One crypto token and multiple workers

Crypto token referenced from another worker

Rekeying without production stop

Achieving rekeying without production stop used to be a bit of a hassle. Special care had to be taken to ensure that at least one signer was available to handle emerging requests.

To everyone's satisfaction rekeying has improved as of version 3.6. Crypto workers now stay active as the keying process does not change the configuration of the crypto token. Entirely superfluous, special measures to avoid production halt are now history.

In practice, the process of rekeying (generating a new key-pair and getting a new certificate for it)  always has to be done before the signer's certificate expires. In the Admin GUI every operation is accessed in the tool bar, from the corresponding buttons (screen shot below) which basically follow these five steps:
  1. Activate the worker
  2. Renew the key-pair
  3. Test the key-pair
  4. Generate CSR
  5. Install certificates

AdminGUI toolbar

Step 2 previously put the worker in an off-line state but with the new crypto worker configuration this is not the case, and it can thus stay active at all times, greatly facilitating re-keying in production.

Trying it out

Let's configure a PDF signer which uses keys provided by a crypto worker.

Set up a new crypto worker configuration by copying the sample config file:
$ cp doc/sample-configs/ \

Edit by changing the keystore path to point to the sample keystore provided:

Apply the configuration (replace WORKERID with ID printed):
$ bin/signserver setproperties
$ bin/signserver reload WORKERID

Activate the crypto token in the crypto worker (password is "foo123"):
$ bin/signserver activatecryptotoken CryptoTokenP12

Set up a new PDF signer using your newly activated crypto worker:
$ cp doc/sample-configs/\

Set to point to the crypto worker:

Apply the configuration (replace WORKERID with ID printed):
$ bin/signserver setproperties
$ bin/signserver reload WORKERID

View the status of the workers:
$ bin/signserver getstatus brief all
Current version of server is : SignServer CE 3.6.2

Status of CryptoWorker with id 1 (CryptoTokenP12) is:
   Worker status : Active
   Token status  : Active

Status of Signer with id 2 (PDFSigner) is:
   Worker status : Active
   Token status  : Active
   Signings      : 0

You can also get more information by writing "complete" instead of "brief" or by using the Admin GUI:
$ bin/signserver-gui

More information

Basic information on SignServer Enterprise is available here. For entire documentation, see

Wednesday, September 17, 2014

Comparing EAC 2.10 to the previous 1.11

EAC 2.10 CV certificates are supported as of EJBCA Enterprise 6.1.0. This is an upgrade from the previously supported EAC 1.11. So, what's new in EAC 2.10 and why would you even need EAC certificates at all?

About Extended Access Control

Extended Access Control (EAC) is the technology used to protect fingerprints (or irises) in EU member states' e-passports, but in some eIDs other information is protected as well. As used in the EU, EAC is defined in the BSI technical specification TR-03110. Based on public key infrastructure (PKI), EAC works as an authentication mechanism between the terminal and the ePassport/eID chip. The specification includes several steps such as both chip- and terminal authentication, but only terminal authentication concerns PKI.

In an EAC process, each inspection system or terminal is given a CV Certificate (CVC) which gives the inspection system privileges to perform specific actions, or to read certain data from the e-passport or eID chip. If the inspection system cannot authenticate itself towards an e-passport or eID chip, the latter will deny access to the data or function.

The news in EAC 2.10

So what are the differences in EAC 2.10 compared to EAC 1.11? As mentioned, 1.11 was designed for ePassports only. With EAC 2.10 the use cases have been extended to other security documents, such as eIDs. In the context of EJBCA, we are primarily interested in terminal authentication (the part where the PKI implements something). Here, the main difference in EAC 2.10 is additional access control templates, used to specify new types of terminals relating to eIDs and digital signatures. There is also room for extensions, but since none have been specified at this stage and no sample certificates are using extensions, this is mostly a placeholder for future additions.

The CVC Terminal Type options illustrated
The news in EAC 2.10 may most easily be illustrated by a few screenshots of the Admin GUI in EJBCA Enterprise 6.2.0.
In EAC 1.11 the only CVC terminal type available  was Inspection System (allowing you to select DG3 and/or DG4). This option still provides an inspection system able to read fingerprints, and/or iris data, from e-passports. In the first image however, you can see the EAC 2.10 CVC terminal type (upper left corner) with two new options next to Inspection System: Authentication Terminal and Signature Terminal. Click the image to enlarge!

EAC 2.10 CVC terminal type options in EJBCA Enterprise 6.2.0.
CVC terminal type

So, what hides behind the new Authentication Terminal and the Signature Terminal options?

The Authentication Terminal can be given fine grained (read and write) control to any data group on an eID, as well as control to additional eID functions such as PIN Management and Age Verification.

Pic 1: CVC access rights of the Authentication Terminal in EAC 2.10, EJBCA Enterprise 6.2.0. Click to enlarge!
Authentication Terminal (pic 1)

Pic 2: CVC access rights of the Authentication Terminal in EAC 2.10, EJBCA Enterprise 6.2.0.
Authentication Terminal (pic 2)

The Signature Terminal on the other hand, can either be given the role of an Accreditation Body or a Certification Service Provider, where each can be allowed to generate signatures and/or qualified signatures.

Pic 1: Body or a Certification Service Provider options in the Signature Terminal in EAC 2.10, EJBCA Enterprise 6.2.0.
Signature Terminal (pic 1)

Pic 2: Body or a Certification Service Provider options in the Signature Terminal in EAC 2.10, EJBCA Enterprise 6.2.0.
Signature Terminal (pic 2)

To sum up the additional features in EAC 2.10:
  • New terminal types for using eIDs.
  • New access controls for mentioned terminals.
  • Possibility for extensions.

More information

Basic information on EJBCA Enterprise PKI is available here.
EJBCA is a registered trademark of PrimeKey Solutions AB in the EU, the United States, Japan and certain other countries.

About the author
Tomas Gustavsson, CEO/CTO of PrimeKey, founder of EJBCA
Contact me at tomas(at)
Follow me on Twitter.
Follow PrimeKey on Twitter.

Tuesday, June 3, 2014

Certificate Transparency with EJBCA Enterprise

As of the release of version 6.0.4, EJBCA Enterprise now supports Certificate Transparency RFC6962.

Google Initiative to Increase https Security

Certificate Transparency (CT) is an initiative by Google to increase security and auditability of the https ecosystem itself. These important aims are accomplished by having CAs (CA services using a software such as EJBCA) issue TLS certificates, which are transparently auditable and exactly reveals which certificates have been issued. The purpose of CT is to create public audit logs of all certificates issued by the public SSL/TLS CAs. For example, this means the owners of a certain web domain can monitor CT Log servers, to see if there are any unknown or suspicious TLS certificates issued for their domain. In addition, the presence of audit records is planned to be required for EV certificates in Google Chrome from February 2015. And perhaps later on for other web browsers and non-EV certificates as well.

Note that Certificate Transparency is only relevant for CAs issuing public SSL/TLS certificates; other types of CAs mustn't use CT at this time. More information can be found on the CT website.
The specification of Certificate Transparency is still being discussed, and a follow up to RFC6962 will likely be available in the not so distant future, including (among other things) a way to handle private subdomains.

How EJBCA Enterprise perceives Certificate Transparency

From the CA's angle, CT works by publishing pre-certificates from itself to the log servers; then in immediate response, Signed Certificate Timestamps (SCTs) are retrieved. Certificate and timestamp exchange is done within a single operation, so requesting an SCT for a certificate also publishes it. The resulting SCTs can then be sent to end-users in any of three different ways: 1) in a certificate extension (embedded when issuing the real certificate), 2) in a stapled OCSP response extension, 3) and/or in a TLS extension (yes, EJBCA does support all of those methods, including any combination of the three).

The EJBCA Admin Guide describes how to configure EJBCA Enterprise for one or more of the above methods.

For more information, contact:
Tomas Gustavsson, CEO of PrimeKey, founder of EJBCA
Follow me on Twitter

Tuesday, April 8, 2014

Securing 4690 POS Terminals with PKI

Point of Sale (POS) terminals, communicate not only to all of the cash registers of a store, but also to central location. In order to secure this particular type of communication, and secure other items on the terminals as well, you need to add keys and certificates managed by a PKI.
To enable POS terminals to enroll for certificates specifically against an EJBCA® Enterprise PKI server, C2 Company and PrimeKey have successfully installed and used the EJBCA Enterprise clientToolBox on several Toshiba 4690 POS terminals.

About POS and Toshiba 4690

Commonly used in POS terminals, the Toshiba 4690 is an operating system which is basically IBM Java and DOS based, using specific 4690 DOS commands. So, if you are used to common computing with Linux or Windows, the Toshiba will appear a bit alien. However, taking the trouble to introduce PKI to the POS system, will enable any user of the 4690 terminals to trade on both strong authentication and encryption.

Steps to Enroll and Renew Certificates

Having IBM Java on board, helped us run EJBCA clientToolBox on Toshiba 4690, and made it possible to enroll and manage certificates. The Toshiba required some figuring out of which new commands to run, but once that was (elegantly) performed by C2, EJBCA clientToolBox was running ever so nicely.
The following steps are only a single example of how you can use the certificate management capability on POS terminals. In fact, there are limitless possibilities.
Use the Native DOS commands to generate keys and CSRs. Once those are generated you can enroll for a certificate against EJBCA, using clientToolBox.
To enroll new terminals for the first time when they are installed in a store, you can use a pre-installed certificate on the POS terminal image. To enroll for the real terminal certificate you use the pre-istalled certificate temporarily, to enable access from the POS terminal to EJBCA. The pre-installed certificate does not need any admin access in the EJBCA system. Do the following:
  • Install a new POS terminal with an image, including a pre-installed communication certificate on the image.
  • Register the POS terminal (with serial number or similar) in EJBCA, and you'll receive a one-time enrollment code.
  • Generate keys and a CSR (to the csr.pem file) on the POS terminal, by using DOS commands.
  • Submit the CSR to EJBCA, to get the signed certificate back, using this command:
./ EjbcaWsRaCli pkcs10req terminalSerial enrollmentCode csr.pem PEM NONE certificate.pem
Now you are ready to remove the pre-installed communication certificate from the image, to finally make the image secure, only by its rightful individual certificate. However, to block the POS terminal from accessing store systems (in case the terminal gets stolen or hacked) the latter step can be revoked.
A single PKI administrator action is found in the above work-flow; registering the POS terminal in EJBCA. This is done in order to authenticate the initial enrollment and make sure that no unauthorized terminal receives real certificates, that is, illegitimate access to store systems. With the final certificate installed, the terminal can automatically renew it before expiry, requiring no PKI administrator action during daily operations.

More information

Fore more information, or to get in touch with C2 for help with securing your POS terminals, contact Chris or me. Cheers!
Chris Chu
chris at
Tomas Gustavsson
tomas at

Wednesday, March 26, 2014

EJBCA Enterprise 6.1.0 released

The PrimeKey EJBCA® team is happy to announce the release of EJBCA Enterprise 6.1.0.
This release resolves several issues, with a few highlights: Increased performance through OCSP improvements; Key Recovery improvements; support for EAC 2.10 (ePassport) access control templates.
Running on the latest technology platforms, EJBCA Enterprise v.6 is so flexible it is suitable for any organization, cloud, social or mobile system. Faster, more resource efficient, more secure and more user friendly than ever.

EJBCA Enterprise *6.1.0* release notes

A maintenance release containing 32 new features and improvements, below a selection of the most noteworthy:
  • New features

    • New OCSP features related to RFC 6960, minimizing size of OCSP responses.
    • Implemented OCSP signing algorithm selection from client requested algorithms.
    • CVC certificate profiles (ePassport PKI) now supports EAC 2.10 access control templates.
  • Improvements

    • OCSP improvements with more cache control settings.
    • Improvements to Key Recovery, enabling encryption key rollover and providing more information about encryption keys.
    • Ability to build and install EJBCA on Windows platforms.
    • The ManagementCA created during default install, now uses SHA256WithRSA.
    • EJBCA compiles cleanly with Java 8, WildFly 8 and Glassfish 4 (running on those platforms however, is not yet supported).
    • EJBCA can now use certificate serial number longer than 64 bits.
    • Many reported minor issues have been fixed, as well as minor GUI improvements.

More information

Basic information on EJBCA Enterprise PKI is available here. For entire technical details view the changelog in the PrimeKey Issue Tracker.
EJBCA is a registered trademark of PrimeKey Solutions AB in the EU, the United States, Japan and certain other countries.