PKCS, pem, der, key, crt,...

Whenever you setup an application with SSL/TLS, you always find yourself juggling files with specific extensions. To make it worse, these extensions change from one documentation to another... Just to find your way there, here is what they mean.

Cryptography is the art of secrecy. It should only be applied to messages sent on public channels, but it almost seems like it applies to itself as some of its concepts can seem obscure.

For example, when setup an application to use secure communications, we use "SSL / TLS". In fact, initially there was only "SSL". We were happy with it, but it had a few small illnesses that had to be treated. In the euphoria of the moment, the IETF took the opportunity to rename it "TLS".

Technically, SSL is therefore considered obsolete and replaced by TLS but, surely for the sake of Eris, some applications and documentation still use the term SSL (i.e. LDAPs), even though they are up to date and use TLS...

Pexels @ pixabay
Pexels @ pixabay

And it's even worse when you try to setup these applications because they use specific files whose extensions seem to have been randomly chosen from one documentation to another: p7,p10, p12,der , pem,key, crt,cer to name the most common.

Fortunately, the concepts behind all of these extensions are pretty straightforward and once understood, these formats aren't that random anymore.

At the root of the problem there are two types of content: keys (PKCS #8) and certificates (PKCS #12). To encode them, we use a specific syntax (ASN.1) which produces raw binary files (DER) which can then be encoded in base64, which are easier to handle (PEM). Finally, to make it easier to recognize the content of the file, we use more explicit extensions for keys (KEY) and certificates (CRT and sometimes CER).

File types

All today files concern a very specific field of applied cryptography: the storage of certificates and their corresponding private keys.

Certificates

When using asymmetric algorithms, those with a private key (to be kept secret) and a public key (to be disseminated), it is often very practical to add information about its owner to the public key. This information and the key constitute a certificate which can then be signed to create a PKI.

To make softwares understand each other, it was necessary to establish a format to organize this data, the famous X.509 (created in 1988 by the International Telecommunications Union). As always in IT, this format has benefited from several successive improvements, finally formalized and adapted to the Internet by the IETF in successive RFCs.

In parallel with public standardization, the company RSA (holder of the patent on the RSA algorithm) has also defined its own formats for its applications, the famous Public Key Cryptographic Standards. Unsurprisingly, the most widely used were subsequently standardized by the IETF via specific RFCs.

RSA Security @wikipedia
RSA Security @wikipedia

When you handle certificates, you can therefore meet, in the vast majority of cases, the following three contents:

In terms of file extensions, you can therefore find certificates in .p7b files (more rarely .p7 and .p7c) or certificate signing requests in .p10 files.

Private keys

Who says certificate says asymmetric cryptography and therefore private key, which must be secured and thus must be stored.

And for that, the format offered by RSA (the company) has come naturally. Finally, the IETF couldn't help but standardize this to RFC, but that doesn't change the format.

Technically, you might encounter keys in .pk8 files but this is rather rare.

Both

Sometimes we prefer to include the certificate and its private key in a single file (i.e. when are setting up LDAP and TLS on an Active Directory).

This time it was RSA that won again but the IETF has not (yet?) Taken the time to standardize it in an RFC (probably because the use case is much more specific).

As soon as you need to integrate the certificate and its private key all at once, you will therefore encounter .p12 files. Usually, these files are encrypted with a password.

Encodings

The previous standards (PKCS and RFC) define the information that must be put in all these files to be considered valid and usable by compatible tools. They also define which information is mandatory and which is optional, as well as the order in which everything is put away.

Binary formats

But whether it is to store them or to transmit them, at some point, you have to serialize all that; transforming these sometimes complex structures into a series of bits, 0's and 1's. And more importantly, you have to be able to express how this serialization is done so that developers can produce compatible tools.

geralt @ pixabay
geralt @ pixabay

And for that, we use another standard, “ASN.1” (for Abstract Syntax Notation One). Developed by various organizations, including the ITU to which we owe the X.509 format. This standard contains both a way of expressing our complex structures and pieces of software to translate them into binary.

For developers who want to easily manage files in these formats, the libtasn1 library is for you.

When it comes to serializing (and unserializing), the ASN.1 standard defines several ways of doing things (adapted to different situations):

Although cryptographic data can be stored or transmitted with these three encodings, it is the DER format that is systematically used. Thus, a file stored in one of the PKCS or X.509 formats actually contains data encoded in DER.

This is rather rare but if you don't want to use the usual PKCS extensions, you may use .der files. To avoid confusing keys and certificates, some have chosen to use the .cer extension for certificates.

The .cer extension is a false friend because it does not designate the CER encoding but the fact that it is a certificate, however it remains encoded in DER (I insist but It's for a good cause).

Text format

Of course, the previous binary formats (DER and therefore PKCS) are very well suited for storage and network transmission, but they become completely unusable if you can only use readable text (i.e. in a terminal or a web form).

Comfreak @ pixabay
Comfreak @ pixabay

For these cases, rather than inventing a new format, the engineers were as short as possible by reusing the method already developed for exchanging encrypted mails, RFC 1421 "Privacy Enhanced Mail" (PEM): encode the binary in base64 and enclose the content with a first header line (e.g.----- BEGIN CERTIFICATE -----) and a final closing line (e.g. ----- END CERTIFICATE -----).

Note that this was a completely anarchic takeover of a "working stuff". Each application were defining their own "PEM format" based on the original PEM (which did not provide anything to store keys and certificates). The idea caught on and with so much success, the IETF standardized it all in 2015 (RFC 7468).

When certificates and keys are exported after encoding in base64 (plus header + footer lines), they are therefore found in .pem files. To avoid confusing the files, we generally prefer to save the certificates in .crt files. And, more rarely but this is what we prefer among the arsouyes, saving the keys in .key files.

These are the files you will come across most often because they are the ones used by most servers. Whether by uploading files directly (i.e. Matomo, GitLab or ESXi) , or by copying their content into their web interface (i.e. pfSense and VitalPBX).

To make it worse, when you add a root certificate on a ubuntu machine, you can upload a .crt file (in /usr/local/share/ca-certificates), then ask ca-certificates to integrate it (with update-ca-certificates). Then, it will create a symbolic link (in /etc/ssl/certs) whose extension is .pem (but points to your .crt).

And after ?

Technically, it's just a matter of storing two kinds of data: certificates and private keys. And to store them in two ways: in binary or as text. Then, all combinations are possible:

Binary files Text files
Generic content .der .pem
Certificates .p7b, .p10, .cer (X.509) .crt (X.509)
Keys .p8 (rare), .p12 (key + certificate) .key

If you come across .der or .pem, it tells you how it is stored (in binary or in text). It is more often keys but as it can sometimes contain a certificate... To be sure, look around, if you find mention of .cer or .crt file, these new ones are certificates and the first ones files were presumably keys.

If you have the file handy, you can also use the file command which will tell you if your file contains a certificate or a key and how it is encoded.

Our advice, if you need to back up your own files, give them specific extensions. If you store in binary, prefer the PKCS series (.p7,.p8, .p10 and.p12). And if you store as text, prefer .crt and .key. This will make it easier for your colleagues or readers to find their way around.