How TLS and HTTPS Work, Plus a Handful of Useful Tools

Illustration of a perplexed person sitting cross-legged with a laptop, from which a large arc of water is gushing out onto the floor.

In this article I will give you a brief overview of what TLS is and how a chain of trust works.

TLS stands for Transport Layer Security. It has a former name which is SSL or secure sockets layer. Most commonly nowadays it is used for HTTPS. So understanding the core concepts behind TLS is very important for any developer and especially for web developers.

But some preparation is required before starting the main topic. You must grasp the main ideas of how cryptography works.

DevOps consulting: DevOps is a cultural and technological journey. We'll be thrilled to be your guides on any part of this journey. About consulting

There are two cryptography types: asymmetric and symmetric. The main difference between them is how they work with keys.

Symmetric one uses the same key for encryption and decryption, when asymmetric uses one key, called public, for encryption and another one, called private, for decryption or vice versa.

A public key is safe to share with others. One example of such a public key is your SSH public key.

While symmetric cryptography is used only for encryption and decryption purposes, asymmetric one is also used for creating and verifying digital signatures.

A signature of data is just encrypted via a private key checksum for this very data.

To verify the signature one should decrypt it using the public key and check for equality with the computed checksum for the data. After that one can be sure of the data integrity.

Now let's move to TLS itself. It provides us with an encrypted tunnel upon TCP connection. In the case of an HTTPS connection, it's the tunnel between your browser process and the process on a server that you are accessing.

TLS also guarantees such important properties as encryption, authentication (for server or for both client and server - in this way it's known as mutual or two-way TLS) and data integrity.

openssl s_client -connect mkdev.me:443 2>/dev/null </dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >mkdev.crt

openssl x509 -text -noout -in ./mkdev.crt

For example let's see the certificate for mkdev.me site.

Typically its content is PEM encoded so to view it in a human-readable form one needs to use the OpenSSL CLI.

The certificate consists of the server DNS name, the server public key, the signature of its content, and few other fields.

If a server has a certificate, then we only need to check its validity, something we will also discuss in a bit.

But given the certificate and the corresponding private key at the server, how can be a TLS connection established?

TLS uses both cryptography types. It uses asymmetric encryption for a master key exchange, but after that, it uses symmetric encryption via the master key for the data being sent. The reason is that symmetric cryptography is less computation-intensive and faster than asymmetric one for encrypted data-stream.

So let's dive into the process of establishing a TLS connection (for the sake of simplicity we omit ephemerals keys and use plain realization that was used earlier in the first versions of TLS)

A client establishes a TCP connection with a server and asks the server for a new TLS connection.

The server sends a certificate with some extra details like supported ciphers versions.

The client validates the certificate, generates a master key, and sends it to the server in encrypted form (via the public key from the certificate) with the chosen cipher version.

The server decrypts the master key via the private key. Now both parties exchanged the same master key that will be used for symmetric encryption of all the data that will be sent later.

One can see the server certificate which content we have already explained and the master key that was generated and exchanged between the client and the server.

But how does the certificate validation work? How can we check that this is the certificate for this very site and it isn't generated by a man in the middle? To check the validity of the certificate you must either have one at your machine before connecting or somehow derive trust from what you already have.

At this moment a chain of trust comes into play.

The certificates that are always trusted are named Root CA or Root Certificate Authorities and are self-signed (which means that the signature can be checked via the public key presented in the certificate). Such certificates are stored at an operation system-level and distributed with OS.

One level below there are intermediate certificates, which can be verified by the public key from certain Root CA.

At the bottom level, there are leaf certificates to which the site certificate belongs to, and its signature can be verified via certain public key from the intermediate certificate.

So to be able to verify this chain of trust the client should be provided with all the chain except the root CA that is already presented on the clients' operation system.

To navigate all possible chains of certificates, each certificate has a unique id. The client is able to validate the site certificate by checking signatures from the bottom up until it reaches the root CA certificate that is trusted by definition.

I think that now you have an understanding of how TLS works and what a chain of trust is with its role in the TLS connection establishment process.

It's not a complete guide about TLS. There are several related topics such as different cipher suites for master key exchange, certificates revocation, fast TLS session resumption, and much more.

You may find more details in a free book named 'High-performance browser networking' in a chapter about TLS Networking. Also one can see a byte-to-byte example of TLS connection with annotations for each byte at this site. But if you really like to understand all the tricky details it's better to familiarize yourself with an RFC 8446 which describes the latest version of TLS.

Also nowadays it's simple to create free certificates via Certbot for servers. All you need is just to select your server and OS to get detailed instructions. For localhost development there is a tool named Minica that simplifies the process of certificates creation for your own local root CA.

Links

Here's the same article in video form, so you can listen to it on the go: