Hashing Algorithms and Checksums in Linux

Say you want to send an important file, how do you ensure that the receiver will get the exact file without any changes ?

Well, the most common approach is use hashing, which translates the file information into fixed sized values, and you can compare these hash values with the original hash values to ensure the file authenticity.

In this guide, we’ll use the most common hashing algorithms that comes pre-installed with any Linux distribution. Below are the hashing algorithms we gonna use in this tutorial:

SHA-256
SHA-512
MD5
BLAKE2

What is Hashing

Hashing is an algorithm used to calculate a fixed-size bit string value from a file. The hash value is a summary of the original data. The file contains a block of data, and hashing transforms this data into a shorter fixed-length key or value representing the original string. A hash is a unidirectional process, and you can’t work backward to get back the original data.

The file hashing is done on the web servers by the web administrators, who then publish the hash results. The users download the files and then apply the same hash method. The two files are identical, and there is no file corruption if the checksum of the original file and the downloaded file are the same.

What is a Checksum

A checksum is a value used to verify the file integrity in data transfer. Verifying a checksum helps to ensure that there is no corruption or manipulation in the downloaded file. Hashing algorithms like MD5, SHA256, SHA512 hashes are commonly used for checksum verification. Checksums only verify the file integrity. It does not provide any encryption or confidentiality.

There are many hashing algorithms, but in this guide, we will be using some of most common types of hashing used for file integrity and checksums in Linux.

SHA-256

SHA256 algorithm generated a unique, fixed-size 256-bit hash. The sha256sum command is used to check file integrity and authenticity. It computes and checks SHA-256 encrypted message digest on Unix-like operating systems. You can use this command in binary and text mode, and both of these modes generate the same SHA-256 hash on Linux.

$ sha256sum [OPTIONS] /path/to/file

The following table shows the options of this command:

-c, –check	Read and check SHA256 sums from the FILEs.
-b, –binary	Read in binary mode.
-t, –text	Read in a text mode.
–tag	Create a BSD-style checksum.

Generate Hashes for Files

As an example, let’s create a simple text file:

$ echo "Hello World" > data.txt

The following command gives the SHA256 checksum of the data.txt file in the current directory:

$ sha256sum data.txt

Output:

d2a84f4b8b650937ec8f73cd8be2c74add5a911ba64df27458ed8229da804a26  data.txt

Verify File Integrity

To verify file integrity of your downloaded file, you can use the -c or --check parameter in sha256sum command. Let’s first create the checksum file with the previous command:

$ sha256sum data.txt > checksum

Now in the real world, this checksum file need to be downloaded along with the actual file, an example of this is when installing Ubuntu image, you can download the SHA256 checksum files along with the image to verify whether the file you have downloaded in your machine is the exact same as theirs.

Now verifying:

$ sha256sum --check checksum

This will look for files listed in that checksum file, and calculates the hash for each of them and compare, here is the output:

data.txt: OK

Another alternative is to use OpenSSL library:

$ openssl sha256 data.txt
d2a84f4b8b650937ec8f73cd8be2c74add5a911ba64df27458ed8229da804a26  data.txt

Let’s add a character to the file and run the checksum again:

$ echo "!" >> data.txt
$ sha256sum --check checksum
data.txt: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match

Password Hashing

If you wish to hash your password with SHA-256, a good tool is mkpasswd command, if it isn’t installed in your machine yet, make sure you install whois:

$ sudo apt-get install whois

After that, as an example, let’s hash "password":

$ mkpasswd -m sha-256 "password" -S '12234567'

Output:

$5$12234567$kmJagEajkVRQYSevoOGR2xT74FA7n0SzHk56blAUKo3

-S is the optional parameter for the salt. You can always use mkpasswd --help or man mkpasswd for details.

SHA-512

SHA-512 is the hashing algorithm to perform hash functions on data given to it. It is the strongest among all SHA-2 hashes and generates a unique, fixed-size 512-bit hash. The sha512sum command behaves the same as sha256sum and it computes and checks the SHA512 message digest.

Generate and Verify Hashes

It is the same process as sha256sum, you basically change sha256sum with sha512sum!

Below is OpenSSL command for SHA-512:

$ openssl sha512 data.txt
SHA512(data.txt)= e1c112ff908febc3b98b1693a6cd3564eaf8e5e6ca629d084d9f0eba99247cacdd72e369ff8941397c2807409ff66be64be908da17ad7b8a49a2a26c0e8086aa

Password Hashing

mkpasswd supports several algorithms and SHA-512 is one of them, this time, we use sha-512 as method type instead of sha-256, and it behaves the same way too:

$ mkpasswd -m sha-512 password -S '12234567'
$6$12234567$T3yqNqxLLLLmY/.JfFhtzQhHOi7uyPg92V0gHMORUoV5jCodNpYo7v9QewZLdb2yM4AHNM5zVdBFA/Yzv8Cq9/

MD5

MD5 is a message-digest algorithm that implements a cryptographic hash function used for message integrity checks. This algorithm creates a unique, fixed-size 128-bit cryptographic hash from data input. The major difference between the Windows and Linux security features is salt strings. The salt string randomly generates a value that is stored with every password. It allows additional security for every password within the system.

Generate and Verify Hashes

The md5sum command handles that for us:

$ md5sum data.txt
e59ff97941044f85df5297e1c302d260  data.txt

Using OpenSSL:

$ openssl md5 data.txt
MD5(data.txt)= e59ff97941044f85df5297e1c302d260

BLAKE2

BLAKE2 is a cryptographic hash function faster than SHA256, SHA512, and MD5. It has two different types, BLAKE2b, and BLAKE2s. BLAKE2b is optimized for 64-bit platforms, and BLAKE2s is optimized for 8 to 16-bit platforms.

Generate and Verify Hashes

The command is the following:

$ b2sum data.txt
01cc021b7aaf1ed89304c61cc48c455b2a4e500812aa9b6ff4256372f9882a6fd4df0291f70f9883268a978e615e540aab12b3f7e0b4b32914e515665aa05edb  data.txt

BLAKE2s with OpenSSL:

$ openssl blake2s256 data.txt
BLAKE2s256(data.txt)= cd0b3b1832c2460e30bc924ebba398f5bf6447478c2532995d4657707340c09c

And BLAKE2b:

$ openssl blake2b512 data.txt
BLAKE2b512(data.txt)= 01cc021b7aaf1ed89304c61cc48c455b2a4e500812aa9b6ff4256372f9882a6fd4df0291f70f9883268a978e615e540aab12b3f7e0b4b32914e515665aa05edb

Conclusion

SHA-256 and SHA-512 are in the same family of SHA-2, there are also SHA-224 and SHA-384 functions, you can use sha224sum and sha384sum normally as well.

SHA-2 is the successor of SHA-1 which is now obsolete and easy to break, the motivation of SHA-2 is to generate longer hashes which leads to higher security than SHA-1.

BLAKE2 hashes are faster than SHA-1, SHA-2, SHA-3 and even MD5 as previously mentioned, and it is even more secure than SHA-2, it is widely used in software including OpenSSL, Noise protocol, and much more.

Even though this tutorial walked you through using MD5, but you shouldn’t use it nowadays, as it is pretty obsolete now, and you should never use it, as it isn’t collision resistant.

Hashing technology is useful to verify if the files are copied correctly between two resources or not. It is amazing how easily you can verify the authenticity of a file with just one command. However, IT is a fast-changing industry, and the security risks associated with technology are also increasing. To keep the security standards high, you must follow the advanced and latest hashing algorithms for security purposes.