A fundamental principle in digital forensics is assuring the integrity of digital evidence. Traditionally, we use hash algorithms such as SHA1 and MD5, feeding the algorithm with each block and each byte within each block in LBA order (a linear bitstream hash).
The picture below depicts the flow of blocks read from a suspect device when creating a traditional EWF (E01) image. Each block is passed through the MD5 algorithm, then compressed, and then stored in the E01 container. Once the image is complete, the linear bitstream hash is saved in the E01 container.
This approach served well for single traditional spinning disks, but as fast storage devices such as SSD and NVMe became available, the hashing approach became a bottleneck - limiting the speed of acquisition. Linear bitstream hashes are particularly problematic with evidence sources containing discontinuities, such as RAM images and partial physical images.
AFF4 Block Map Hashing
In order to address the above challenges, Evimetry employs a technique based on segment hashing (also known as block hashing). Rather than creating a single hash of all bytes in the forensic image, the forensic image is broken up into equal sized blocks, and a hash stored per block. In addition to allowing speed increases when run on multi-core machines, this has the added advantage of identifying at a finer granualarity where hash mismatches occurr should that be the case.
The picture below depicts the contents of an AFF4 physical image. Visible in the "Image Stream" layer, one can see blocks of data from the suspect device compressed and stored. Below that are the block hashes corresponding to each block. By default, Evimetry calculates an MD5 and a SHA1 for each block stored in the image (anthough this is configurable).
It is convenient to have a single hash associated with an image, such that it is straightforward to copy into a report or provide to other parties. Evimetry (and the AFF4 evidence container) provide for this by generating a single SHA512 hash of all of the block hashes. The approach is similar to the Merkle Tree approach used assure integrity in systems such as BitCoin and Git.
For brevity the full hashing scheme isn't described here. For more detail, read the paper Wirespeed: Extending The Aff4 Container Format For Scalable Acquisition And Live Analysis.
Verifying AFF4 Images
Images are verified by Evimetry after acquisition by default. To verify an image accessible as a file from your computer at any time, simply use the Controller, under Tools | Verify local image. For images accessible via an Evimetry Agent, verification is presented as an option for images displayed in the Images tab in the Controller.
This recalculates the hash from the image and compares these with the stored hash values.
Cross tool validation
A traditional linear bitstream hash can easily be computed for AFF4 images by use of the Controller, via Tools | Calculate Linear Hash.
Evimetry's acquisition fidelity and completeness can be compared with a traditional E01 imaging tool using the following steps as a template:
- Acquire suspect drive using an E01 acquisition tool. Note the linear bitstream SHA1 & MD5 hashes of the image.
- Acquire suspect drive using Evimetry.
- Calculate the linear bitstream hash of the AFF4 image using the Evimetry Controller.
- Compare the E01 linear hashes with the linear hashes computed in the last step (they should match).
A note on block hash reproducibility
In order to achieve the speed improvements of AFF4 Block Map Hashing a compromise is required. Due to concurrency, for the same suspect device the ordering of block hashes may be different from one acquisition run to the next. When this occurrs, this leads to different AFF4 block map hashes. Using the linear hash technique, one would get exactly the same hashes.
This has no effect on forensic soundness as the image bitstream remains an identical copy of the source device, as can be demonstrated by calculating a linear bitstream hash where needed. For more information see the presentation: Advancing the AFF4 to the challenges of Volatile Memory & Single Hashes
The AFF4 block hashing technique has been subjected to rigorous scientific scrutiny leading to publication in the leading academic peer review conference, DFRWS, in 2015. No refutations or challenges to the vaidity of the technique.
Evimetry's implementation of block hashing has been in active use since 2014 and accepted by leading practitioners and forensic units globally.
In 2017 the AFF4 Standard v1.0 was defined, which included the block map hashing technique.