copyright notice

accesses since March 20, 2006

---

Disk Wiping by any Other Name

Hal Berghel and David Hoelzer

DISK WIPING BY ANY OTHER NAME

With the advance of modern computer forensics tools, disk wiping (aka, data/disk wiping, disk/data erasing, etc) has become increasingly important in the protection of proprietary, confidential, private, and personal information for the law-abiding computer user. Because of Windows' ubiquity n the desktop, SoHo , and notebook markets, a cornucopia of disk wiping utilities are now available. We wondered just how effective these utilities are cleansing disks.

Our interest in disk wiping began with the observation that the Windows built-in utility, cipher, wipes disks by filling a file, EFSTMPWP, with enough data to consume all available non-allocated space. Most noteworthy was our observation that EFSTMPWP could on occasion take up so much space that the OS no longer had room to breath and would hang up. Windows wouldn't reload until EFSTMPWP was deleted by booting to a non-resident OS.

Our appetite whetted, we sought to observe the behavior of some of these utilities and compare the results. Though we document our results below, our greatest interest lies in understanding why these utilities produced the observed results. Toward that end, we'll begin with a brief overview of the Windows' NTFS file system. Then, when we'll link the NTFS data structures to the disk residue.

NTFS v.5

NTFS (Next Generation File System) v.5 is the most common Windows file system in use today. It has some clever features that add efficiency, especially when it comes to file searching. The most important disk structure on an NTFS drive is the MFT (Master File Table) which is stored in the root directory of the volume as the file, “$MFT”. The MFT is used to keep track of how the disk blocks are allocated and what is stored in them. The MFT records are themselves rather flexible, having resident and non-resident “attributes” based on the type of information being stored. For instance, a standard file would have attributes like “$STANDARD_INFORMATION”, containing security identifiers, file access and modification times, owner identifiers, etc. A standard file will also have at least one “$FILENAME” attribute that is used to describe the filename (a second “$FILENAME” is not uncommon, used to represent the old DOS 8.3 convention). The actual file content is either stored in the MFT entry itself (if it's small enough) or referenced by the “$DATA” structure within the file's MFT entry. Storing a file within the MFT is indicated by the “resident” attribute; else $DATA points to the blocks where data is stored and the attribute is marked as non-resident

Directories share $STANDARD_INFORMATION and $FILENAME attributes, but include additional structures like $INDEX_ROOT and $INDEX_ALLOCATION that make up the B-tree structure used to keep track of directory entries. The use of B-trees adds complexity to the identification of a deleted or moved file's parent directory since the entire B-tree is resorted every time a file is deleted. This usually overwrites the old reference to the deleted or moved file. In essence, $INDEX_ROOT and $INDEX_ALLOCATION are the structures that make disk organization at the file name layer possible.

DISK WIPING 101

While there are many variations on this theme, the preferred approach to disk wiping at this writing seems to involve the creation of a new file containing a wiping pattern (e.g., all zeros, all ones, random zeroes and ones). This “single-file” approach takes advantage of the host file system for efficiency, because the “pattern” is created and applied to all available free disk space (including, most importantly, deleted files). This makes it unnecessary to deal with blocks, clusters, and sectors individually. We ignore partially allocated disk space (e.g., RAM slack and file slack) for present purposes. The general theme is this: Windows doesn't delete files, it simply marks the physical space that the files occupied as unallocated and available for reuse. If a disk wiping utility obliterates all of the unallocated space it will, inter alia, obliterate the space formerly occupied by deleted files. It's just that simple.

After the free space has been wiped, some utilities make an effort to scour through the $INDEX_ROOT and $INDEX_ALLOCATION of the directories to be sure that everything has been cleared out. For all but one of the products that we tested, this is the extent of the wiping that is accomplished. Even after this second step, considerable data residue usually remains.

Recall our discussion of the MFT structures. The directories themselves are simply constructs to allow for the user's organization of the items on the disk. The MFT entries connect the user level to the data level. While some of the wiping tools did seem to make changes to the contents of the deleted MFT entries, we found that most of these tools consistently failed to remove all information.

Figure 1: MFT Residue

Figure 1 shows that we can clearly derive file names, both 8.3 and long (Unicode) file names. The tools, with one exception, make no effort to overwrite the old MFT entries. This is a problem for two reasons: first, the file and directory names are commonly indicative of content. One might infer from such information the nature of the business, the level of confidentiality, names, and so forth. Worse yet, if the files are small enough (i.e., if the resident attribute was set) the $DATA will contain all of the original data (cf Figure 2).

Figure 2. Filename and Data Residue

What went wrong? Remember that the disk wiping utilities typically wipe all unallocated space. In order to be confident that this approach works completely, it is incumbent on the user to determine exactly where the data is (allocated space, unallocated space, slack space, MFT resident). It is unreasonable to expect users to have that level of awareness. In this case the data is stored within the $MFT and the $MFT_MIRROR which are allocated space! While there may be slack space associated with them, the area where MFT entries exist is clearly not slack space! For these reasons, most disk wiping utilities miss them.

THE EXPERIMENT

To determine how some of the popular utilities handle disk sanitization, we copied a subdirectory from an NTFS disk to several NTFS-formatted, SANDISK 256mb memory sticks. The subdirectory structure included files with alternate data streams (ADS), word processing documents, programs, and graphics. The test sequence involved:

1. using Windows to erase all files and directories
2. using a utility to wipe the memory stick
3. examination of the remnants, including
1. directory entries
2. timestamps
3. file contents

For post-erasure disk analysis, we used WinHex, Access Data's FTK, and a piece of custom C code (see URL Pearls, below). Figures 3a and 3b illustrate the underlying digital forensics with two different forensics tools.

Figure 3a – Winhex 12.75 integrated with X-Ways Forensics. Note the presence of MFT metadata and persistent filenames.

Figure 3b – Access Data's Forensics Tool Kit 1.61. Note directory structure and file name residue.

SUMMARY OF FINDINGS

The following is a list of utilities and observed results:

CONCLUSION

What we found is that information that in our test environment only one product, Evidence Eliminator, eliminated enough of the data that it fell within our comfort zone. It is clear that most disk wipers leave behind a lot of telltale information that may have proprietary or security implications. Caveat emptor is appropriate here: disk wiping utilities (with the single exception), especially including the built-ins, may leave enough metadata residue for an observer to tell a lot about you and your organization. And if the files are small enough, the entire files are left untouched.

We emphasize that these results need to be taken in context. First and foremost, we limited our concern to data residue that could potentially be recovered with software. The reason for this is that hardware recovery is expensive enough to make casual snooping impractical. The use of sophisticated magnetic sensors and electron microscopes to recover erased data places most of the risk in the realm of governments and 3-letter agencies that may be more likely to use digital surveillance and real-time capture (e.g., Carnivore, Magic Lantern). We note that some of the disk wiping utilities we used did have features that purport to mitigate hardware recovery.

Second, we did not test all storage environments. For that reason, we provided the setup configuration settings so others may duplicate the experiments in their own environments. We predict that an NTFS file system on any medium will behave in a similar, but not necessarily identical, fashion. We have experienced one disk wipe on an 80 gig external USB drive that produced more residue than we found on the memory sticks, even with Evidence Eliminator. So a word of caution is appropriate.

Third, we didn't make any effort to clean the registry hive. “Messing with the registry is really dangerous,” says Microsoft – and we take their word for it. Telltale residue is left behind in the registry without question – how valuable this information is to an onlooker is open to conjecture. Some vendors, such as Evidence Eliminator, encourage the use of registry cleaning tools such as Microsoft's own REGCLEAN, but our fear of turning our workstations into boat anchors disabused us of any temptation to run it. For those who are tempted, beware that REGCLEAN has been reported to cause as many problems as it fixes, and REGCLEAN only cleans HKEY_CLASSES_ROOT which is not usually the most trouble-prone part of the hive. HKLM, for example, is unaffected by REGCLEAN. If that doesn't scare you away, consider that Microsoft no longer supports REGCLEAN!

Finally, there is another category of product that we didn't test – the so-called disk sanitizers or disk purgers. These are products that are marketed with people who intend to re-purpose or re-cycle their computers in mind. In the absence of empirical test results, our advice would be to favor those that claim compliance with appropriate government standards and receive high marks in trade reviews.

URL PEARLS

For those bold enough to search your disks for residue yourself, a copy of the C code that we created for this project is available online at www.cyber-defense.org/MFT_Extractor.php.

Cipher is a built-in core utility for Microsoft's Encrypted File System. As such it's capability goes far beyond disk cleaning. Functional descriptions may be found by entering cipher /? from the command prompt. The /w option is the disk wiping parameter. For additional information on 3 rd party vendors: CyberScrub (www.cyberscrub.com), Evidence Eliminator (www.evidence-eliminator.com), WinCleaner (www.wincleaner.com). PGP Shred and PGP Wipe are utilities within PGP Desktop Professional ( www.pgp.com ).

Hal Berghel is an educator, administrator, inventor, author, columnist, lecturer and sometimes talk show guest. He is both an ACM and IEEE Fellow and has been recognized by both organizations for distinguished service. He is the Associate Dean of the Howard R. Hughes College of Engineering at UNLV, and his consultancy, Berghel.Net, provides security services for government and industry.

David Hoelzer is a Director of the Internet Forensics Lab at CCR and a faculty member of the SANS Institute.  He is the owner of CyberDefense (www.cyber-defense.org), a computer security and forensics consultancy.