1. Acting Under Emotions

Data loss is a very emotional situation. Users are desperate to get their data back *now*, and may act unpredictably. This includes downloading, installing and trying a bunch of data recovery tools without being aware of the usual precautions. As a result, lost data is getting irrecoverably destroyed, overwritten by the very applications intended to recover the files. Any actions performed under distress should be avoided.

2. Overreliance on Cloud Services

Cloud storage and cloud backups are in fashion these days. However, online storage capacities may be limited compared to humungous amounts of information stored locally. Internet transfer speeds and traffic costs can also be the limiting factor. Sometimes it’s not just faster but much cheaper to recover several terabytes worth of data rather than downloading them back from a cloud storage.

3. Force-Rebuilding Corrupted RAID

This is something new for 2013. More and more data is stored on RAID arrays, including RAID’s built into NAS storage systems. When data on one of the disks comprising the array gets corrupted and the disk falls out of the array, any attempt to forcefully rebuild the array will most likely result in further corruption, loss and destruction of data. Instead of rebuilding the array, customers should use the right technique combined with the right tools (e.g. Diskinternals RAID Recovery).

4. Using Obsolete Backups

Recovering from an old, obsolete backup will overwrite any remaining information with old, obsolete data. If you are recovering information from a backup, make sure you are using a fresh, complete and healthy backup file. Ideally, restore the backup onto a different hard drive – just to make sure you really get everything back.

5. Acting Too Late

Acting too soon while being under strong distress will do no good. However, waiting for too long while continuing using the system containing missing information is also a bad idea. If you want some time to cool down, switch off that computer until you’re ready to recover it!

Storage media of these types can also fail. When considering your recovery options, it is important to understand the mechanics of these devices. Depending on the exact type of a storage device, and depending on the type of a problem, one or another data recovery tool should be used.

Single External Drives

Single hard drives are the easiest to recover. Whether they are enclosed into a USB enclosure or used inside a streaming media player, you can always remove the drive and connect it directly to your PC.

The choice of a data recovery tool will then depend on what happened to the drive. Can you see the disk as a drive letter, but not the files? A rather inexpensive tool called The Undelete will help you.

Don’t see the disk as a drive letter, and all you want to recover are multimedia files such as pictures and videos? D-Photo Recovery or Music Recovery will help.

Don’t see the drive letter, and want to recover all kinds of files? HDD Recovery Pro is a bit expensive, but does a great job.

Recovering Multi-Disk NAS Units

Recovering network storage (NAS) devices with multiple hard drives can be a much more complex task, especially if a complex controller was employed. Inside, some NAS devices may use exotic file systems, while in many multiple-disk configuration a RAID array will inevitably become part of the equation. Removing the many disks from the NAS unit and connecting them directly to ATA cables in your PC may not be possible due to lack of enough available cables / ATA slots. In addition, this may not do much good since you’ll be dealing with a broken RAID array without using the original RAID controller.

If this is the case, you’ll need a tool such as Diskinternals RAID Recovery. This tool can reconstruct the broken RAID array even if no original RAID controller is present. Supporting many types of exotic file systems used by many NAS devices, it’ll give you a pretty good chance of successful recovery. However, this level of service comes with a steep price tag of around $250. Sure, you get to keep the tool after the recovery, but whether or not the content of your NAS storage device is worth the money is a question only you can answer.

Reasons Leading to Loss of Data

There are many different reasons causing loss of information.

Hardware Failures

Despite supposedly increasing reliability of data storage systems, hardware failures remain the number one cause of data loss. As many as 56% of all our customers experienced hardware failures. It is difficult to detail the reasons of such failures within this group, but, empirically speaking, hard drives tend to fail, sometimes prematurely. We’ve heard about “clicking noise”, “unusual noises”, “drive not recognized in BIOS” and many similar reports. Firmware bugs are relatively common, but they are only responsible for about 6% of all hardware-related failures.

User Error

As many as 24% of all problems were determined to be caused by user error. This includes accidentally deleted files; documents “stored” in the Recycle Bin folder; formatted memory cards (“pushed the wrong button”) and sometimes formatted or repartitioned hard drives. OS upgrades also qualify in this category, with a significant number of users selecting the “clean install” option or choosing to freshly partition their disk space.

Application Errors

Software errors are responsible for 16% of data loss situations. This includes applications corrupting their databases, office tools crashing and corrupting documents being edited, and similar issues. Most of the time, the corruption applies to only one file or a group of files; however, sometimes the entire file system gets corrupted, and one of our higher-level tools must be used to retrieve lost information from the disk.

Malware

Despite common belief, malware is only responsible for some 2% of data loss situations. More often than not, the loss is not caused by direct activities of malware applications, but rather instabilities they introduce into the system. Of course, there are borderline situations when users, fed up with malware activities, opt to get rid of a virus by formatting the disk and/or reinstalling the OS. We decided to classify these cases as “user error” instead of “malware”.

Natural Disasters

Electrical problems, fire and flooding are responsible for yet another 2% of all issues. While sometimes all that’s needed is transferring the (working) hard drive to another (working) system, sometimes the storage device gets damaged and needs professional attention.

The SSD Issue

Solid-state drives (SSD) are a special case when it comes to data recovery. The operation of garbage collection and the TRIM command can potentially destroy information after even a quick format in a matter of minutes regardless of what the user does or does not do. Generally speaking, most SSD’s will lose information forever after a format.

However, there are exceptions to this rule, and your particular SSD may not be affected. Data is still recoverable after you format an SSD drive if any single condition from the below list is not met.

• Your SSD drive supports TRIM;
• Your operating system supports TRIM (you’re using Windows Vista, Windows 7, Windows 8, 2003 or 2008 server, or newer version of Windows);
• Your SSD drive was formatted with NTFS (Windows does not implement TRIM on FAT-formatted volumes);
• The SSD is either built-in or connected via SATA/eSATA interface (USB, FireWire and LAN connections do not support the TRIM command);
• The SSD is not part of the RAID (TRIM is not supported on RAID arrays).

Note that if any one (or more than one) condition from the above list is not met, information from your SSD drive can in fact be recoverable.

Quick Format vs. Full Format

In Windows, you can use a quick format or full format operation. Quick-formatting the disk simply erases the file system, leaving all the files well alone. By using a data recovery tool (e.g. HDD Recovery Pro) that supports low-level disk scan, you can still recover many types of files.

As opposed to quick format, full format is supposed to destroy the data by filling the disk with zeroes. This, however, is a misconception: early version of Windows (up to and including Windows XP) did NOT write zeroes onto the disk being formatted; instead, the system just verified integrity of the disk surface by reading all of its sectors. Therefore, HDD Recovery Pro) can recover disks after a full format operation – but only if an old version of Windows was used to format the disk.

You can download HDD Recovery Pro from http://the-undelete.com/

Situations where the TRIM command is not enabled include:

• Unsupported operating system. In Windows world, only Windows Vista, Windows 7 and Windows 8 (and their server versions) are TRIM-compatible. If an SSD was used in a PC running an older OS (e.g. Windows XP), the TRIM command was never issued.
• The file system. Even if the PC is running Windows, if you formatted an SSD drive with any file system other than NTFS, the TRIM command will not be issued. Windows only supports TRIM on NTFS formatted volumes.
• Connection type. Only those drives connected directly via SATA or eSATA will recognize the TRIM command. TRIM is not supported over USB or LAN. Therefore, remote and most external disks will behave just like ordinary hard drives (with the exception of external drives using an eSATA link).
• Storage configuration. SSD’s will not support TRIM if assembled into a RAID. Moreover, certain motherboards will not pass the TRIM command even to a stand-alone SSD drive if RAID support is enabled in BIOS for the SATA port connecting to the SSD.
• Deleted files were stored in a crypto container, and the TRIM option was disabled within the container’s settings (disabling TRIM is their default behavior due to security concerns).
• TRIM is disabled in OS drivers.

Disabling TRIM greatly reduces the SSD’s writing speed, increases its wear level and reduces its lifespan. Disabling TRIM or using SSD drives in configurations where TRIM cannot be used is generally not recommended by their manufacturers. However, if data loss did occur, it’s worth checking if TRIM was actually enabling – or not.

Finally, certain types of files, such as resident files and files kept in cluster slack space, will not be affected by the TRIM command even if TRIM was enabled. Such files can be recovered by using comprehensive disk scan (PowerSearch), a feature of certain higher-grade data recovery tools such as The Undelete or HDD Recovery Pro.

Data Recovery Labs: The Quality Aspect

The quality of all data recovery labs varies. I’ll elaborate: not all data recovery labs will provide the same level of service for quite the same money. Many things depend on their experience, benches, hardware and software tools they use for diagnosing and fixing the problem.

See, the very same issue with a hard drive can often be fixed in many different ways, leading to the same result (data read off the disk) but at very different costs. The same failed hard drive can be read by removing the plates and reading them on a bench (a very labor-intensive procedure requiring expensive specialized hardware) or by simply replacing a burned motor, which will cost just a few dollars. Unfortunately, too many data recovery labs lack expertise or experience required to correctly diagnose a problem, and this is not an overgeneralization.

I had a (physically) failed hard drive once. No software would see it. Computer’s BIOS would not see it. Yet, the spindles were spinning up, and the heads were making just the right noises. So I brought this up to the first recovery lab.

“It must be the main board”, they told me. “Find another drive just like this one, swap the main board, and that’s it. Or we can send it to the clean room and have its plates removed and read. That’ll cost you at least $300. Don’t want to do that? Fine. You owe us $20 diagnostics fee please.”

Wow. Okay. So I brought that disk to another company, making sure they don’t charge any sort of “diagnostics fees” upfront. “It’s a failed head control motor”, they said. “Around $300 to fix.” “And if it’s not?”, I asked. “You still have to pay for the labor”, they said. So I left.

Approaching the third company, I already knew what questions to ask. They looked at the disk and quoted $150 for data recovery. Fine. A few days later I received a call: “Your data is ready”. When I was there to pick up a pen drive with recovered data, I wondered what the problem was. “Worn out”, they said. “Heavily. We had to connect it to the bench and manually modify the reallocated sector table and some other structures.” Obviously there were no main board and no head control motor issues.

These guys were the authorized service center for brand X hard drives. As any OEM-authorized service center, they had the right people and the right hardware to diagnose exactly what’s wrong with that hard drive.

When recovering information from unallocated disk sectors, data recovery applications will perform some form of content-aware analysis, detecting the location of recoverable files by certain characteristics typical for each file format.

In order to perform unallocated space analysis, you will need a tool supporting the technology such as The Undelete. In order to engage unallocated space analysis, select “Comprehensive Recovery” mode (as opposed to “Quick Recovery”) from the Data Recovery Wizard. The tool will then analyze the file system, detect which disk sectors are not occupied with existing files, and attempt the recovery by scanning the available disk space.

PowerSearch

In data recovery tools sold by The Undelete, the PowerSearch algorithm is used to perform unallocated space analysis. PowerSearch uses characteristic signatures typical to certain file types (i.e. the “JFIF” signature for .jpeg files) in order to detect the beginning of a file, then analyses the file’s header in order to determine the file’s exact length.

Tools such as The Undelete employ PowerSearch to perform unallocated space analysis. However, PowerSearch can do more than just analyzing unused sectors. In tools such as HDD Recovery Pro, PowerSearch can be used to analyze the entire content of a volume or a physical device (e.g. a hard disk, pen drive or memory card), looking for missing data everywhere – even if no file system is available. PowerSearch allows recovering data from formatted and repartitioned disks with empty file systems, as well as devices that have damaged or corrupted file systems.

Limitations of File Carving

File carving sounds amazing. But if it’s such a great technique, why don’t all the data recovery tools use it? Why using “inferior” signature search instead?

File carving is an extremely complex task. Multiple attempts may be required in order to reconstruct a single file. Sometimes a skilled human operator must provide assistance during the recovery. The required computation resources and time make file carving only suitable for extremely important files such as those making criminal evidence. In a word, file carving is not feasible to use in order to automatically recover “everything” (user-created content) from the hard disk, as the entire operation would probably take days or even weeks of meticulous analysis.

Another limitation of file carving is the limited number of formats supported by the algorithm. Typically, the file carving process can support pictures in a few most common formats; Office documents in several popular formats; videos produced by popular devices, and a few other formats. Due to extreme complexity of data carving, the number of supported formats will probably never reach the numbers of formats supported by simpler signature search algorithms.

Fast, reliable and fully automatic recovery is possible with tools supporting signature search such as The Undelete or HDD Recovery Pro. These tools will scan the entire hard drive looking for characteristic signatures of known file types, extracting the files even if the file system is not present on the disk.

What Is Data Carving?

Data carving is a technique used by forensic investigators to recover fragmented files from hard disks with missing, damaged or corrupted file system data. The technique is based on a variation of signature search, an algorithm commonly used in many commercially available data recovery tools such as The Undelete or HDD Recovery Pro. With signature search, a data recovery tool can scan the entire surface of the hard disk looking for characteristic signatures of known file types, detecting and analyzing their headers in order to determine the exact file length.

While the idea of file carving is similar to that of signature search algorithms, the carving approach is essentially different in that it applies a significant amount of heuristics in order to reassemble files being recovered from multiple fragments. In the absence of the actual file system, assembling a file from multiple fragments scattered all over the disk is a task far from trivial. The file carving process makes extensive use of information obtained from known file structures, analyzing the content of the file and applying heuristic algorithms in order to reconstruct a working copy of a file from multiple fragments.

Signature Search vs. File Carving

Now, signature search algorithms also analyze information obtained from known file structures in order to recover files. What’s the difference?

Signature search algorithms only analyze limited information about the files such as file headers in order to determine the length of the file. Signature search assumes that the file was stored in a single continuous chunk – which is sometimes not true due to disk fragmentation.

File carving algorithms were specifically designed to take disk fragmentation into account. The file carving technique takes info account all types of information contained in files including file headers and actual file content. The technique makes extensive use of heuristics regarding the typical structures of known file formats as well as how file systems handle fragmented data. By using information obtained from multiple sources, the file carving process can guess which fragments belong together, effectively reconstructing fragmented files from multiple pieces.

Complexity

The abundance of features integrated into the NTFS file system bring us the other side of the coin. The complexity of the new file system technically restricts its use in portable devices as well as devices that generally lack computational power.

High Overhead

The NTFS brings another downside to smaller media: the large overhead. How much extra disk space is used for journaling documenting file system transactions may not matter in a case of a desktop hard drive, but every bit counts with smaller flash drives. For a fairly average 4 GB flash drive, an NTFS volume will provide significantly less space than a FAT partition. For this reason, of not for others, removable drives are usually carrying the FAT file system.

Lack of Interoperability and Legal Restrictions

NTFS is a proprietary Microsoft file system. In general, it’s rarely licensed out to third-party manufacturers. The combination of complexity, high overhead and legal restrictions lead to lack of interoperability with third-party manufacturers. If you format a memory card with NTFS, chances are you’ll be pretty much able to use that memory card with Windows PCs and Windows PCs only. No digital camera or MP3 player (with few exceptions) will support NTFS – but every device out there that accepts memory cards will support a variation of the FAT file system.

The FAT File System and its Variations

FAT is the older, simpler file system that was used in Microsoft operating systems since the times of DOS. There are several variations of the FAT system. FAT12 (the original FAT) was used for formatting magnetic media such as diskettes. At the time, diskettes holding 1.44 MB (that’s one and a half megabytes, not gigabytes) of data were popular, so 12 bits were enough to address the entire file system. The appearance of larger hard drives required an extension of the address space, so FAT16 was born. The appearance of file names longer than 8 characters (with 3-character extension) demanded yet another modification of the standard, while the introduction of even larger hard drives led to the development of FAT32, a file system allowing to address large capacity disks. Finally, the latest development is exFAT, a file system capable of addressing huge space but mostly used on removable media.

Choosing FAT over NTFS

Normally, you would choose FAT over NTFS if you need one or more benefits provided by the older file system over its more complex successor. Use FAT for smaller footprint, broader compatibility and device interoperability. You can’t use anything other than FAT in most digital cameras, smartphones and MP3 players. Finally, for very large flash drives you will use exFAT, a file system that’s compatible with less devices than the original FAT32 but that’s required if you want to address all that free space.

FAT Reliability and Data Recovery Issues

Let’s take it: FAT is less reliable than NTFS. If you use FAT on your desktop hard drive, chances you’ll need a data recovery tool rather soon are high. Flash drives and memory cards are used in a different (lighter) manner, so you can usually get by without a data recovery tool for a while. If you need to recover information from a flash drive or other FAT-formatted device, use either a specialized tool such as FAT Recovery, or a universal data recovery tool such as HDD Recovery Pro.