Cloning and Data Recovery Using Gddrescue

[PCX]

Main Uses for HDD Cloning.

- Upgrading from a smaller HDD to a larger HDD without losing data or programs (you will have to either resize the main partition to take up the whole drive or use the unallocated space as additional storage)
- Replace failing hard drive. (make sure you are using a hard drive that is the same size or larger.)
- Pull recovery and partial OS partitions for reinstall when no disk or product key is available. There are usually better ways around this though.
- Recovering data from drives that will not mount


There are other uses for HDD cloning, but these are the main ones that I use it for.


Installation

Open up the terminal in Ubuntu and type in

sudo apt-get install gddrescue

Using Gddrescue

There are several uses for Gddrescue, including data forensics and recovering data for damaged disks, but I will only go into detail about cloning HDDs.

First I will start by breaking down this code

sudo ddrescue -v -r3 /dev/sdb /dev/sdc logfile

sudo gives the user administrative privileges, you will still need to enter your password

-v is the switch that prints out the results as the cloning process is done

The -r3 switch is the number of times Gddrescue will retry to recover information from a bad block. It is really upto you how many times you want to retry bad blocks. It will take you several hours on one try alone. I usually opt out of this option unless I have ample time. Just make sure you run a chkdsk /r.

/dev/sdb is the source (HDD to be cloned) location

/dev/sdc is the target (HDD to be copied to) location

The locations of the hard drives may obviously be different depending on your set-up and location of the drives. For instance, you may actually be cloning /dev/sdc to /dev/sdb

sudo ddrescue -v -r3 /dev/sdc /dev/sdb logfile

The easiest way to find the location of the drives you are cloning is to use disk utility. Make sure you double and triple check the drive location before you start cloning, you do not want to clone the empty drive onto your customers HDD. IT WILL COPY EMPTY SPACE!

logfile is the name of the logfile saved for each cloning process. This logfile will allow you stop in the middle of a cloning process, unplug the hard drives and plug them back in at a later time and begin where you left off. You can name this anything you like, for instance, Dell-1545. Just make sure that if you stop and restart later, that you use the same logfile name. An example of this would be

Cloning process starts

sudo ddrescue -v -r3 /dev/sdb /dev/sdc Dell-1545

Stopping the cloning process

CTRL + C

Swap out drives, do some diagnostics, see the real progress of the cloned drive (via plugging it into the original computer), etc.

Start cloning process again

sudo ddrescue -v -r3 /dev/sdb /dev/sdc Dell-1545

Keep in mind, that the location of the drives can change, so double and triple check them to make sure that if they need to be changed, that you change them appropriately. As long as you keep the same logfile name, it will not matter where the drives are located.


Copying Single Partitions:

This would be good if you are only trying to recover the recovery partition. You will find that will not always work due to the MBR being missing. If you are going to copy all the partitions but the main partition, make sure you maintain the same partition structure on the new drive. You can do this in Gparted.

sudo apt-get install gparted

To copy a single partition, simply ad the partition number to the end of the drive letter. For instance

sudo ddrescue -v -r3 /dev/sdc1 /dev/sdb1 logfile

You can easily find this in gparted or disk utility.

 

[PaulRome]

Excellent post. Thanks.

 

[Encrypted Existence]

Thanks for the info. I just recently installed Linux Mint 12 on one of my bench machines. I am really enjoying it. Already installed gsmartcontrol. Ddrescue is next. This post will surely help me with the basics. I am just wondering why I waited so long to start using Linux.

 

[iisjman07]

Nice post; for anyone wondering how to find out the device files of your source/target disk you can run 'sudo fdisk -l' (that's a lowercase L), and that will list the disks attached to the machine

 

[Painless]

Thanks for your step by step instructions - including explanations.

I am in the throes of learning to use Linux terminal and most of the instructions for using ddrescue are confusing.

Thanks for your help.

 

[PCX]

No problem guys, I am glad it was helpful. This is a tool that I use at least a couple times a week. There is more that you can do with gddrescue, but for the most part, this will cover most of what you guys will need it for.

 

[cyde_ePhex]

Thank you. This is a great post.

 

[Herodotus]

Out of curiosity, would this work if using linux off a live usb? Thinking maybe in terms of attaching an external hdd via usb to laptop, then run this using live usb to copy laptop contents onto the external hdd.

 

[PCX]

Out of curiosity, would this work if using linux off a live usb? Thinking maybe in terms of attaching an external hdd via usb to laptop, then run this using live usb to copy laptop contents onto the external hdd.

I believe so, but I would not suggest it. The whole point in doing a clone for the purposes of a data recovery is to get the data off as quickly and safely as possible before it fails. USB (at least USB 2.0) is way slower than a SATA interface, thus increasing the chances the hard drive failing before it completes. The other danger of course is any time a hard drive is just kinda floating around and is able to be moved, you are increasing the chances of it being jostled, dropped or disconnected. My suggestion is to get a cheap desktop specifically for cloning, diagnostics, data recovery, backups ect. and put Linux on that desktop. Get a couple front loading SATA bays and use them instead of a USB interface.

Now, one could argue that if a hard drive is failing, then its probably not going to go much faster than the rate of USB 2.0 anyways, but I still would rather not take my chances. When it comes to data recovery, take every precaution possible.

 

[andcorptech]

My suggestion is to get a cheap desktop specifically for cloning, diagnostics, data recovery, backups ect. and put Linux on that desktop. Get a couple front loading SATA bays and use them instead of a USB interface.

Now, one could argue that if a hard drive is failing, then its probably not going to go much faster than the rate of USB 2.0 anyways, but I still would rather not take my chances. When it comes to data recovery, take every precaution possible.

This is a good point. We have 3 machines dedicated to data recovery, and lately they have all been working 24/7. If you are working with a really bad drive be prepared for it to take a long time. Our longest clone took 16 days, but we let it go and it finally got there with all the critical data saved. I was so tempted to take it off the bench and use other methods but I resisted and It paid off. I may not recommend leaving it this long in all cases, but its a hard call when your dealing with a really borked drive.

 

[anonymous Mac Tech]

I believe so, but I would not suggest it. The whole point in doing a clone for the purposes of a data recovery is to get the data off as quickly and safely as possible before it fails. USB (at least USB 2.0) is way slower than a SATA interface, thus increasing the chances the hard drive failing before it completes. The other danger of course is any time a hard drive is just kinda floating around and is able to be moved, you are increasing the chances of it being jostled, dropped or disconnected. My suggestion is to get a cheap desktop specifically for cloning, diagnostics, data recovery, backups ect. and put Linux on that desktop. Get a couple front loading SATA bays and use them instead of a USB interface.

Now, one could argue that if a hard drive is failing, then its probably not going to go much faster than the rate of USB 2.0 anyways, but I still would rather not take my chances. When it comes to data recovery, take every precaution possible.

I've got to say I have every opportunity to use either SATA, USB, or Firewire. Most of the time I like firewire overall due to the fact there is no processor involvement with firewire. (Meaning much more consistent speeds and less chance of the system you are booted to freaking out in the middle of a clone), where USB and SATA everything is channeled through the processor. However there are times that for whatever reason USB will connect better with the drive than FW (or vise-verse) so use what works best for the situation. The most noticeable factor to me is airflow. Getting air to a failing drive does wonders many times. So leaving a drive in whatever machine its installed into or stuffing a drive into a SATA bay on a desktop is more likely to suffocate the drive than anything (unless its a wide open case with fan directly pointed at the malfunctioning drive). Also SATA most likely is what is what your test machine is booted from so putting a malfunctioning drive on the same bus as what you are booted from (like mentioned above referring to processor involvement) is going to cause all kinds of problems.

 

[PCX]

I've got to say I have every opportunity to use either SATA, USB, or Firewire. Most of the time I like firewire overall due to the fact there is no processor involvement with firewire. (Meaning much more consistent speeds and less chance of the system you are booted to freaking out in the middle of a clone), where USB and SATA everything is channeled through the processor. However there are times that for whatever reason USB will connect better with the drive than FW (or vise-verse) so use what works best for the situation. The most noticeable factor to me is airflow. Getting air to a failing drive does wonders many times. So leaving a drive in whatever machine its installed into or stuffing a drive into a SATA bay on a desktop is more likely to suffocate the drive than anything (unless its a wide open case with fan directly pointed at the malfunctioning drive). Also SATA most likely is what is what your test machine is booted from so putting a malfunctioning drive on the same bus as what you are booted from (like mentioned above referring to processor involvement) is going to cause all kinds of problems.

Yeah, you most definitely want a desktop with lots of airflow and and an open loading bay. I got a bunch of these that gives the drive plenty of room to breath as well as lots of air flow in a full size case. That said, I find it harder to connect a failing hard drive via USB vs a SATA controller in most cases due to the reasons you listed above. It just makes the computer run like junk and the other hard drives also have a harder time connecting if on the same SATA controller, which is one of the reasons why I have more than one.

Its also a good idea to have a decent processor as well. Testing multiple hard drives (especially if some are failing) or cloning them will soak up the processor pretty quick. I personally use an AMD Phenom II Quad Core Processor in my rig.

 

[gikstar]

That said, I find it harder to connect a failing hard drive via USB vs a SATA controller in most cases due to the reasons you listed above. It just makes the computer run like junk and the other hard drives also have a harder time connecting if on the same SATA controller, which is one of the reasons why I have more than one.

Do I understand you correctly?... you use a motherboard in your bench system that has two on-board sata controllers (many newer motherboards have a separate controller for Sata 3Gb and Sata 6Gb) or you use an add-on sata controller pci or pci-e card to attach the second hard drive to.

If this is the case, I will check out my several bench systems and add the cards if needed. I've never given this a second thought. I've just got extra long Sata cables and Ide cables hanging out of the open case and attach the drives, with the drives standing on end to get as much air flow through them.

Thanks for this tip, it's a good one.

 

[PCX]

Do I understand you correctly?... you use a motherboard in your bench system that has two on-board sata controllers (many newer motherboards have a separate controller for Sata 3Gb and Sata 6Gb) or you use an add-on sata controller pci or pci-e card to attach the second hard drive to.

If this is the case, I will check out my several bench systems and add the cards if needed. I've never given this a second thought. I've just got extra long Sata cables and Ide cables hanging out of the open case and attach the drives, with the drives standing on end to get as much air flow through them.

Thanks for this tip, it's a good one.

Yeah, I have a separate SATA controller via PCI-E. It also allows me to test up to 8 or more hard drives at once.

 

[Encrypted Existence]

Do I understand you correctly?... you use a motherboard in your bench system that has two on-board sata controllers (many newer motherboards have a separate controller for Sata 3Gb and Sata 6Gb) or you use an add-on sata controller pci or pci-e card to attach the second hard drive to.

If this is the case, I will check out my several bench systems and add the cards if needed. I've never given this a second thought. I've just got extra long Sata cables and Ide cables hanging out of the open case and attach the drives, with the drives standing on end to get as much air flow through them.

Thanks for this tip, it's a good one.

I agree. I have been using the on board controllers but I will be purchasing add-ons in the near future. Thanks.

 

[Herodotus]

I believe so, but I would not suggest it. The whole point in doing a clone for the purposes of a data recovery is to get the data off as quickly and safely as possible before it fails. USB (at least USB 2.0) is way slower than a SATA interface, thus increasing the chances the hard drive failing before it completes. The other danger of course is any time a hard drive is just kinda floating around and is able to be moved, you are increasing the chances of it being jostled, dropped or disconnected. My suggestion is to get a cheap desktop specifically for cloning, diagnostics, data recovery, backups ect. and put Linux on that desktop. Get a couple front loading SATA bays and use them instead of a USB interface.

Now, one could argue that if a hard drive is failing, then its probably not going to go much faster than the rate of USB 2.0 anyways, but I still would rather not take my chances. When it comes to data recovery, take every precaution possible.

Alright. Thanks for the advice. I forgot that Linux doesn't need a high end machine to do certain tasks. Thanks again.

 

[Kim101]

Thanks for the useful tips, PCX! I will stick to SATA bays.

 

[bytes]

I'm not trying to hijack this thread but,

Can anyone recommend a decent front loader?

Thanks in advance!

 

[Encrypted Existence]

I'm not trying to hijack this thread but,

Can anyone recommend a decent front loader?

Thanks in advance!

I am currently using this one. By no means indestructible but it does the job well and is fairly cheap.

http://www.amazon.com/gp/product/B0024CV3SU/ref=oh_o05_s00_i00_details

 

[PCX]

I am currently using this one. By no means indestructible but it does the job well and is fairly cheap.

http://www.amazon.com/gp/product/B0024CV3SU/ref=oh_o05_s00_i00_details

Just do not use the top 2.5 bay for failing 2.5 HDDs that you are cloning. You want to give it plenty of breathing room.