Question

95

How would I speed up a full disk dd?

rated 0 times [ 95] [ 0] / answers: 1 / hits: 160056 / 2 Years ago, sun, october 30, 2022, 4:05:20

I am doing a dd on two identical drives with this command:

 dd if=/dev/sda of=/dev/sdb bs=4096

Both hard drives are the exact same model number, and both have 1TB of storage space. /dev/sda uses a blocksize of 4096. /dev/sda is a local drive and /dev/sdb is a remote caddy. I might be able to use the following protocols:

USB2.0 HighSpeed (Currently the plan)

Gigabit Over-The-Network clone (Really do not want to even try this)

USB3.0 (If I find my other drive caddy)

eSATA (If I find/buy a cable)

SATA (If I find/buy a cable, gotta love laptop CD drives)

Is there a way to run this drive copy that takes less that 96 hours? I am open to using tools other than dd.

I need to clone the following partitions (including UUIDs)

Fat32 EFI Partition (*)

NTFS Windows Partition (*)

HFS+ OSX Partition

EXT4 Ubuntu Partition (*)

Swap Partition (*)

* Supported by Clonezilla

I have tried Clonezilla (and it was MUCH faster), but it does not support HFS+ smart copying, which I need. Maybe the newest version supports this?

When I made my first clone, I did all of the partitions except HFS+ and it went very quickly. (No more than 3 hours total)

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

housecoarmer

Add To Favorites

Follow

Total Points: 434

Total Questions: 116

Total Answers: 120

Location: Albania

Member since Sun, Nov 22, 2020

4 Years ago

answered 2 Years ago ligenvirt · Accepted Answer

In my experience, I don't think there is something faster in the command line as dd. Adjusting the bs parameter can increase the speed, for example, I have 2 HDD that I know have a read/write speed greater than 100 MB/s so I do this:

dd if=/dev/sda of=/dev/sdb bs=100M

There is also pv (Needs to be installed first) that checks for the fastest speed on both drives and then proceeds on cloning. This has to be done of course from root:

pv < /dev/sda > /dev/sdb

With PV I got 156 MB/s

The nice thing about pv apart from the speed is that it shows the progress, current speed, time since it began and ETA. In regards to HFS+ I would not know, am just trying to help on the "speed" part. With pv or a very optimized bs parameter, you can do a 4 TB drive in less than 7 Hours (6 Hours 50 Minutes at a current speed of 150 MB/s).

enter image description here

I did a couple of tests with the connection types you were using and others I had available. I was using the Asus Z87 Pro and the Intel DZ68DP. This were my results, but first we need to know that the theoretical speeds for many transfer rates (Raw speeds) are just that, theory. Doing real tests revealed they are between 40% to 80% of that raw speed. This tests can change depending on Device used, connection type, motherboard, type of connecting cable, filesystem type and more. With that in mind, this is what I got (I only tested Write speed to the Device, read is typically higher):

Connected Device  -  Connection Type  -  Speed (Write Speed)

  USB 2.0                 USB 2.0              25 MB/s

  USB 3.0                 USB 2.0              35 MB/s

  USB 3.0                 USB 3.0              73 MB/s

  eSata                   eSata                80 MB/s

  Sata 2G HDD             Sata 2G              120 MB/s

  Sata 3G HDD             Sata 2G              140 MB/s

  Sata 3G HDD             Sata 3G              190 MB/s

  Sata 2G SDD             Sata 2G              170 MB/s

  Sata 3G SDD             Sata 2G              210 MB/s

  Sata 3G SDD             Sata 3G              550 MB/s