The I/O size is a reason why it’s better to use cp than dd to copy an ISO to a USB stick. cp automatically selects an I/O size which should yield good performance, while dd’s default is extremely small and it’s necessary to specify a sane value manually (e.g. bs=1M).
With “everything” being a file on Linux, dd isn’t really special for simply cloning a disk. The habit of using dd is still quite strong for me.
Like i use to say; dd nowadays is best used as a scalpell (to cut bit sizes) not a shovel (to move chunks of data).
For me, I only got to 5 Gbps or so on my NVMe using dd and fine tuning bs=. In second place there was steam with ~3 Gbps. The same thing with my 1 Gbps USB Stick. Even though the time I saved by more speed is more than made up by the time it took me to fine tune.
Sorry for the nitpick, but you probably mean GB/s (or GiB/s, but I won’t go there). Gbps is gigabits per second, not gigabytes per second.
Since both are used in different contexts yet they differ by about a factor of 8, not confusing the two is useful.I think it is gigabits per second, but to be clear: Whatever dd reports as speed after finishing (or with status=progress)
That’s in bytes. A modern NVMe drive can do about 7 GB/s (more than 10 for PCIe 5.0 drives). Even SATA could handle 5 Gbit/s, though barely.
Yup, my PCIe NVMe should be readable at 8 GB/s and writeable at 7 GB/s afaik. I can’t reach that speed tho, no matter which bs.
For those that like the status bar dd provides, I highly recommend pv
pv diskimage.iso > /dev/usb
Interesting. Is this serious advice and if so, what’s the new canonical command to burn an ISO?
Recently, I learned that booting from a dd’d image is actually a major hack. I don’t get it together on my own, but has something to do with no actual boot entry and partition table being created. Because of this, it’s better to use an actual media creation tool such as Rufus or balena etcher.
Found the superuser thread: https://superuser.com/a/1527373 Someone had linked it on lemmy
Wow. I’ve been using dd for years and I’d consider myself on the more experienced end of the Linux user base. I’ll use cp from now on. Great link.
Good to hear, I’ve only been in the Linux World for a few years myself, but I was very surprised too. Through I don’t think that using cp is any different in terms of creating boot records and a partition table.
Nowadays I just use Ventoy
Same for me. Ventoy is pretty amazing and keeps most of my isos on it. Sadly, sometimes it’s not capable of doing the job, for example when I installed proxmox (based on Debian 12) this week, ventoy couldn’t do it. Apparently this is a known issue in ventoy.
But yeah, for most isos, ventoy is the way of you install OSes somewhat often, as it contains partition layouts and boot records regardless (I think).
Huh thanks for the link. I knew that just dd’ing doesn’t work for windows Isos but I didn’t know that it was the Linux distros doing the weird shenanigans this time around
It is really informative! Spread the word.
Thanks for the tip. Not that I plan to read up on the matter and make the next cold installation even more anxiety-inducing that it already is. Sometimes Linux would really benefit if there were One Correct Way to do things, I find. Especially something so critical as this.
There is. Just use a media creation tool, like Rufus. dd’ing onto a drive is a hack.
But how trustworthy is Rufus? This is a pretty critical operation, after all.
Assuming you have a brand-new Windows laptop in front of you, how do you go about getting Linux on it? Genuinely interested to know. Last time I had to do this, I went via Windows Powershell or whatever it’s called, and used
dd. Seemed like the option involving the least untrusted parties.Personally I think that the distros should be taking charge of this themselves, and providing the .exe installer as well as the ISO.
Why would you count Rufus and balena etcher not trustworthy? Sounds like you’re to deep in the paranoia, which I completely understand, but gets just impractical “Man yelling at cloud” depending on how deep you are.
dd is just another program too, why trust dd? Linux is just another Program too, why trust Linux? And so on. You can audit every (OSS) Program if you want in theory, but let’s be real, no one does that because time is better spent elsewhere.
Nothing hacky. Nowadays, most distro iso are hybrid iso.
cp /path/to/image /dev/diskAs I understand it, there isn’t really a canonical way to burn an ISO. Any tool that copies a file bit for bit to another file should be able to copy a disk image to a disk. Even shell built-ins should do the job. E.g.
cat my.iso > /dev/myusbstickreads the file and puts it on the stick (which is also a file). Cat reads a file byte for byte (test by cat’ing any binary) andredirects the output to another file, which can be a device file like a usb stick.There’s no practical difference between those tools, besides how fast they are. E.g. dd without the block size set to >=1K is pretty slow [1], but I guess most tools select large enough I/O sizes to be nearly identical (e.g. cp).
[1] https://superuser.com/questions/234199/good-block-size-for-disk-cloning-with-diskdump-dd#234204
Given that this is the crucial first step in installing Linux on a new computer, the fact that there is so much mystery and arbitrariness around it seems to me pretty revealing and symptomatic of Linux’s general inability to reach ordinary folks.
Thanks for the information.
This isn’t mystery, they’re saying any old command that prints out or copies a file’s contents will do.
If you need to use a tool that “just works” without growing your own understanding, there’s plenty of GUI-centric bootable USB writers out there.
Do cp capture the raw bits like dd does?
Not saying that its useful in the case you’re describing but that’s always been the reason I use it. When I want every bit on a disk copied to another disks, even the things with no meta data present.
As long as you copy from the device file (
/dev/whatever), you will get “the raw bits”, regardless of whether you usedd,cp, or evencat.That makes sense. Thanks!
/ As of Jan 2034, 512KiB is determined to be the perfect blksize to minimize system call overhead across most systems.
as of Jan 2034 there is no humans so thus we can discontinue tools for humans
Feb 2034: grammer correction: there are no humans.
This is the best summary I could come up with:
One of the interesting improvements to note with GNU Coreutils 9.5 is that the cp, mv, install, cat, and split commands can now read/write a minimum of 256KiB at a time.
Previously there was a 128KiB minimum while this has been doubled in order to enhance the throughput of Coreutils on modern systems.
The throughput with Coreutils 9.5 thanks to this change increases by 10~20% when reading cached files on modern systems.
This default I/O size update was last adjusted a decade ago.
GNU Coreutils 9.5 also has fixed various warnings when interacting with the CIFS file-system, join and uniq better handle multi-byte characters, tail no longer mishandles input from /proc and /sys file-systems, various new options for different commands, SELinux operations during file copy operations are now more efficient, and tail can now follow multiple processes via repeated “–pid” options.
More details on all of the GNU Coreutils 9.5 changes via the release announcement.
The original article contains 263 words, the summary contains 155 words. Saved 41%. I’m a bot and I’m open source!
Btw do you know how uutils (rust rewrite of GNU coreutils) is doing?
Wrong license still but I just read an article saying they are making progress on compatiabity with the GNU coreutils
What means wrong license? Does it need to be GPL(v3)?
Ideally, moving away from copyleft is step backwards for user tools
Huh? Wasn’t at least cp supposed to determine optimal block size automatically?
Btw, what was the tool again, that watches some kernel functionality to get progress for multiple such tools?
progresshttps://github.com/Xfennec/progressThanks! Name of the tool checks out, lol.
