Question

25

Can I copy large files faster without using the file cache?

rated 0 times [ 25] [ 0] / answers: 1 / hits: 22898 / 3 Years ago, wed, october 20, 2021, 2:06:37

After adding the preload package, my applications seem to speed up but if I copy a large file, the file cache grows by more than double the size of the file.

By transferring a single 3-4 GB virtualbox image or video file to an external drive, this huge cache seems to remove all the preloaded applications from memory, leading to increased load times and general performance drops.

Is there a way to copy large, multi-gigabyte files without caching them (i.e. bypassing the file cache)? Or a way to whitelist or blacklist specific folders from being cached?

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

onomicsiberated

Add To Favorites

Follow

Total Points: 217

Total Questions: 98

Total Answers: 107

Location: Luxembourg

Member since Sun, May 28, 2023

1 Year ago

answered 3 Years ago imberst · Accepted Answer

There is the nocache utility, which can prepended to a command like ionice and nice. It works by preloading a library which adds posix_fadvise with the POSIX_FADV_DONTNEED flag to any open calls.

In simple terms, it advises the kernel that caching is not needed for that particular file; the kernel will then normally not cache the file. See here for the technical details.

It does wonders for any huge copy jobs, e. g. if you want to backup a multi terabyte disk in the background with the least possible impact on you running system, you can do something along nice -n19 ionice -c3 nocache cp -a /vol /vol2.

A package will be available in Ubuntu 13.10 and up. If you are on a previous release you can either install the 13.10 package or opt for this 12.04 backport by François Marier.