Firefox 3 has a hard-to-fix bug that causes linux to write to disk every time you visit a write page. But it doesn't just write Firefox stuff -- it causes your entire system to dump all changes to disk. Unfortunately, your USB key might only have a 6 MB/s write speed, causing everything to freeze up.
It's kind of like putting magnets under your pillow to improve health. Maybe it's having an effect, but I can't tell. I install it anyway:
sudo aptitude install preload
On solid state storage, space is expensive. Ubuntu uses a huge amount of space will all the programs it installs. The /usr folder contains your programs, and it is usually 1.8 GB. Using squashfs, it can be compressed to 0.7 GB. Since read speeds are so slow, you can actually gain performance because there is less data to read. I've adapted these instructions from here.
Install squashfs and unionfs:
sudo apt-get install squashfs-tools unionfs-tools
Add the following lines to /etc/modules:
unionfs squashfs loop
Remove apparmor. Otherwise, the cups print server will stop working:
sudo apt-get purge apparmor
Make space for the filesystem:
sudo mkdir -p /.filesystems/usr/overlayCompress your filesystem:
sudo mksquashfs /usr /.filesystems/usr/usr.sqfsAdd these lines to /etc/fstab:
/.filesystems/usr/usr.sqfs /usr squashfs ro,loop,nodev 0 0 unionfs /usr unionfs nodev,noatime,dirs=/.filesystems/usr/overlay=rw:/usr=ro 0 0Switch to runlevel 1. (Ubuntu will close all open programs, then prompt you what to do. Choose opening a root shell)
sudo init 1
Move aside the old /usr directory and create a new mount point:
mv /usr /usr.old mkdir usr
Test whether you previously edited fstab successfully by typing:
mount -a
If you get error messages or your /usr directory shows up empty, either fix your fstab or undo the changes before continuing.
Now reboot and make sure it all works:
reboot
If it works, remove the /usr.old directory to reclaim the space.
This tip can also lead to data loss. If you do it, you will have to always shut down your computer properly from now on, because unexpected power failures will lead to data loss.
Linux usually ensures that all changes are written to disk every few seconds. Since disk writes are so slow, you can change your system to keep things in memory longer. All changes will be written to memory, and the excruciatingly slow writes to happen in the background while you continue working. This has an instant, noticeable effect, but it can lead to data loss.
Add these lines to /etc/sysctl.conf, and reboot.
vm.swappiness = 0 vm.dirty_background_ratio = 20 vm.dirty_expire_centisecs = 0 vm.dirty_ratio = 80 vm.dirty_writeback_centisecs = 0
The problem: using this tip means that your system stops writing changes to disk until you shut down or type "sync" at a command line. If your system loses power unexpectedly, you will get bad blocks. I did. You can limit the amount of data loss in the event of a power failure to one minute by setting vm.dirty_writeback_centisecs = 6000.
A side effect is that shutting down your computer will may take several minutes where it appears to be doing nothing. Don't cut the power until it's done, because it is busy writing all those changes to disk.
I added the lines from step 4 to /etc/sysctl.conf, and it has fixed my problem !
Runs perfectly now !
Thanks heaps Steve !!
Unfortunately, I bricked the installation twice with lubuntu and ventoy. after moving the usr to usr.old the bash forgets all commands. googling came up dry. I tried bin/mkdir usr etc. I think it is a quirk of ventoy and its scheme. fortunately, I backed up the 45 gig persistent.dat file and booted to windows to recopy to the usb hard drive.
Lubuntu worked best of 11 distros I tried on a memorex usb thumb drive.
We really need a icon to click to sync on the quick tray.
I could either control D to continue or hit enter. I assumed control d. It then tells me that a duplicate entry in fstab, made me wait 2 minutes, then shows me about 12 lines.
The usb stick is flickering, but I am not sure how long to wait before giving up and reburning the usb stick.
The swapiness suggestion seemed to work, and the firefox suggestion worked, except for the fact I needed to type about:config to turn off the cache.
- USB flash drives and SD card media have a finite number of read/writes cycles (typically 10k to 100k), before becoming unusable.
- Minimise read and write cycles on Flash drives if need to prolong the device life. Else make back-ups (over a network perhaps).
- Consider separating the core OS and apps from your session data and media files etc. One approach is using a write-protected SD card in a card reader as a main OS drive, and doing read/write duty intensive tasks on another device (regularly backed up using Flash Drive, cheap SSD, NFS/cloud etc).
- There's some confusion over whether Flash Drive optimised filesystems help reduce the wear on these devices. These filesystems from are still evolving in Linux (Ubuntu at least) and reliable Flash Drive suppliers claim have little to any impact. It's more the controller chip on the Flash Drive itself and how well that manages the duty cycle which counts. Also how well optimised the OS and apps are to minimise read/write duties.
- Likewise re transfer rates and performance, USB2 v USB3 has little bearing (unless dealing with very large files or large data streams) the controller chip on the Flash Drive is usually the performance bottleneck factor (unless you're moving 100's of MB's or GB's of data).
Some tips from a reliable Flash Drive supplier in your selection of using one as your primary OS drive:
1. Look for Type A memory type and ISO-9001:2008 factory certified manufactured Flash Memory for best quality and longest life.
2. SLC (Single Level Cell) NAND Flash Memory is best of all, but more expensive.
3. One of the best Flash Drive chip controller manufacturers is Silicon Motion (SMI) used in some of the better brands.
4. Rule of thumb: Buy a bigger Flash Drive than you need..ie 32GB if you need only 16GB, or 64GB if you need 32GB..the larger the drive the less concentrated the wear and so likelihood of early failure.
One reliable source for Flash Drives claims that 10 reads = 1 write. I always read write overheads were more costly than reads on this media!
Has anyone had any real-world experience of running a fully-fledged OS off Flash Drive or SD card media and any idea of life expectancy (with or without read/write duty cycle management)? I'd be curious to know how this compares to magnetic disk drives which Wikipedia claims average 6 years in an enterprise setting.
I'm running a Flash Drive system right now and find it a very usable and cheap option compared to continually replacing HDD's, however Flash Drives are incredibly slow when it comes to writing even relatively small files and do have a limited number or write capacity before they fail. It's said they fail a lot earlier than magnetic HDD's. Nevertheless, the very low cost, ease of backup and portability all offer benefits over HDD's unless you need massive amounts of data storage!
Therefore, to extend the life of your Flash Drive the ideal scenario is to write as little as possible on your main OS Flash Drive, or maybe not at all, write-protect it and have a separate data write-enable storage media alongside!
unionfs must use .squasfs has RO. The rate compresion of this filesystem is due to his structure. With ro fs you can purge all metadata not used. If a folder have 3 files only need metadata for these thee files instead the standard. And without data append, you can merge files (put together). And compress data if you wish.
After that unnecesary explanation, the bug is swap ro-rw flag, so:
unionfs /usr unionfs nodev,noatime,dirs=/.filesystems/usr/overlay=ro:/usr=rw 0 0
Will pretty if rw could be tmpfs
mount -t tmpfs -o size=20% tmpfs-usr /mnt/t-usr
and then
mount -t unionfs -o nodev,noatime,dirs=squashfs=ro:/mnt/t-usr=rw /usr
This actually makes disk read faster. Booting is quicker.
If a lot of new packages are installed updated, one can repeat TIP3, starting with "Make space for the filesystem" but make make /overlay1 and usr1.sqfs, edit fstab accordingly, go down to init 1, sudo mount -a: no errors? Reboot and delete usr.sqfs and /overlay
I also did the "dirty" stuff with sysctl.conf: This makes a HUGE difference in speed. I assume it is a little dangerous, but on a laptop, a power failure is not too much of an issue unless you don't let power management shut the computer down when the battery gets really low.
THANKS!
The challenge: Install Linux on a really old laptop. The catch: It has only 32 MB of RAM, no network ports, no CD-ROM, and the floppy drive makes creaking noises. Is it possible? Yes. Is it easy? No. Is is useful? Maybe...
Let's say you have millions of pictures of faces tagged with names. Given a new photo, how do you find the name of person that the photo most resembles?
In the cases I mentioned, each record has hundreds or thousands of elements: the pixels in a photo, or patterns in a sound snippet, or web usage data. These records can be regarded as points in high dimensional space. When you look at a points in space, they tend to form clusters, and you can infer a lot by looking at ones nearby.