See sound without drugs
I have created an application that just turns on the microphone and continually plots the FFT magnitude of what it records. It allows control over the window size and sampling rate.
Optimizing Ubuntu to run from a USB key or SD card
Posted 15 years ago
If you've installed Ubuntu on a USB key or SD card, you are probably experiencing the annoying slowness of Firefox. It freezes up for a couple of seconds every time you click a link. Like many things on Ubuntu, it doesn't work right out of the box and needs some tweaking. Fortunately, by following the tips below, you can make your USB or SD card based linux system fly!
Tip 1: Stop Firefox from writing to disk
Firefox 3 has a hard-to-fix bug that causes linux to write to disk every time you visit a write page. But it doesn't just write Firefox stuff -- it causes your entire system to dump all changes to disk. Unfortunately, your USB key might only have a 6 MB/s write speed, causing everything to freeze up.
- Under the Privacy setting, uncheck "Keep my history for..."
- Under the Advanced Tab, select Network and ensure that you use up to 0 MB of disk space for the cache.
Tip 2: Use preload
The preload daemon is a program that constantly looks at the programs you are running and figures out which ones you are most likely to use. When you start your computer, it automatically loads these programs and library from disk in the background, so when you start firefox, for example, it will pop up right away. The background is described in the author's Master's thesis.It's kind of like putting magnets under your pillow to improve health. Maybe it's having an effect, but I can't tell. I install it anyway:
sudo aptitude install preload
Tip 3: Compress your files
This tip can wreck your system, and to undo it you will need to be able to use a command line editor like nano, emacs, or vim. At worst you will need to mount the USB key on another linux system to recover (by editing /etc/fstab). If you can't do that, then skip this tip.On solid state storage, space is expensive. Ubuntu uses a huge amount of space will all the programs it installs. The /usr folder contains your programs, and it is usually 1.8 GB. Using squashfs, it can be compressed to 0.7 GB. Since read speeds are so slow, you can actually gain performance because there is less data to read. I've adapted these instructions from here.
Install squashfs and unionfs:
sudo apt-get install squashfs-tools unionfs-tools
Add the following lines to /etc/modules:
unionfs squashfs loop
Remove apparmor. Otherwise, the cups print server will stop working:
sudo apt-get purge apparmor
Make space for the filesystem:
sudo mkdir -p /.filesystems/usr/overlayCompress your filesystem:
sudo mksquashfs /usr /.filesystems/usr/usr.sqfsAdd these lines to /etc/fstab:
/.filesystems/usr/usr.sqfs /usr squashfs ro,loop,nodev 0 0 unionfs /usr unionfs nodev,noatime,dirs=/.filesystems/usr/overlay=rw:/usr=ro 0 0Switch to runlevel 1. (Ubuntu will close all open programs, then prompt you what to do. Choose opening a root shell)
sudo init 1
Move aside the old /usr directory and create a new mount point:
mv /usr /usr.old mkdir usr
Test whether you previously edited fstab successfully by typing:
mount -a
If you get error messages or your /usr directory shows up empty, either fix your fstab or undo the changes before continuing.
Now reboot and make sure it all works:
reboot
If it works, remove the /usr.old directory to reclaim the space.
Tip 4: Use memory instead of disk
This tip can also lead to data loss. If you do it, you will have to always shut down your computer properly from now on, because unexpected power failures will lead to data loss.
Linux usually ensures that all changes are written to disk every few seconds. Since disk writes are so slow, you can change your system to keep things in memory longer. All changes will be written to memory, and the excruciatingly slow writes to happen in the background while you continue working. This has an instant, noticeable effect, but it can lead to data loss.
Add these lines to /etc/sysctl.conf, and reboot.
vm.swappiness = 0 vm.dirty_background_ratio = 20 vm.dirty_expire_centisecs = 0 vm.dirty_ratio = 80 vm.dirty_writeback_centisecs = 0
The problem: using this tip means that your system stops writing changes to disk until you shut down or type "sync" at a command line. If your system loses power unexpectedly, you will get bad blocks. I did. You can limit the amount of data loss in the event of a power failure to one minute by setting vm.dirty_writeback_centisecs = 6000.
A side effect is that shutting down your computer will may take several minutes where it appears to be doing nothing. Don't cut the power until it's done, because it is busy writing all those changes to disk.
Steve Hanov makes a living working on
Rhymebrain.com,
rapt.ink,
www.websequencediagrams.com,
and Zwibbler.com. He lives in
Waterloo, Canada.
Post comment
edit
Tricky
three years ago
I am running Ubuntu from a USB3 stick and it would freeze every minute or so.
I added the lines from step 4 to /etc/sysctl.conf, and it has fixed my problem !
Runs perfectly now !
Thanks heaps Steve !!
edit
d garber
three years ago
Update: I did get everything working with lubuntu and rufus.
Unfortunately, I bricked the installation twice with lubuntu and ventoy. after moving the usr to usr.old the bash forgets all commands. googling came up dry. I tried bin/mkdir usr etc. I think it is a quirk of ventoy and its scheme. fortunately, I backed up the 45 gig persistent.dat file and booted to windows to recopy to the usb hard drive.
Lubuntu worked best of 11 distros I tried on a memorex usb thumb drive.
We really need a icon to click to sync on the quick tray.
edit
d garber
three years ago
The directions were good until start init. After typing start init there are several options, and I do not know which to do.
I could either control D to continue or hit enter. I assumed control d. It then tells me that a duplicate entry in fstab, made me wait 2 minutes, then shows me about 12 lines.
The usb stick is flickering, but I am not sure how long to wait before giving up and reburning the usb stick.
The swapiness suggestion seemed to work, and the firefox suggestion worked, except for the fact I needed to type about:config to turn off the cache.
edit
Paul L.Rogers
nine years ago
Anyone using a USB Flash drive as their primary OS drive (as I now do), should consider:
- USB flash drives and SD card media have a finite number of read/writes cycles (typically 10k to 100k), before becoming unusable.
- Minimise read and write cycles on Flash drives if need to prolong the device life. Else make back-ups (over a network perhaps).
- Consider separating the core OS and apps from your session data and media files etc. One approach is using a write-protected SD card in a card reader as a main OS drive, and doing read/write duty intensive tasks on another device (regularly backed up using Flash Drive, cheap SSD, NFS/cloud etc).
- There's some confusion over whether Flash Drive optimised filesystems help reduce the wear on these devices. These filesystems from are still evolving in Linux (Ubuntu at least) and reliable Flash Drive suppliers claim have little to any impact. It's more the controller chip on the Flash Drive itself and how well that manages the duty cycle which counts. Also how well optimised the OS and apps are to minimise read/write duties.
- Likewise re transfer rates and performance, USB2 v USB3 has little bearing (unless dealing with very large files or large data streams) the controller chip on the Flash Drive is usually the performance bottleneck factor (unless you're moving 100's of MB's or GB's of data).
Some tips from a reliable Flash Drive supplier in your selection of using one as your primary OS drive:
1. Look for Type A memory type and ISO-9001:2008 factory certified manufactured Flash Memory for best quality and longest life.
2. SLC (Single Level Cell) NAND Flash Memory is best of all, but more expensive.
3. One of the best Flash Drive chip controller manufacturers is Silicon Motion (SMI) used in some of the better brands.
4. Rule of thumb: Buy a bigger Flash Drive than you need..ie 32GB if you need only 16GB, or 64GB if you need 32GB..the larger the drive the less concentrated the wear and so likelihood of early failure.
One reliable source for Flash Drives claims that 10 reads = 1 write. I always read write overheads were more costly than reads on this media!
Has anyone had any real-world experience of running a fully-fledged OS off Flash Drive or SD card media and any idea of life expectancy (with or without read/write duty cycle management)? I'd be curious to know how this compares to magnetic disk drives which Wikipedia claims average 6 years in an enterprise setting.
edit
Paul L.Rogers
nine years ago
What about using a Linux Flash File Systems (JFFS2 and YAFFS) optimised for these devices?
I'm running a Flash Drive system right now and find it a very usable and cheap option compared to continually replacing HDD's, however Flash Drives are incredibly slow when it comes to writing even relatively small files and do have a limited number or write capacity before they fail. It's said they fail a lot earlier than magnetic HDD's. Nevertheless, the very low cost, ease of backup and portability all offer benefits over HDD's unless you need massive amounts of data storage!
Therefore, to extend the life of your Flash Drive the ideal scenario is to write as little as possible on your main OS Flash Drive, or maybe not at all, write-protect it and have a separate data write-enable storage media alongside!
edit
Victor
13 years ago
I think that 'Tip 3' have a little bug.
unionfs must use .squasfs has RO. The rate compresion of this filesystem is due to his structure. With ro fs you can purge all metadata not used. If a folder have 3 files only need metadata for these thee files instead the standard. And without data append, you can merge files (put together). And compress data if you wish.
After that unnecesary explanation, the bug is swap ro-rw flag, so:
unionfs /usr unionfs nodev,noatime,dirs=/.filesystems/usr/overlay=ro:/usr=rw 0 0
Will pretty if rw could be tmpfs
mount -t tmpfs -o size=20% tmpfs-usr /mnt/t-usr
and then
mount -t unionfs -o nodev,noatime,dirs=squashfs=ro:/mnt/t-usr=rw /usr
edit
Brian Harkness
14 years ago
On Jaunty, it seems unionfs is already in the kernel so that does not need to be installed, but IT DOES still need to be added to /etc/modules. Other than that, TIP 3 works wonders with space. I had 680 MB space on my EEE-PC 701 4GB surf, now I have 2GB.
This actually makes disk read faster. Booting is quicker.
If a lot of new packages are installed updated, one can repeat TIP3, starting with "Make space for the filesystem" but make make /overlay1 and usr1.sqfs, edit fstab accordingly, go down to init 1, sudo mount -a: no errors? Reboot and delete usr.sqfs and /overlay
I also did the "dirty" stuff with sysctl.conf: This makes a HUGE difference in speed. I assume it is a little dangerous, but on a laptop, a power failure is not too much of an issue unless you don't let power management shut the computer down when the battery gets really low.
THANKS!
I have created an application that just turns on the microphone and continually plots the FFT magnitude of what it records. It allows control over the window size and sampling rate.
Finding awesome developers in programming interviews
In a job interview, I once asked a very experienced embedded software developer to write a program that reverses a string and prints it on the screen. He struggled with this basic task. This man was awesome. Give him a bucket of spare parts, and he could build a robot and program it to navigate around the room. He had worked on satellites that are now in actual orbit. He could have coded circles around me. But the one thing that he had never, ever needed to do was: display something on the screen.Coding tips they don't teach you in school
Some time-saving shortcuts for C code that will make your coworkers scream. In Awe.
UMA's dirty secrets
Recently, many carriers have started offering UMA, or WiFi phones. These are cell phones with WiFi capabilites. Don't be fooled -- you won't be able to get free calls and run skype on them. The UMA technology is meant to extend the carrier's cellular network into your home using your broadband internet connection.Keeping Abreast of Pornographic Research in Computer Science
Burgeoning numbers of Ph.D's and grad students are choosing to study pornography. Techniques for the analysis of "objectionable images" are gaining increased attention (and grant money) from governments and research institutions around the world, as well as Google. But what, exactly, does computer science have to do with porn? In the name of academic persuit, let's roll up our sleeves and plunge deeply into this often hidden area that lies between the covers of top-shelf research journals.
Installing the Latest Debian on an Ancient Laptop
The challenge: Install Linux on a really old laptop. The catch: It has only 32 MB of RAM, no network ports, no CD-ROM, and the floppy drive makes creaking noises. Is it possible? Yes. Is it easy? No. Is is useful? Maybe...
Succinct Data Structures: Cramming 80,000 words into a Javascript file.
jQuery creator John Resig needs a little help storing lists of words in his side project. Let's go overkill and explore a little known branch of computer science called Succinct Data Structures.
VP trees: A data structure for finding stuff fast
Let's say you have millions of pictures of faces tagged with names. Given a new photo, how do you find the name of person that the photo most resembles?
In the cases I mentioned, each record has hundreds or thousands of elements: the pixels in a photo, or patterns in a sound snippet, or web usage data. These records can be regarded as points in high dimensional space. When you look at a points in space, they tend to form clusters, and you can infer a lot by looking at ones nearby.
A Quick Measure of Sortedness
How do you measure the "sortedness" of a list? There are several ways. In the literature this measure is called the "distance to monotonicity" or the "measure of disorder" depending on who you read. Here, I propose another measure for sortedness.