Unix and shell programming stuff

My private Procmail pages are not yet up; for now, have a look at the Procmail FAQ I maintain.

Contents

• Unix history
• Shell programming
• Unix books
• GNU utilities
• Procmail
• Emacs
• Perl
• VNC
• Open Source / Free Software
• Miscellaneous other stuff

Referenced local pages

• This page
• Some notes on installing Debian on a Libretto
• Useless Use of Cat Award and a couple of subpages; a glob example and a backticks example
• A short explanation of the ARG_MAX issue
• A local HTMLized version of Steve Bourne's original shell tutorial
• A local copy of an article by Tom Christiansen; GUIs Considered Harmful
• Local copies of the README and NEWS files for some GNU packages (links below)

Unix History

The Unix community has a rather strong interest in its own history. I have tried to gather some retro-Unix stuff and links to good stories here.

• Dennis Ritchie's home page has a lot of interesting historical material. Many classic papers and fun anecdotes.
• I happened to stumble over the Bell Labs apocryphal sources for the Unix 7th Edition manuals. The original Bourne Shell tutorial in this package, by Steve Bourne, is available in a HTMLized version I made.
• The 4.4BSD Programmer's Supplementary Documents and 4.4BSD User's Supplementary Documents are available from the FreeBSD Documentation site (along with a whole lot of other interesting stuff). It's supposed to be up-to-date documentation, but a lot of the articles are classical ones inherited from the good ol' eighties (and frankly speaking, some of them have quite clearly not been updated since then).
• http://dmoz.org/Computers/History/Operating%5FSystems/Unix/
• Plan 9 from Bell Labs might not (or might) be the future, but there's some stuff from the past on the site, too.

Shell programming

This is another piece that became so big that it had to be broken out into its own section.

• Heiner's Shelldorado is a very good source. Check it out. (The start page looks like it's a small site, but the links page alone is a one-month project, if you take it seriously!)
• I have a HTMLized copy of Steve Bourne's original sh tutorial
• The Consultix shell quoting guidelines
  (thanks to Brian Hiles for the pointer. Boo hiss to Consultix for spamming me -- twice. That sucks.)
• comp.unix.shell FAQs (mirror at ftp.funet.fi; you're in Finland now)
• ... particularly Csh Considered Harmful, but do check out the others, too
• The bash FAQ (it's finally on RTFM too, but it's good to have a reference to the original ... the site is occasionally not responding, though; you might want a link to the faqs.org copy.)
• The sed FAQ explains a lot of things that get asked in comp.unix.shell all the time
• Check out shc, the shell script compiler
• Paul D. Smith has a page about GNU Make
• For more links, the Open Directory Project has a good selection of shell programming links, too. (Better than Yahoo's Dead Links Forest, at least.)
• There's a list of regular expression links I've collected, although there is no definitive "one stop shop" site there
• I compiled a Useless Use of Cat Award page because I couldn't find one anywhere ... Have fun with it.

Shell programming for newbies

If I can find a good centralized links page, all the better. For now, this is mostly stuff I picked up on comp.unix.shell and might not even have checked out myself yet.

http://www.herring.org/techie.html

Very extensive link collection, haven't looked at it more closely yet.

(Thanks to Michael A. Powe for this one.)

http://www.unix.com

Online forum, includes good newbie forums

If you have more and/or better links, I'd love to hear from you -- and please let me know if any of these links break, Unix shell programming pages seem to be very prone to link rot.

Unix books

Just a quick pointer to some classics.

The Unix Programming Environment, (Kernighan & Pike 1984)

Old but nevertheless good. An excellent primer on shell programming, but also includes a lot of stuff not related to shell programming, such as Make, Troff, and the Unix standard library (for C programmers etc).

The errata page has some helpful contributions by yours truly :-)

The Practice of Programming (Kernighan & Pike 1999)

Good programming habits; examples in C, Java, and Perl. Not particularly Unix-centric (actually I believe the authors use Plan 9) but very much on the right wavelength for anybody who likes Unix.

Mastering Regular Expressions (Friedl 1997)

Must read. Must. Read.

Advanced Programming in the Unix Environment (Stevens 1992)

C programming and the library; explains and relates to various standards such as POSIX, XPG3 (now XPG4, of course), SVID, etc. (Somewhat ignorant of Linux, but that's probably only for the better.)

Unix System Administration Handbook (3rd ed.; Nemeth, Snyder, Seebass & Hein et al. 2001

I don't yet own the 3rd edition, but I own a copy of the 1st and 2nd eds.

The new edition now also covers Red Hat and FreeBSD, where the previous edition was predominantly "expensive" vendors (HP-UX, Irix, OSF/1, Solaris [now free, of course], SunOS, and BSDI).

GNU Utilities

A constant need for many is to find Unix shell utilities which do not stink. Frequently user-level software on many platforms is written to barely conform to spec, so that the programmers can spend more time to introduce new and better incompabilities with the competition in their kernel-level code.

I have found the GNU utilities to generally be robust, user-friendly (in the Unix sense, i.e. if you know what you're doing, it's doable), mostly devoid of unnecessary restrictions (both in terms of who's allowed to use and copy them and in what input they will accept) and often simply much better than what Sun, HP, Digital, NetBSD (which is otherwise very nice) and generally everyone else would want you to have. (Linux comes with GNU utilities preinstalled by default.)

There's a sort of web page at http://www.gnu.org/ but it doesn't contain very much about the GNU packages I'd like to see introductions to. Perhaps you should just cruise around in ftp://prep.ai.mit.edu/pub/gnu/ (or one of the many many mirrors worldwide) and see what you find.

As for the standard Unix command-line tools, I have local copies of some distribution files (these versions are growing oldish, though) for the fileutils (README and NEWS), sh-utils (README and NEWS), textutils (README and NEWS), findutils (README and NEWS), and grep (README and NEWS) stored locally.

If you're at all interested in Unix shell programming, you want these.

Of course, GNU's Not Unix and Unix is not GNU, but they obviously have something in common. (See also Emacs and Perl.)

Procmail

If you download this, make sure to get version 3.13.1

• Introductory material: Infinite Ink's Mail Filtering and Robots page and when you've read that, visit Infinite Ink's Procmail links
• I have collected a mirror of stuff I find useful at http://members.xoom.com/procmail/ -- check it out.
• Tangential material: Eli the Bearded's addressing FAQ and an Autoresponder FAQ from Hell by yours truly
• My own feeble attempt at a Procmail FAQ could be improved a great deal, I'm sure. I also have a page with Procmail links. Make sure to send me your comments!
• A page about debugging Procmail recipes (and also something about getting Procmail to run in the first place) which I actually might try to fold into the FAQ one day, or perhaps it should only sit here for a while until it feels ripe. Feedback welcome!

Emacs

Of course, Emacs isn't Unix, and Unix isn't Emacs, but people who recognize quality tend to like both. (See also Perl.)

A great part of this section is just stuff I cut && pasted out of the main section, because it started to look a bit like "Emacs and what else".

Note that the Emacs FAQ is nowadays included in the distribution. For newish GNU Emacsen, press C-h F to view the FAQ in a buffer.

http://tiny-tools.sourceforge.net/emacs-elisp.html

Jari Aalto's huge list of Emacs Elisp references -- including a lot of pointers to downloadable add-in packages.

http://www.emacs.org/

Augh. This used to be a useful site. Then it was revamped. Then it was down. Then it sucked even worse after having been revamped again. Then it has been oscillating between "down" and "useless" and "revamped but still useless" with some all too short deviations from this pattern.

Occasionally ftp://ftp.emacs.org/ and / or http://ftp.emacs.org/ has contained useful stuff, but don't count on that, either.

Mirror of the LCD archive

Local European mirror, of course, at Funet, of course

Here's the searchable version of the LCD archive.

Emacs implementations

Of course, there's only one True GNU!

Craig Finseth's book

The Craft of Text Editing: Emacs for the Modern World is now available online (exclusively)

w3-mode

-- documentation and all!

(What kind of documentation, I hear you ask ...)

Other gripes:

• It's too big and slow (I used to think so about v2, but look at v4!)
• The next version inevitably breaks things that worked fine in the previous version that you had learned to rely on
• ... and the key bindings keep a-changin', forget about "plug and play" upgrades, it's a major reconfiguration and relearning experience each time

Still, it's better than not having a browser inside Emacs.

VM the mail reading package

My gripes:

• It is not "extensible"

  Lots of menus etc are defined with defconst so you can't customize things without a lot of workarounds.

• It is not "self-documenting"

  Looking for documentation for what a particular function does? Forget it. It's not there.

• It is not "programmable"

  This is a corollary of the two previous items. There is nothing there to prevent you from trying, per se, but you need to be masochistic or stupid or very very bored if you want to try to do anything serious.

It's still a lot better than RMAIL, or than using something else than Emacs as your mail environment. And don't get me started about Gnus ...

Various useful VM links:

• The VM FAQ by Brian Gorka (now maintained by Samuel Mikes)
• Emmett Hogan's VM Add-Ons page
• Samuel Mikes's FAQ about using Procmail with VM (see the Procmail section above if you wonder what this means -- you don't want to miss it)

Trey Jackson's index of MIME tools for Emacs

The Church of Emac$

http://www.splode.com/~friedman/software/emacs-lisp/

Noah F's Lisps (including some VM add-ons, too)

If you are looking for my Open Source spook.lines file, scroll down a screen or two.

Perl

Of course, Perl isn't Unix, and Unix isn't Perl, but people who recognize quality tend to like both. (See also Emacs.)

The purpose of most computer languages is to lengthen your resume by a word and a comma.

   -- Larry Wall

The WWW is shock full of Perl links. I get along fairly well with stuff directly or indirectly available from the following:

• http://www.perl.com/
• http://www.tpj.com/
• M-J Dominus' Perl Paraphernalia

I'm not a very eager Perl links collector, though. YMMV.

VNC

It's screen for X. No, it's a multiplatform graphical screen. It's worth to kill for. But it's free. 'Nuff said. http://www.uk.research.att.com/vnc/

Here are some useful extras (some of them are also linked from the AT & T pages):

• Here's a DOS viewer: http://www.complang.tuwien.ac.at/nino/dosvnc.html
• Here's a monitor program for watching several VNC desktops: http://www.wilson.co.uk/Software/vnc/VncMonitor.htm
• And here's a canned search for the precompiled Debian packages (follow the links and you'll find the client package as a "suggested addition" on the server's download page)

Open Source / Free Software

Of course, Open Source / Free Software isn't Unix, and Unix[tm] most definitely isn't Open Source / Free Software. But there are reasons why this is here nevertheless. (See also Emacs and Perl.)

This is really just a quick link collection in case you're trying to pick up on this topic.

• http://www.opensource.org/
• http://www.gnu.org/
• http://www.slashdot.org/
• http://www.tuxedo.org/ (Eric Raymond's site)
• Search on dmoz.org for matches on "open source" (check out some of the material about dmoz.org itself, too)
• http://www.ora.com/ and http://www.perl.com/ frequently have open source-related material on their start pages
• Salon Magazine has run an impressive series of articles about the free software / open source movement
• http://www.wired.com/ will also obviously have a large amount of relevant material. Here's a canned search: http://www.wired.com/news/news/search_results_news?words=open%20source
• Forbes Magazine article (yes, even the "duh, my quarterly bottom line" suits are catching up)

A (meager, but still) contribution of mine is the subversive Open Sounce Conspiracy spook.lines file which you can download here. (The Emacs spookmime.el package by Noah Friedman is curiously not available from his collected works page but you can pick up a copy from the VM Add-Ons page.)

Miscellaneous

Libretto notes

In case you missed it earlier, I have written up some notes on installing Debian on a Libretto.

http://www.jwz.org/

Attitude. Weirdness. At least Jamie works hard to deserve his fifteen minutes in the spotlight.

Free Unix OCR software

http://documents.cfar.umd.edu/ocr/

"Unix is the ultimate computer virus"

Ironically, the guy who coined the phrase "Unix is the ultimate computer virus" was rather critical of Unix and C. The guy is Richard P. Gabriel and the quotation is from his 1992 paper Lisp: Good News - Bad News - How to Win Big

... which brings us to the Unix Haters.

Don't be fooled by the silly name, this is not "Windoze r00lz so FOAD" lusers on a rampage. Some of the criticism is very insightful.

We Are Smart

(A-g-a-i-n, originally snatched from www.perl.com's What's New)

Of course, this contains a Useless Use of Test!

	if test -z `ps -fe | grep whom` 
	then 
		echo ^G 
 	fi 
	# Let's see for whom the bell tolls. 

Or is it Useless Use of Backticks? Anyhow, the way to code that is

	if ps -fe | grep whom; then ...

Oh! And another one:

	cat menu | grep shrimp

Yes -- this is the classic Useless Use of Cat but I shall leave fixing it as an exercise to the reader.

Perhaps we aren't so smart after all?

The Tao of Backup

... Slick design, if your browser can display it.

Dumbest Y2K Quotes Contest

And here's a link to Tom Christiansen's article about the Y2K hype contest (recommended reading for IT managers)

How to print FrameMaker preformatted PostScript on A4 paper

This is one of the few things I (indirectly) found an answer to from http://www.askjeeves.com/ -- this is not an endorsement; I've failed to find more answers there than I care to remember.

This has nothing with Unix to do, of course, but it's information I could have killed for on more than one occasion.

http://www.math.auc.dk/~nwp/ps/fm4/ is the place to go. Bookmark, mail it to all your friends, send beer to the guy who made the page. (Bookmark the parent page also.) While you're frothing anyway, persuade a nearby mirror to mirror it, too!

Mysteriously, the FMLeve1 patch tip is also available from sites like http://java.sun.com/docs/a4.html and http://www-sop.inria.fr/koala/lehors/xpm.htm

If you sort of enjoyed these links, perhaps the misc section of my regular hotlist will be found to be entertaining.

Other programs

• par is a program for reformatting paragraphs (somewhat like the standard fmt utility, only smarter).
• lftp -- finally a decent command-line ftp client. Good-bye, ncftp -- good riddance. (Okay, so this one isn't exactly rock solid either.)
• sls is a hacker-friendly version of ls(1) by Rich Baughman for those times when you need a good simple interface to file sizes, modification times, access times, etc. in a "less user friendly" format :-) than you get with ls
  I've been receiving requests for pointers to the original author and/or a version without Y2K problems. I've been in touch with the author and received a copy of his most recent source, which does not have a Y2K problem, but still contains some Sunisms.
  A patched version with some contributed patches is maintained by yours truly and available from http://www.iki.fi/era/unix/sls.tar.gz or just the diffs from the above in http://www.iki.fi/era/unix/sls-diff.gz
  (Somehow I originally thought this was a program by Rich $alz but that is not the case. Apologies to both Riches for the confusion. Also, for a very long time, the tarball was broken. My sincerest apologies to everyone who tried to download it before December 2001. I'm very embarrassed.)
• webcopy is a simple but versatile tool for fetching web pages. I got this in 1995, before wget was even conceived, but I find this one still works better for me. (wget is incredibly frustrating in some respects. I should perhaps put my rants about it on a separate web page some day :-) I note that many hosts on pgp.net use webcopy, too, so I guess I'm not the only one who likes this one.
• Procmail -- see above and elsewhere.
• Emacs -- ditto.
  If you are looking for my Open Source Spook Lines, scroll back a screen or two.
• Did I mention VNC?
• Perl?
• CVS?