Ok guys, here is the rough draft of what I plan on writing about Linux From
Scratch. It really is very rough. The copyright information is mostly so
no-one steals it before I get to finish it. Once its finished, I don't really
care what you do with it =)
Thanx ahead of time for all suggestions and feedback.
----------------------
Linux From Scratch
Compelling Reasons to Compile
or
Not For The Faint of Heart
By Scot Mc Pherson
Caveat
I use a few phrases interchangibly. Base system is a phrase I use both
for the original install distribution linux system, and it is also my
description of what a completed LFS system is, it's a base system. Sorry for
the confusion, but base system works very well for both situations. Reading
it in context should make it clear what I mean. I try to make it clear in the
sentence or paragraph which I mean.
This white paper may be reproduced and distributed at will so long as the
following conventions are adhered to. This document must be duplicated
exactly as is, you may not use anything out of context and you may not use
peices of it in an article or research paper or other publication. If you
want to publish it, go ahead but I ask that you inform me first and give me
the opportunity to polish it. It must be published in its entirety.
Introduction
I am not going to go through the typical introduction to Linux that many
authors have provided seemingly with each article written about Linux. I am
going to start by saying that Linux is an addition to the Unix family of
operating systems. It was written by a Finnish hacker named Linus Torvalds
and was initially released in the fall of 1991. It and its acceptance has
grown ever since. That is all on that.
Do you think you know a lot about linux? Well unless you are some sort of
ubberhacker or Linus or Alan Cox, LFS (Linux From Scratch) has something to
teach you. Until you actually have to hunt down tar.gz files on the net and
figure out how to compile and install them successfully and to your liking,
you haven't experienced linux. You are reading this either because you want
to be convinced to use LFS or you want to find something to debunk. Either
way, read on. At the end I am going to go through a point by point pro and
con to LFS, and I am also going to cover some very common arguements both for
and against LFS. The funniest part about writing this is that I discovered
that the most common arguements I hear both for and against LFS are almost
identical to those arguements between Linux users at large and MS Windows
users. So if you think you have something to say on the MS vs Linux front,
then you can bet you can apply the same arguements here.
Today we have many choices in how we implement linux systems. We have
offerred to us easily over 100 different flavors of linux operating systems
ranging from the emmensely popular and well recognized distributions such as
RedHat, Debian, Mandrake, Slackware, and SuSE all the way down to more
obscure implementations like YellowDog, Wolverine and other distributions
skilled hackers have put together. Lets not forget the utility distributions,
such as tomsrtbt which are used to diagnose and repair your system when all
else fails. So much to choose from, and yet they are the result of other
people's efforts and therefore other peoples visions of how a system should
operate.
Have you ever had that moment when you become disatisfied with your
current favorite distro and renew the hunt for the perfect distro? I know I
was in that situation when RedHat-7.0 was released. I was the perfect RedHat
customer and user. I purchased a copy of every CD RedHat produced from
3.somethingerrather all the way up to 6.2. RedHat was a dream to work with.
It was stable, it did what I configured it to do, and it didn't argue, too
much. I never even considered looking elsewhere. Then RedHat-7.0 hit the
shelves. Oh man, what a bummer that was. The distribution grew to outragious
size and was thoroughly broken to-boot (an expression, not system boot.) I
went through installing it 50 times and running up2date and all the other
fancy schmancy fix it schemes provided on www.redhat.com, but yet it always
seemed to fail to meet my expectations of stability and usability. I couldn't
compile software, and to make matter even worse they pulled the carpet from
beneath me when they completely reorganized how the filesystem was arranged.
/etc had 100's of files even if I wasn't using most of them and /dev was full
of devices I never even heard of. Now let's be clear. I am not bashing
RedHat, I think its a fine company which produces a fine product. My issue
was and is with distributions as a whole. If I were to suggest a distribution
to a newcomer of linux, I would suggest they begin with either RedHat,
Mandrake, Slackware, or SuSE. But for me, I needed something more (or less
really.) Enough was enough.
The Search Begins
I began my search. I won't list all the distributions I researched and
tried, there are too many. But I will give you a summary of what I came
across. I found SuSE with too much stuff and too big if you where downloading
and wanted the neet stuff, Debian was way too old (XFree86 v 3?) unless you
want to play with the CVS versions, and what the heck is dpkg? It was very
confusing sorting through all the deb files. At least RedHat provided a
pretty complete discription of what each obscure RPM provided. Slackware was
probably the cleanest system I found, but you had to choose to install large
packages to get one or two things out of the package. Slackware presently is
my favorite distribution I don't consider "Linux From Scratch" (LFS as I will
be calling it from here on out) a distribution. LFS is a system you compile
completely from source code and not an installed distributed linux, if I did
lump it in with distros Slackware would be my 2nd favorite. Mandrake is
pretty much RedHat compiled for i586s, which is a good feature if you are
performance conscious. Later I will explain why. Ok none of these was doing
it for me. Some were worse some were better but since I was searching for
"my" new system it had to be perfect.
After a bunch of other systems I came across a website in November of
2000 which at the time was rather obscure http://www.linuxfromscratch.org. I
browsed the website and I talked to the people on the mailing lists and the
irc chatroom (irc.linuxfromscratch.org). It was fairly easy to convince me
that I wasn't going to find what I wanted in other distros, I had already
proved that to myself.
The First Attempt
I being a person of wanting ensured stability began with LFS-2.4.2. Let
me explain something here. "Linux From Scratch" is the name of a book. Its
not a distributed set of binaries, and isn't even a distributed set of source
code. http://www.linuxfromscratch.org does provide the necessary and some
option source code as a convenience to its user, developer and tester base,
but you have rest assured they are unadultered copies of the orginal sources.
To say it again, they are merely gathered together in a single place for our
convenience. If you wish to hunt for and gather all of the sources yourself,
be my guest. When you have done so, do a checksum on them and the files you
can download from www.LFS.org (remember I am shortcutting the name). You'll
find you wasted a lot of time, but sometimes experiencing hardship for
yourself is the best way to learn. Anyway back to the issue at hand.
It took me some 3 weeks to compile my first system, although I have an
excuse or rather a few excuses. I have my own family, my wife (Ginger) and my
2 boys (Davis 3, and Morgan 10 months). So between my family and my
meticulous and methodic combing of the LFS book contributed to the length of
time it took me that first time. When I had completed it, I erased the old
distribution I used as a development system to build this one (yes you need a
linux system to build LFS, but I will get to that later too.) And now what?
Hmm. Oh man, what do I do now. I can't do anything. Sure I have a linux
system and its brand spanking new in a few senses of the word, but it doesn't
do anything. Boy did I have a lot to learn, I hadn't really thought this
through. Ok, so I re-installed the old linux system so I could get out on the
internet and download a few client packages like lynx, and lukeftp. Ok NOW I
have the tools I need to get more tools. Oh well, I guess I didn't think it
through first.
Continuing My Education
The second time I compiled LFS, it took maybe 3 days and now that I have
done it over 200 times I can get it done in about 4 hours on a fast computer
around 500MHz (including installing the base system - a modified RedHat). It
shouldn't take you 3 weeks your first time. I was one of those people that
had to read and check and recheck each command I hand typed. And like I said
earlier I had my family competing for my time. You can copy and paste with
GPM if you like. I typed it out the first few times, because I was concerned
about the indenting and such things. Now I realize I didn't need to worry
about it and I am now the GPM copy and paste king.
Slowly I added to my fileserver various bits and peices of
necessary/optional code and built myself many systems. From LFS-2.4.2 I moved
on to 2.4.4 which was much nicer. After 2.4.4 the version 3 prereleases were
coming out. I have always been afraid of beta software. But with some proding
of the LFS staff and user base I tried LFS-3pre2. I skipped 3pre1 altogether.
Man what I difference. It was much easier to build, and seemed more solid. I
kept on building systems (oh hey, did I mention this becomes a very addictive
hobby?? Well it is VERY addictive) and became slightly dissatisfied with
3pre2, and I started developing my own "fork" of LFS. 3pre3 was released and
I hated it (well hate is a strong word, but I didn't like it anyway). I
skipped 3pre4, and finally a few months ago (Aug2001) LFS-3.0 was released.
Immediately a problem was discovered regarding stripping of libraries, which
rendered a system useless. Ugh...ok now my next step into the development
world. The CVS version of LFS was considered to be plenty stable, and again
with proding, I was convinced to try it. Remember this is a book, not
software. CVS versions of LFS are not exactly cutting edge either. CVS could
easily be considered the "current state of the stable LFS". Testing of LFS
systems is mostly dealt with in the bugzilla system and changes aren't
implemented into CVS until and unless the changes have been tested by the
author and project leader of LFS (Gerard Beekman) and bunch of users which
make sure a system is regression tested before the system is committed to
CVS. Regression testing is a pretty thorough test of almost every part of the
LFS base system. Regression testing includes compiling and using GCC,
XFree86, QT, KDE and GNOME. This might not seem like much, but until you have
tried it, don't scoff. Testing by compiling, installing and using those 5
packages is a very thorough testing of almost each and every peice of
software included in the LFS book. If a system can successfully compile and
run those packages and desktops, then its pretty darn stable.
These Days
Now a days I get to use my systems. I am pretty much finished with
tweaking and tuning and recompiling everything. My systems do what I want
them to do, and unless something serious happens (like I get a new project at
work or a new computer, or one of my system crashes so bad that even back-ups
don't work) I won't be rebuilding LFS too soon. I like things the way they
are and its nice. I have a network of 6 systems. I have a p150 w/32MB RAM
which is my firewall, apache and ftp server...a Dell Optiplex Piii866w/256 MB
RAM which runs as my fileserver and back-end to my web and ftp services and
also houses my /home directories (all using /dev/ndb). My GNOME-1.4
workstation is an original AMD700 w/512 MB RAM and an AMD k2-333 which is my
wife's WIN98SE box (hey what can I tell you?). I have an AMD486 w/16 MB RAM
which I am going to build into my new firewall/router ONLY box, and return
the p150 to service as the front end to my net services without running as a
firewall/router. I have a p200 w/160MB of bad RAM which when I get the RAM
replaced I will use that as my SETI box and also use for my C/C++ coding.
Once again all /home directories are housed on my fileserver using net block
devices, except for the firewall. I am learning how to program better than
just being able troubleshoot some existing code. I run samba on all my boxes
for print services into the fileserver. If you have a 386 you want to get rid
of, give me a holler.
Suggestions When Building LFS
A lot of this won't make sense until you have read the LFS book. I
suggest reading the book either before reading this, or coming back to this
section again after you have read the book.
Take the time to plan your partitions intelligently. Smart partitioning
alleviates fragmenting of system files and this improves performance. If I
have two hard drives I do things similarly, but I make sure my system files
are always on primary partitions ALWAYS...if the extended partition gets
messed up, then I don't loose EVERYTHING, and can often recover from the
failure. If I have multiple drives, I put a swap partition on all of them. I
always put swap in an extended partition so I have my primary paritions for
system partitions. This is the partitioning scheme I use:
/dev/hda1 / 130 MB
/dev/hda2 /boot 4-12 MB (depends on your hard-drive)
/dev/hda3 /usr 300-1200 MB (depending on s[ace and what I plan
on doing, this is system software)
/dev/hda5 <SWAP> 48-256 MB (I usually just assign a default 128MB,
but if I am short on space, or have more space than I know what to do with I
adjust)
/dev/hda6 /usr/local 300-1200 MB (Depending on space and what I plan
on doing, this is for your user software)
/dev/hda7 /tmp 100 MB (you want this partitioned for a few
reasons, so that temp files don't overrun your system
/dev/hda8 /var 20-500MB (depending on space and how much logging
and state information you want to store, you definitely want to partition
this too, to keep runaway logs from taking over your system...it CAN happen,
I have seen it)
/dev/hda9 /opt 300-1200 MB (depending on space and what you plan
on doing, this is for GNOME, KDE, etc and your GUI Software, entirely
optional as its name implies)
/dev/hda10 /home Whatever is left...
Use CVS... Its perfectly stable, and its just a book, but CVS does seem
to be the most stable. Funny that. Really use the nightly CVS snapshot.
Do it twice. I mean it. Some poeple think its a religious thing, but its
not. If you LFS, and then immediately LFS again, you divorce your system
further from the distribution you started with, by an extra 2 steps. People
argue that it's ineffective, arguing that by created the chrooted environment
that one has created an environment protected from the initial base system. I
don't quite buy that, because the software you compile into chroot is created
using the glibc and other libs resident on that initial base system. The
second LFS process ensures that your LFS is built by LFS, and honestly this
is the best way I know how to control a system. By ensure consistency, I know
what I am going to have as a result. LFS is the best system to build LFS
with. I have tested the results and its highly suggested by me. Install
Distro linux to /dev/hda10 above, build LFS into /dev/hda9 or /dev/hda6 or
any partition that is larger than 1000MB. The next time you build LFS, you
are going to build it into the final system using all your partitions. Also,
if you partition thusly, use another partition for $LFS/usr/src. This will
help you reduce the necessary space for $LFS/usr if you need to keep it to a
minimum. You will still need/want at least 300 MB for $LFS/usr if you plan on
building a "complete" linux system. Reduced /usr partitions are mostly for
specialized linux systems that won't use many of the common tools found there.
Start with RedHat, although its a mess its the easiest to install. All
you need to do when installing RedHat is select Custom System. Partition
Smartly, and install RedHat into the /dev/hda10 above. Select Development
when you are selecting packages (don't worry about anything else, all you
need is already selected (except maybe IRC if you want to chat or get help
while you are doing this). When you are done installing RedHat, immediately
recompile gcc and replace it with gcc-2.95.2.1 which is a patched gcc-2.95.2
that was made specifically for building against RedHat's faulty compiler, do
this even if you have updated RedHat's GCC or have installed 7.1 or 7.2. You
are ready to LFS.
Use the linux-2.2.19 kernel. Unless you have a good reason to use 2.4
(such as netfilter), then don't bother. Hunting for a good 2.4 kernel that
works with your hardware and just plain works at all is a royal pain in the
tookas. I consider linux-2.4 to still be in development regardless of what
Linus happens to say. linux-2.2.19 always works, always. Its the most stable
kernel in existence right now. The 2.4 kernel's stability and usefulness
depends a LOT on your hardware and what you plan on doing, don't use it if
you don't have to. That means in both places, chapter 6 and chapter 7.
Most software you can upgrade, replace or remove with abandon. There are
three you can't (or really don't want to.) They are your kernel, glibc and
ncurses, all for much the same reason. These packages include headers and
libraries that get compiled into almost every peice of software on your
system. Sometimes these headers get changed enough in a new revision to make
a drastic difference on how your system is compiled, and you can break
something. When you replace your kernel, make sure you keep your old kernel
for an extended period of time, in case you discover something wrong with
your system running the new kernel. FYI I don't delete old kernels at all.
Glibc _can_ be replaced, but be very warned, if it doesn't build right or if
it breaks you are _royally screwed_ .. ncurses is another set of libraries
and headers that much of your software depends on, if it changes enough you
will break a lot of software that isn't system critical, but will prevent you
from doing many of your daily chores...again don't do it...You have been
warned.
Stay away from gcc-3.* its broken, I don't care what anyone else has told
you or whether you have seen it compile a kernel or not. If you do, you'll be
sorry you did when you finish your system and find out the system is broken
here and there and you are trying to figure out why. Use gcc-2.95.3-2, it
works its nice and its stable.
Use glibc-2.2.4, although it was meant to fix problems with gcc-3.* its a
major improvement over glibc-2.2.3 regardless of gcc selection.
Download both lukeftp and lynx BEFORE you begin and place them in the
tarball directory that houses your LFS sources. You'll save time and
frustration by not having to reboot back into the old system to get this
stuff.
Know what your system settings need to be before you begin, especially
settings for routing and eth0 or ttyS0 or whatever. This will save
frustration. If you need to refer to your Base System's settings, then go for
it.
Keep Notes. If you build LFS following the book to the letter, then to
book is your note book. If you diverge even a tinsy bit, write it down and
keep it handy. Keep notes on how you install all software, you will need it
if you need to remove something or need to remember where you installed it,
because some other package doesn't seem to find it on its own. Don't loose
your notes...This is important. You could end up installing a library in two
locations by accident and that can cause all kinds of hell later in the
future.
The first thing you should install after LFS is GPM, including before you
start LFSing the second time...trust me...read the hint to get it compiled
otherwise you won't get it compiled.
hints are found at http://hints.linuxfromscratch.org go there, download
them all and read them...read GPM first.
Learn to like the console...its has much better, more stable software
than X. I use X sometimes when I really want graphics support, but console
really is better for functionality, it uses less resources and the software
almost always much much faster ... 9 out of 10 linux gurus agree. Ok that's
more generally linux oriented and not LFS oriented, so I will stop there.
Pros and Cons
Here is the part where that you linux die hards should appreciate. You
should notice the similarity between what is said here, and what is said in
the MS vs Linux war.
PROS
The very first and foremost pro for using LFS is Education. Flat out,
this beats any other advantage that LFS has to offer. By the time you have
learned how to compile your own linux system using only the book for tips and
hints, you have become a system developer and not just user...Don't get me
wrong, you'll still have lots to learn about system administration and other
stuff before you are a guru, but you are more guru now than ever before. You
will learn how to modify and maintain source code. You will learn how your
computer works, and right down to low level CPU functions if you go the hard
core optimization route.
Stability. This is the second most important reason for building LFS.
When you take the time to regression test your system and fix any minor bugs
that do occur occasionally, you'll have arguably the most stable system you
could find. In essence this is by the way, what distribution developers do.
In their own fashion they build their own specific LFS and test rebuild and
test and test and then package.
Performance. LFS is native. That means it was built on the system it is
running on. If you allow gcc to discover the sort of system hardware and
software you are using, you'll have a pretty good optimized system that by
default will outperform any distribution on the market, guaranteed. There is
a reason for this. Distributions need to build linux systems for the widest
set of platforms. Within a family, this means compiling their distribution
for the lowest common denominator (i.e. the Intel 80386 in the case of the
intel family and its compatibles). The exception being Mandrake which targets
the Pentium Classic as the lowest platform. Essentially this makes your
Athlon1.2 GHz computer a 1.2 GHz 80386. None of the advantages are the newer
architectures are used, because they can't be if the software must also run
on a 386. So you gain a great deal of performance, up to 30% for just the
default compile time options in some cases.
Lean-ness. LFS will be very small compared to similar installations from
Distributions. If you were to strip RedHat down to the same files that you
finish with on a LFS system. It would still be a lot bigger, this is in part
for the same reasons why you gain performance with LFS. As well, instead of
having to hunt down files that aren't going to be used, you instead are faced
with the joy of only installing that which you absolutely want or need. I
have a fully featured GNOME workstation loaded with software. The
installation is big, but its much smaller than a RedHat GNOME workstation,
and it never crashes. I have only installed software as needed and so I know
there is nothing superficial or superfluous about my software.
Customizability. This could really be put anywhere in the ranking of
pros. It might be your first priority or your last. This is also very easily
the most used arguement for why people might select linux over MS Windows,
and it also the most used arguement for building LFS. You can do anything you
want with LFS including putting together a fat desktop and multimedia
workstation with GNOME, KDE and GNUStep automatically booting into runlevel 5
and having xmms playing your favorite playlist and watching your favorite DVD
movie with mplayer. If you don't like using /usr/local in the traditional
manner and want to install all user software in /usr/local/PACKAGENAME go for
it. Your PATH and ls.co.conf will be huge, but hey it can make software
removal a breeze (rm -rf /usr/local/PACKAGENAME). Alternatively, you could
fashion a DOS-like filesystem structure if you really want. Yes you could
customize RedHat to that extent, but go ahead and try...Its a much bigger
head-ache than customizing LFS. Here's the kicker and anti-arguement for
anti-LFSers, I'll be done long before you. LFS just lends itself to
customization since its very basis is being compiled entirely from sources.
Compartimentalization. This is really a combination of the last two
above. With the lean-ness and custimizability inherent in LFS. LFS is well
suited for singular tasks that are best suited for older hardware freeing up
newer hardware for hard-core tasks. Everyone knows about using [3,4,5]86s as
firewall, but how about using a 586 for a firewall and a 486 for dhcp, WINS
resolution and DHS, you could pack a few other seldom used services as well
on there and it'd still perform really really well, and maybe better than a
firewall running those services too. You can leave the firewall alone to do
it job as a router/NAT/firewall and so it performs better too. Ok, I am
getting into that general linux territory again. Stop.
Experimenting. You can play all you want, and if you break something you
probably now have the wherewithall to figure out what you broke.
Understandibility. When you are finished building your complete linux
system with all the junk you could want on it and tweaked to the brim, I want
you to compare your system with a RedHat or Debian or any other distribution
system. Look at /etc...geez what the hell is all that crap? Compare your
~/.bash_profile ~/.bashrc and ~/.bash_logout with RedHat's ... See any
difference? A huge one I will wager.
Support. LFS builders are some of the MOST knowledgable linux users in
the world. If you hop onto irc.linuxfromscratch.org you will be greeted by
friendly faces, people who are MORE than willing to help out a fellow LFSer.
If they can't help you directly on irc, then you will likely be directed to
the mailing lists or the news server (which is a mirror of the lists.) You
can also search the mailing list archives to see if someone has already
encountered and solved your problem. Its very likely. On a not so rare
occasion a fellow LFSer might even try and duplicate the problem on their end
to see if they can solve the problem. LFSers are almost by definition problem
solvers.
CONS
Education. You will be forced to learn how to do this, and it does take
some time to become accostomed to this new method of system building. Gone
are the days of just sticking in a CD and booting. You will have to make
intelligent choices, scrap ideas and start anew when an idea doesn't pan out.
Notice this is also listed in PROS.
Time. It does take time to develop a system. As you become more familiar
with compiling a system completely from scratch the time IS reduced VERY VERY
much. Again my first time it took 3 weeks (with legit excuses), and now it
takes me 3-4 hours. Before you think of this as too much of a disadvantage,
how many times have you sat in front of a linux installation screen and
labored over what to install and how. And how many more times have you spent
4 hours installing linux only to say to yourself...Oh man I wish I hadn't
done that. Its the same deal, with familiarity it doesn't really take much
more time to build linux than it does to install it (of course I AM talking
about on faster computers, a 386 can take days to compile LFS where a CD will
take maybe 5 hours.)
Frustration, Bruising and Headaches. Sometime when you are trying to
install some software, you'll have one heck of a time. You'll bang your head
on the desk (hence the bruises) and you kill anyone that comes within arms
length. This isn't the norm, but it does happen sometimes. Usually it will be
due to some earlier installed, and required software not being recognized
properly or source code that isn't up to the current state of unix/linux.
Before you beat yourself silly, just hop onto irc.linuxfromscratch.org.
Someone is likely to have run into the same problem, or something like it and
can give you a hand, within minutes usually.
Proprietary. As much as customization is an advantage, it also becomes a
disadvantage. You have no choice but to customize, or work very hard to
adhere to a set of standards such as the File Heirarchy Standard, and others.
You will never find RPMs for your system or deb files..Occasionally a tgz or
tar.gz file will work as packaged. You pretty much are destined to compile
everything.
Experimenting. You will have to do it. Experimenting is how you get much
software to work, but once you do get it to work, it really works.
Package Management. Once again...No RPM, No Deb, the occasional tgz file
works. There are people who have installed RPM or dpckg and apt-get, and they
build their own packages for their own future use, but unless you follow the
same standards as the people who made the packages, you can't use them. Most
people make them for their own use so they can add and remove less often used
software at will.
Support. If it doesn't work, you don't have a vendor to blame and fix
things for your. Its all on your shoulders.
Configuration. You have to hand code almost every aspect of your linux
system. Although there are some provided rc scripts and /etc configuration
files that can be downloaded from ftp.LFS.org, you will probably want to
change them...In addition any software you install above and beyond the LFS
system will not have premade configuration files. You will have to learn how
to do that yourself.
Man Pages. This is really a PRO but I add it here for a reason. There are
no manuals for LFS, no QUE no UNLEASHED or other overview manuals such as
those you can get for distributed linux. Man Pages will become your friends,
and application specific manuals (such as O'Reilly's) will become the meat
and potatoes of your IT library. In fact you might want to throw out your old
linux manuals, because they ARE vendor specific and won't be much help at all
with LFS, they may even confuse you.
Summary
Linux From Scratch isn't for the faint of heart. It is meant as a
launching platform from which you can build system to do your bidding as you
wish it. Go now, young Jedi. And remember, "Use the source"
Copyright Nov. 2001
Ackowledgements
Gerard Beekman - Owner, Project Leader, Author and Principle Developer and
Principle Tester of "Linux From Scratch Project" and Friend
Beverly Beekman - Mrs. Gerard, gracious supporter of the Linux From Scratch
Project
Jesse Tetanka (HIghOS) - IRC OP, Developer, Tester and Friend. We miss you
Jesse.
Jeremey Jones (mca) & roryo - Testers and Co-Authors of Gnome.txt, the GNOME
hint (much better than my hint) and Friends, sometimes
The list goes on and on, all the way down to minor contributors and just
people we like (and some we don't), then at the very bottom of the list is me.
This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 18:26:53 EDT