Managing your email with fetchmail, procmail and mutt

Let's face it, email can be a pain. Sure it's useful, but between the social network stuff, the marketing emails and other stuff, there's a lot to wade through, and that's without even counting the spam. Graphical email clients may be full of features and relatively easy to use, but after a point it becomes worthwhile considering a more powerful solution.

My favourite email client these days is mutt. I have a small mail server (actually a Pogoplug running Debian, since it's got better ARM support than Ubuntu) that I use to back up all my email from my Gmail account, as well as several others, and it's an extremely efficient way to deal with your email. If necessary, I can view it in a graphical client like Mozilla Thunderbird, but for most of my needs mutt is more than sufficient. Traditionally, the Unix way of doing things is to have many smaller applications that will do one job and do it well, and once you get a setup like this working, it's incredibly powerful and flexible as a result.

So today we're going to cover creating a similar setup on whatever computer you want. If you have a machine that is always on, you could set up an SMTP and IMAP or POP server (I use Postfix and Dovecot), obtain a fully-qualified domain name and run your own mail server, which is a powerful and flexible option. Or you could just get and sort your email automatically, or just back it up offline.

We're going to create a simple setup in this tutorial that will use fetchmail to grab emails from one or more email accounts, then use procmail to filter them into the folders you designate, before using mutt to read them. From here it's fairly trivial to expand upon this - you could set up SpamAssassin to filter out spam emails, for example.

Get the required packages in the usual way:

sudo apt-get install procmail fetchmail mutt

Once these are installed, you can start working on the configuration files. Don't panic! I've found that fetchmail, procmail and mutt are all well-documented, there's plenty of great reference materials online, and once you're done you can easily move your configuration files to a new computer if necessary.

So first of all, create the .forward file in your home directory with the following command:

echo "|/usr/bin/procmail" > .forward

Of course, you could do this with your favourite text editor, but why bother waiting for it to open? This just displays |/usr/bin/procmail to standard output and then redirects it into a new file, thus saving you keystrokes!

Next, you need to create the configuration for fetchmail. Create a new file in your home directory called .fetchmailrc and enter something like this:

set daemon 3600
set logfile /home/user/.fetchmail.log
set no bouncemail
poll pop.gmail.com proto POP3 auth password no dns user "user@gmail.com" pass "password" is user keep ssl

mda "/usr/bin/procmail -d %T"

This example is for a Gmail account and uses the POP3 protocol to download email (I find that for fetchmail, POP is probably a better choice than IMAP in most cases since with IMAP, if you've already read something it won't get downloaded via IMAP). Note the first line - this specifies an interval in seconds between fetchmail checking for new mail. Here I've set it to download once per hour since I use this for backing up my email, but set it to whatever you want. If you want you can then add more email accounts after the end of the settings for that one, but before the line that starts with mda, which tells fetchmail to send all the fetched emails to procmail for delivery, so for instance you could download your work and home email into the same mailbox.

Don't forget to change user to the name of your user account on your machine, and the email address and password for those for the email account(s) you want to use fetchmail with

You also need to set the permissions correctly for your .fetchmailrc. Enter the following command to correct these:

chmod 700 .fetchmailrc

So with fetchmail configured, we'll move on to procmail. Now, procmail is an insanely powerful tool and I can only hope to give you a very brief overview of what it can do, including:

• Automatically sort emails from or to specific email addresses into specified folders


• Break up mailing list digests into individual messages (this is very useful!)


• Automatically respond to emails


• Work as a spam filter (although something like SpamAssassin is considerably more effective

So create a new text file in your home directory called .procmailrc and enter the following:

SHELL=/bin/bash
PATH=/usr/sbin:/usr/bin
MAILDIR=$HOME/Maildir/
DEFAULT=$MAILDIR
LOGFILE=$HOME/.procmail.log
LOG=""
VERBOSE=yes

Once you have the filter working how you want it, you may wish to consider changing VERBOSE to no so the log is not so large.

Now you need to think about what kind of format and folders you want to use. The two mail formats for mail files are mbox and Maildir. While mbox uses a single text file to represent a mailbox, Maildir uses a folder that contains three further folders called cur, new and tmp. Maildir is generally considered superior so we will go with that, however if you want to use mbox instead, all you need to do is drop the trailing / from the mailbox name - the setup above uses a Maildir called Maildir, so if you wanted to use a single mbox file called Mail, you would change that line to MAILDIR=$HOME/Mail for example.

Assuming you want to use Maildir, create a folder in your home directory called Maildir, then go into it and create three directories called cur, new and tmp. Then create three more called .Drafts, .Sent , and .Spam and create the cur, new and tmp directories inside each of them. If you'd like to create more folders for specific purposes, create them in Maildir with the name preceded by a . and create the same cur, new and tmp directories inside each one. For instance, if you want to have a mailbox for emails relating to Linux, you might create it as .Linux inside the Maildir folder.

Now return to your .procmailrc. When configuring procmail, you're creating a series of rules, and if an email matches one of those rules, the action specified in the rule will be taken. As a simple example, here's a rule from my .procmailrc:

# Drop all emails from National Express East Anglia, as there is no good reason to retain these
:0:
*^From:.*(noreply\@nexusalpha\.com)
/dev/null

In this case, these emails are essentially worthless after I have read them, as they're merely to advise if my train to or from work is on time, so I see no reason to keep them around. Rules for procmail are referred to as recipes, and generally consist of three parts.

The first part doesn't mean much - it simply designates the start of the recipe. However, the trailing colon ensures that only one message gets written at the same time - this isn't really necessary for Maildir, but it's a good idea to use it if you're using mbox.

The second part describes a pattern that an email should match. Here, we're looking for emails from the email address noreply@nexusalpha.com. Note that as procmail uses regular expressions, some characters will need to be escaped by preceding them with a \ character, including @ and . characters. Note that you can also specify multiple addresses by separating them with a pipe character, as in this example:

*^From:.*(bob\@company\.com|dave\@company\.com)

This will match any of the email address quoted in the rule. Also note that you can include more than one place to find the string you're searching for at once by placing it in brackets and separating them with a pipe, as in this example:

*^(To|Cc|Bcc):.*(bob\@company\.com|dave\@company\.com)

You can search for a string in From, To, Cc, Bcc, Subject, or practically anywhere else in an email if you want. You can also require that multiple conditions be met, as in this example:

*^From:.*(bob\@company\.com)
*^Subject:.*(joke)

This example would match anything from bob@company.com with joke in the subject.

Finally, the third part shows what should be done with the email once it's been matched against the pattern. In the example above, we sent it to /dev/null, effectively deleting it, which is useful as long as you know you don't want to keep email that matches that pattern. More normally, you're going to want to put it in a specific folder. Here's how you might do that:

:0:
*^Subject:.*(Linux)
$MAILDIR/.Linux/

This would send anything with Linux in the subject line to the folder .Linux. By writing some rules carefully, you can generally rely on procmail delivering them to the right location. When you receive emails, procmail will work down the configuration file until it finds a pattern that matches. If it doesn't the email will go in the default folder specified above.

Npw, a couple of very handy tricks I've learned with procmail. First, a lot of spam emails tend not to be specifically addressed to your email address, so a lot of them can be caught by sending anything not specifically sent to your email address to the spam folder, as in this example:

:0:
* !(To|Cc).*(bob\@company\.com)
$MAILDIR/.Spam/

Here the ! negates the condition so that anything not for bob@company.com gets sent to the Spam folder. Note that this will also catch many mailing list emails, however, since they are also often not sent directly to your email address.

Finally, another favourite of mine is this one, which will break up a mailing list digest into individual messages using the formail tool:

:0:
*^From:.*(mailinglist)
| formail +1 -ds procmail

Note that if you then want this to go into a specific folder, you'll need to set up another rule to catch the individual messages. Fortunately, these are normally sent to the mailing list address, so you should be able to use something like this after the recipe above:

:0:
*^(To|Cc):(mailinglist)
$MAILDIR/mailinglist/

If you want to know more, refer to the man pages for procmail, procmailex, procmailrc and procmailsc. In particular, procmailex contains lots of great examples, and there are loads of procmail recipes around on the web that you may want to borrow snippets from.

Finally, time to configure mutt. Here's a sample mutt configuration for you - save this as .muttrc:

set mbox_type=Maildir

set spoolfile="~/Maildir/"
set folder="~/Maildir/"
set mask="!^\\.[^.]"
set record="+.Sent"
set postponed="+.Drafts"

mailboxes ! + `\
for file in ~/Maildir/.*; do \
box=$(basename "$file"); \
if [ ! "$box" = '.' -a ! "$box" = '..' -a ! "$box" = '.customflags' \
-a ! "$box" = '.subscriptions' ]; then \
echo -n "\"+$box\" "; \
fi; \
done`

macro index c "?" "open a different folder"
macro pager c "?" "open a different folder"

# Move spam to the Spam folder more easily
macro index X "s=.Spam\n" "file as Spam"
macro pager X "s=.Spam\n" "file as Spam"

set sort = 'threads'
set sort_aux = 'reverse-last-date-received'
set auto_tag = yes
ignore "Authentication-Results:"
ignore "DomainKey-Signature:"
ignore "DKIM-Signature:"
hdr_order Date From To Cc
alternative_order text/plain text/html *
auto_view text/html

# Editor
set editor = "vim"

# Colours
color index red black ~N
color index red black ~O

This is a fairly simple muttrc that I use, but you may want to take note of a few things. This shows any unread emails in red, making it easy to see at a glance. By hitting c, you can switch to a different mailbox, and by hitting Shift-X, you can send an email to the Spam folder.

With this in place, you should be ready to go. Enter the following command:

fetchmail -vk

This will launch fetchmail. Note that if you have a lot of email to download, it can take a while. Also note that your .procmailrc may not be perfect and you may find things winding up in the wrong folders, so you may wish to make a backup - there is a good example of how to do this in procmailex.

Have fun getting fetchmail, procmail and mutt working, and feel free to share some of the procmail recipes you find or come up with here if you think they're something others may be interested in! I've found the sheer power of procmail addictive - to quote Douglas Adams, "I am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand.", and after you've gotten stuck into configuring procmail, you may well agree!

Using Screen

The bash shell is great, but it can be awkward to use because a task will continue to run until it's completed. It's possible to push a task into the background, but this isn't the best way to do so. What you really need is a way to have more than one shell operating at once.

Now, on a graphical desktop this is no problem, as you can just open a new terminal, or a new tab in the same terminal. However, this is less practical on a command-line only system, or one you've remoted into via SSH. But there is a solution, and it's called Screen.

Screen is a terminal multiplexer - it allows you to access multiple terminal sessions inside a single terminal window or remote terminal session (including SSH). Think of it as being like having a tabbed terminal, but in the command line. You can leave Screen running and detach from a session, then reconnect to it later, so for instance if you had an Ubuntu Server install you could connect to it via SSH, issue a command to update the system, then detach and attach again later once it had finished.

Screen is almost certainly preinstalled on your system, but if not you can install it via apt-get in the usual way (sudo apt-get install screen). Once you have it installed, you can start it by issuing the following command:

screen

You're now running screen! You should be confronted by a standard copyright notice (you can remove this by editing /etc/screenrc and removing the hash at the start of the line that reads #startup_message off).

All of screen's internal commands require you hit Ctrl-a, followed by the appropriate key for what you want to do. Try hitting Ctrl-a, then c, and you'll create a new virtual console. Then hit Ctrl-a, then n to cycle to the next virtual console, or Ctrl-a then p to go to the previous one. Ctrl-a, then " will get you a list of all currently running virtual consoles that you can select from using the cursor keys, while Ctrl-a then d will detach you from your current screen session (it will continue to run in the background, so you could potentially log out and leave the processes to run while you do something else). When you want to reattach, just start screen with screen -r and it will reattach you to the running processes. To stop screen, just exit each of the running consoles with Ctrl-d or the exit command in the usual way.

Screen is of limited use on a graphical desktop, but you can still use it there to get used to the idea. It's most useful when dealing with command-line systems. For more details, check out the man page for Screen, or try searching on Google. It's a great tool that makes a lot of tasks easier, and is indispensable if you have to log into a server via SSH a lot. It's also quite handy if you use a lot of command-line tools because it means you can have everything in one terminal session - for instance, you could have mutt in one screen, irssi in another, lynx in a third, Vim in a fourth, and be compiling code using GCC in a fifth, all at the same time, and in a way that makes it easy to switch between them.

It's also worth checking out byobu, which is an Ubuntu-specific tool that makes Screen a lot easier to use if you're new to it. Byobu makes it easy to set Screen up according to your preferences, without having to edit the configuration files by hand.

SSH - using the shell remotely

If you came to Ubuntu from a Windows background, or are otherwise familiar with Windows (let's face it, not many of us are in a position where they never use Windows at all!), you'll probably be familiar with Windows Remote Desktop, which is often used as a troubleshooting tool. You can do the same kind of thing in Ubuntu, but because the Linux command line is so much more powerful and flexible than its Windows counterpart, it's often much more practical to use a command-line only tool for the same purpose. This is often a better solution because sending a constantly updated image of your desktop to somewhere else is inevitably going to use a lot more bandwidth than a few shell commands, and it's very likely you can resolve the issue from the shell. It can also be useful for connecting to servers, shell accounts, web hosts etc.

Telnet was the original method of connecting to a remote computer, however it has declined in popularity due to security issues. SSH has largely replaced it because it's more secure, and that's what I'm going to tell you about today.

SSH requires two components to work - a client and a server. Think of it like surfing the web - when you visit a web page, you use a client (the web browser) to connect to a server (the web server). Similarly, with SSH you use an SSH client to connect to an SSH server on the machine you want to connect to. The most common SSH client and server on Linux are maintained by the OpenSSH project. Ubuntu ships with the OpenSSH client by default, and if you have a machine you'd like to be able to connect to remotely via SSH, just install the openssh-server package in the usual way.

How do you use it? It's simple. Open a terminal and enter something using the following format:

ssh user@server

Let's go into this in a little more detail. First of all, user is just the user name you want to log in as on the remote server. For instance, if you've installed openssh-server on your desktop and your user account is called eric on that machine, then you should put eric as the username. If you want to connect to a shell account in the name of ebarnes, then it should be ebarnes.

The server can be either a domain name or an IP address. So if you have a shell account on a machine which has the domain name myfreeshellaccount.com and your user name is ebarnes, you'd enter ssh ebarnes@myfreeshellaccount.com. If you want to connect to a machine that doesn't have a domain name associated with it, then you'd need to enter the IP address to connect to. There's nothing stopping you connecting to another machine on your home network via this method, so if you have an old machine running Ubuntu Server, for instance, on your home network, and it's been assigned the IP address 192.168.1.4 by your router, you can just connect like this:

ssh ebarnes@192.168.1.4

If you'd like to be able to use SSH to connect to your home machine from outside your home network, that's a bit more demanding. You'd need to either obtain a static IP address from your ISP, which can be expensive, or use a service like DynDNS, and would also need to set up port forwarding on your router. However, there's all sorts of possibilites that offers, such as managing BitTorrent downloads remotely by using a command-line client like rTorrent, or administering your machine remotely over SSH.

Once you're connected, SSH works just like any other shell session - you can do exactly the same things with SSH as you would in the terminal. When you're finished, just type in exit or hit Ctrl-D as you would normally to close the shell and that will finish your session.

Note that you aren't limited to connecting to an SSH server from Linux or Unix hosts. Although most Linux or Unix-based operating systems ship with an SSH client, you can use PUTTY to connect from a Windows box (so, for instance, if you have a desktop running Ubuntu with an OpenSSH server installed, and you want to remote into it from your laptop running Windows 7, you can do that). There's even SSH clients for a number of smartphones, including the iPhone and Android, so you can remote into your machine from anywhere.

If you want to try it out, and you have an old desktop you don't use, it's quite interesting to install Ubuntu Server on it, connect it to your router and disconnect the monitor, managing it entirely via SSH. It can be a useful resource to have around, and offers a lot of possibilities, such as:

• File server
• uPnP server (allows you to stream content to many electronic devices, including the Sony PlayStation 3)
• BitTorrent box

By managing it entirely via SSH, you don't need the additional overhead of running a desktop, and can devote more system resources to whatever job you want it to do. You can use SSH to install whatever software you want via apt-get to customise it for the job you want it to do.

Alternatively, if you don't have an old machine spare or don't wish to try this, why not try it in a virtual machine? Virtualbox is good for this - just install Ubuntu Server in a VM, and under the network settings, attach it to a bridged adapter. This will mean the virtual machine gets allocated its own IP address by your router, and you can connect to it via SSH.

If you have a web hosting account, you may be able to connect to this via SSH - the host will provide the details if this is an option. Finally, another option is to get a shell account - essentially it's a user account on a remote server. I highly recommend devio.us, who are a free shell account provider running on OpenBSD.

Have fun with SSH! Next time I'll show you a few tricks to make your experience with it better!

Going further with environment variables

First of all, I apologise for how long it's taken to get round to this (merely days away from being a full year) but I've been busy! I was frantically busy at work for much of the last year, and have my own web development studies to deal with too, so I've only just gotten around to doing this post. Hopefully I should be able to maintain a better frequency of posts in future!

Last time I explained a bit about how environment variables work. Now we'll get to use them a little. For instance, say you want to set up a shortcut to a specific folder called mystuff to make it easy to get to. Well, you can easily set up an environment variable to hold the path to that folder. Just enter something like this:

export mystuff=/home/user/my/stuff

This will set mystuff up as a local variable that you can call at any time in the same way you'd refer to it normally, like this:

cd $mystuff

Note the dollar sign is only used when calling it, not when setting it.

If you want to set variables permanently, just add them to the .profile file in your home directory. Then, when you next start up a shell, it will automatically load these settings.

There's a number of different settings you can add to customise your bash shell. For instance, the other day I added this to my .profile so that Vim was set as my text editor when using Subversion:

export SVN_EDITOR=/usr/bin/vim

I also added the following to my shell account at Devio.us (who, incidentally are hands down the best shell account providers I have ever found, and I recommend) so I could use Vim in colour, even over SSH:

export TERM=xterm-256color

There's a huge range of settings you can change to customise your bash shell, so I won't attempt to cover them all. Instead, I'll refer you to the guide in the Ubuntu documentation. A few I will mention, however, are PAGER (typically /usr/bin/less), EDITOR (the default text editor, you can set it to /usr/bin/vim, /usr/bin/emacs, or /usr/bin/nano, or any other CLI editor you want), VISUAL (the visual editor, normally /usr/bin/gedit, although may be different on Kubuntu or Xubuntu), or BROWSER (pretty much what you'd expect).

Now, a related subject to environment variables is aliases. These are handy shortcuts you can write for commonly used commands, and can be a great timesaver. For instance, if you often connect to a specific host via ssh, you can write an alias for it. To set up an alias, just enter something like the following:

alias example="ssh user@server"

Now, just enter example, on its own, into the shell, and it will run the command. But, of course, this is only temporary. To make it permanent, set up a file in your home directory called .bash_aliases and add the alias you want to set up to it. Once you've done that, every time you start a new shell, all the aliases you've defined in the file will be available to you!

I hope this has given you some idea of how to customise your bash shell to your own liking and improve your workflow in the process!

Introducing Environment Variables

What are environment variables? If you're even vaguely familiar with any programming language you'll have heard the term variable, which is a method of storing some information, such as a letter, number, or string of text, in a convenient manner that makes it easy to manipulate. Just as an example, here's a line of Python code (you don't have to know Python, I've just used it because it's the programming language I know best):

name = raw_input("Please enter your name: ")

This should be fairly clear. The variable here is name, because it holds the response to the question.

Now, the shell is in fact a programming language of sorts, so you can do the exact same thing in the shell. However, the syntax (grammar) is different, as it differs between programming languages. Here's an example which sets up the variable example to represent a string of text:

matthew@trinity:~$ example="This is an example"

This should be easy to follow. Please note, however, that the bash shell, at least, is less forgiving than many programming languages - it won't tolerate spaces between the elements of the command. Also, while you can set up a variable using a number or a single word and not use quotes, if you want to put in a string of more than one items, you MUST use either single or double quotes.
Now you've set this, it's easy to view it using the echo command, which for those of you with programming experience is like print in Python or Basic. Note when printing the variable you precede it with a dollar sign:

matthew@trinity:~$ echo $example
This is an example

Environment variables are divided into two types - global and local. Local variables are limited to the current shell session - in other words they're lost once you exit, and if you have two shell sessions and define a local variable in one of them, it's not available in the other. Global variables are present throughout the system, and are often defined for you by the system. By convention, they are usually expressed in uppercase.
You can view the global environment variables with the printenv command:

matthew@trinity:~$ printenv
KDE_MULTIHEAD=false
SSH_AGENT_PID=5704
DM_CONTROL=/var/run/xdmctl
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=efec6dfd9a5f464a8c8a6760490c6b47-1243367179.617199-1707425974
XDM_MANAGED=/var/run/xdmctl/xdmctl-:0,maysd,mayfn,sched,rsvd,method=classic
GTK2_RC_FILES=/home/matthew/.gtkrc-2.0-kde:/home/matthew/.kde/share/config/gtkrc-2.0
GTK_RC_FILES=/etc/gtk/gtkrc:/home/matthew/.gtkrc:/home/matthew/.kde/share/config/gtkrc
GS_LIB=/home/matthew/.fonts
WINDOWID=31468213
KDE_FULL_SESSION=true
USER=matthew
LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.svgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:
SSH_AUTH_SOCK=/tmp/ssh-YTSIDl5600/agent.5600
SESSION_MANAGER=local/trinity:/tmp/.ICE-unix/5751
DCOP_YAKUAKE_SESSION=4
KONSOLE_DCOP=DCOPRef(yakuake,konsole)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
DESKTOP_SESSION=default
KONSOLE_DCOP_SESSION=DCOPRef(yakuake,session-1)
PWD=/home/matthew
KDE_SESSION_UID=
LANG=en_GB.UTF-8
HISTCONTROL=ignoreboth
HOME=/home/matthew
SHLVL=1
XCURSOR_THEME=DMZ-White
LOGNAME=matthew
LESSOPEN=| /usr/bin/lesspipe %s
DISPLAY=:0
LESSCLOSE=/usr/bin/lesspipe %s %s
COLORTERM=
DCOP_YAKUAKE_TERMINAL=4
_=/usr/bin/printenv

These variables can be viewed just like any other. I'll point out three for you in particular. SHELL represents the default shell you have set:

matthew@trinity:~$ echo $SHELL
/bin/bash

HOME represents your home directory:

matthew@trinity:~$ echo $HOME
/home/matthew

And finally, PATH represents where your system looks by default for executable files. The different folders are show, separated by commas:

matthew@trinity:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

This last one is worth mentioning because it can cause problems. Your PATH variable essentially refers to all the places your system will look for an executable. For instance, Firefox on my system is in /usr/bin/firefox. If I type firefox into the terminal and press Enter, the system will search through all the folders in PATH until it finds a file called firefox in one of them, whereupon it executes it.

Note that the current directory (which would be represented by a period/full stop)does not appear. That's a security feature common to most Unix-like operating systems - by not looking in the current directory by default, it makes it harder to run a recently downloaded executable file, and that along with the permissions system is one of the things that makes Linux more generally secure than Windows. This is why if you want to run an executable file in the current directory, you need to precede its name with ./ to show that it's in the current directory, such as ./configure for example, or provide the full path to it, such as /home/matthew/configure.

Local environment variables don't have a specific command to print them. However, you can use the set command to display all environment variables, both global and local. I haven't repeated this here because there was a LOT of it, but if you want to try it just enter the following:

matthew@trinity:~$ set

Next time, we'll take a further look at environment variables, and go into more details about how you can use them.

INX

If you've been following along for a while, you should have picked up a fair bit of knowledge about how to use the command line. You should be comfortable using a non-graphical text editor such as Vim or nano, installing software using apt-get or dpkg, and viewing text files using cat or less. But I strongly suspect you may not be ready to use the command line exclusively!

However, you may be surprised by just how much can be done from the command line. Did you know you can listen to music from the command line? Or watch videos? Read email? Chat via IRC or IM? Even surf the Web (and not just using a text-only browser such as Lynx, I'm talking about with graphics, although you shouldn't expect the same experience as you get in Firefox).

Furthermore, there's a lot you can get from an interactive teaching method, rather than just following along with my blog posts. For that reason, I highly recommend that if you can spare the bandwidth, you download a copy of INX. It's a live CD, based on Ubuntu, that doesn't include either a desktop or an X window server. In other words, it's command line only!

Now, I'm not dreaming of suggesting you install this in place of your regular Ubuntu or Ubuntu-derived distro install, but it's a great teaching tool. It includes the excellent Ceni network manager, which I've encountered before in Sidux, another distro which like Ubuntu is based on Debian. Ceni is surprisingly intuitive to use and isn't appreciably harder to use than Network Manager in Ubuntu.

INX is also a great showcase for GNU Screen, which is included along with a full tutorial on how to use it. There's also a great introduction which shows just how much the command line can do. On top of that, there's a very easy tutorial that teaches you some of the basics of the bash shell, in an interactive and fun way. All the way through you're given menus to navigate, and you'll never be left at a blank prompt with nothing obvious to do (unless you ask to!).

INX is a great way to learn more about the Linux command line, and I strongly recommend everyone who reads this downloads a copy if they can, and either runs it in Virtualbox or burns it to a CD and boots from it. It's based on Ubuntu Hardy so it should work fine on most hardware. Try it out, and even if you always swore you'd never touch the command line you'll soon lose your fear and will begin to appreciate just why so many Linux and Unix users are ardent fans of the command line. It's an interface that in many ways is more powerful and quicker to use than a point-and-click graphical desktop, and INX will help you get the best out of it. And as it's a live CD you don't have to worry about messing anything up!

Redirection

If you've understood pipes reasonably well, redirection should be easy to understand as well. It's a similar concept in that it involves doing something with the output of a command.

Redirection allows you to send the output of a command into a text file, or send the contents of a file as input to a command. It's used in a similar fashion to pipes, and is represented by the > symbol.

Here's an example:

ls -A > list.txt

Try running this and it will generate a list of all the files and folders in your home directory and save it as list.txt. If you then open the created text file using less or cat, you can see the effect for yourself.

Please note, however, that if you use this with an existing file it will be overwritten. However, by using the double redirection sign(>>) you can append the results to the file, so they are added to whatever is there. Try running a variant of the same command:

ls -A >> list.txt

If you've done it right, there will be two entries for each item in list.txt.

You can also use redirection in the opposite direction to send the contents of a file to a command in order to be written to standard output, as in this example:

matthew@trinity:~/Development/Python/PPAB2ED$ grep game < list.txt
game_over_2.py
game_over2.py
game_over.py

In this case I'd already used ls -A > list.txt to provide a list of the files in a directory I'd used for Python programming. This command allows grep to sort through the file and pick out those lines that have the word "game" in them. OK, there are other ways to do this using pipes, such as cat list.txt | grep game, but this demonstrates how redirection works

You can also combine redirection each way within one command. In this case, I use grep to sort through list.txt for lines featuring the word "game", but I then send the output of that to a new file, game_list.txt:

matthew@trinity:~$ grep game < list.txt > game_list.txt
matthew@trinity:~$ cat game_list.txt
game_over_2.py
game_over2.py
game_over.py

I've had to shorten the path to fit this in, but it's still in the same directory. This demonstrates how you can combine redirection in both ways.

It's quite common to see redirection used in shell scripts (programs written in bash) for various purposes, such as capturing errors, editing text files (I've seen it used to edit your /etc/apt/sources.list in order to add new repositories) and other purposes. It's not of use to the average user as often as pipes are, but it's still worth knowing about.