Introducing Environment Variables

What are environment variables? If you're even vaguely familiar with any programming language you'll have heard the term variable, which is a method of storing some information, such as a letter, number, or string of text, in a convenient manner that makes it easy to manipulate. Just as an example, here's a line of Python code (you don't have to know Python, I've just used it because it's the programming language I know best):

name = raw_input("Please enter your name: ")

This should be fairly clear. The variable here is name, because it holds the response to the question.

Now, the shell is in fact a programming language of sorts, so you can do the exact same thing in the shell. However, the syntax (grammar) is different, as it differs between programming languages. Here's an example which sets up the variable example to represent a string of text:

matthew@trinity:~$ example="This is an example"

This should be easy to follow. Please note, however, that the bash shell, at least, is less forgiving than many programming languages - it won't tolerate spaces between the elements of the command. Also, while you can set up a variable using a number or a single word and not use quotes, if you want to put in a string of more than one items, you MUST use either single or double quotes.
Now you've set this, it's easy to view it using the echo command, which for those of you with programming experience is like print in Python or Basic. Note when printing the variable you precede it with a dollar sign:

matthew@trinity:~$ echo $example
This is an example

Environment variables are divided into two types - global and local. Local variables are limited to the current shell session - in other words they're lost once you exit, and if you have two shell sessions and define a local variable in one of them, it's not available in the other. Global variables are present throughout the system, and are often defined for you by the system. By convention, they are usually expressed in uppercase.
You can view the global environment variables with the printenv command:

matthew@trinity:~$ printenv
KDE_MULTIHEAD=false
SSH_AGENT_PID=5704
DM_CONTROL=/var/run/xdmctl
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=efec6dfd9a5f464a8c8a6760490c6b47-1243367179.617199-1707425974
XDM_MANAGED=/var/run/xdmctl/xdmctl-:0,maysd,mayfn,sched,rsvd,method=classic
GTK2_RC_FILES=/home/matthew/.gtkrc-2.0-kde:/home/matthew/.kde/share/config/gtkrc-2.0
GTK_RC_FILES=/etc/gtk/gtkrc:/home/matthew/.gtkrc:/home/matthew/.kde/share/config/gtkrc
GS_LIB=/home/matthew/.fonts
WINDOWID=31468213
KDE_FULL_SESSION=true
USER=matthew
LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.svgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:
SSH_AUTH_SOCK=/tmp/ssh-YTSIDl5600/agent.5600
SESSION_MANAGER=local/trinity:/tmp/.ICE-unix/5751
DCOP_YAKUAKE_SESSION=4
KONSOLE_DCOP=DCOPRef(yakuake,konsole)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
DESKTOP_SESSION=default
KONSOLE_DCOP_SESSION=DCOPRef(yakuake,session-1)
PWD=/home/matthew
KDE_SESSION_UID=
LANG=en_GB.UTF-8
HISTCONTROL=ignoreboth
HOME=/home/matthew
SHLVL=1
XCURSOR_THEME=DMZ-White
LOGNAME=matthew
LESSOPEN=| /usr/bin/lesspipe %s
DISPLAY=:0
LESSCLOSE=/usr/bin/lesspipe %s %s
COLORTERM=
DCOP_YAKUAKE_TERMINAL=4
_=/usr/bin/printenv

These variables can be viewed just like any other. I'll point out three for you in particular. SHELL represents the default shell you have set:

matthew@trinity:~$ echo $SHELL
/bin/bash

HOME represents your home directory:

matthew@trinity:~$ echo $HOME
/home/matthew

And finally, PATH represents where your system looks by default for executable files. The different folders are show, separated by commas:

matthew@trinity:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

This last one is worth mentioning because it can cause problems. Your PATH variable essentially refers to all the places your system will look for an executable. For instance, Firefox on my system is in /usr/bin/firefox. If I type firefox into the terminal and press Enter, the system will search through all the folders in PATH until it finds a file called firefox in one of them, whereupon it executes it.

Note that the current directory (which would be represented by a period/full stop)does not appear. That's a security feature common to most Unix-like operating systems - by not looking in the current directory by default, it makes it harder to run a recently downloaded executable file, and that along with the permissions system is one of the things that makes Linux more generally secure than Windows. This is why if you want to run an executable file in the current directory, you need to precede its name with ./ to show that it's in the current directory, such as ./configure for example, or provide the full path to it, such as /home/matthew/configure.

Local environment variables don't have a specific command to print them. However, you can use the set command to display all environment variables, both global and local. I haven't repeated this here because there was a LOT of it, but if you want to try it just enter the following:

matthew@trinity:~$ set

Next time, we'll take a further look at environment variables, and go into more details about how you can use them.

INX

If you've been following along for a while, you should have picked up a fair bit of knowledge about how to use the command line. You should be comfortable using a non-graphical text editor such as Vim or nano, installing software using apt-get or dpkg, and viewing text files using cat or less. But I strongly suspect you may not be ready to use the command line exclusively!

However, you may be surprised by just how much can be done from the command line. Did you know you can listen to music from the command line? Or watch videos? Read email? Chat via IRC or IM? Even surf the Web (and not just using a text-only browser such as Lynx, I'm talking about with graphics, although you shouldn't expect the same experience as you get in Firefox).

Furthermore, there's a lot you can get from an interactive teaching method, rather than just following along with my blog posts. For that reason, I highly recommend that if you can spare the bandwidth, you download a copy of INX. It's a live CD, based on Ubuntu, that doesn't include either a desktop or an X window server. In other words, it's command line only!

Now, I'm not dreaming of suggesting you install this in place of your regular Ubuntu or Ubuntu-derived distro install, but it's a great teaching tool. It includes the excellent Ceni network manager, which I've encountered before in Sidux, another distro which like Ubuntu is based on Debian. Ceni is surprisingly intuitive to use and isn't appreciably harder to use than Network Manager in Ubuntu.

INX is also a great showcase for GNU Screen, which is included along with a full tutorial on how to use it. There's also a great introduction which shows just how much the command line can do. On top of that, there's a very easy tutorial that teaches you some of the basics of the bash shell, in an interactive and fun way. All the way through you're given menus to navigate, and you'll never be left at a blank prompt with nothing obvious to do (unless you ask to!).

INX is a great way to learn more about the Linux command line, and I strongly recommend everyone who reads this downloads a copy if they can, and either runs it in Virtualbox or burns it to a CD and boots from it. It's based on Ubuntu Hardy so it should work fine on most hardware. Try it out, and even if you always swore you'd never touch the command line you'll soon lose your fear and will begin to appreciate just why so many Linux and Unix users are ardent fans of the command line. It's an interface that in many ways is more powerful and quicker to use than a point-and-click graphical desktop, and INX will help you get the best out of it. And as it's a live CD you don't have to worry about messing anything up!

Redirection

If you've understood pipes reasonably well, redirection should be easy to understand as well. It's a similar concept in that it involves doing something with the output of a command.

Redirection allows you to send the output of a command into a text file, or send the contents of a file as input to a command. It's used in a similar fashion to pipes, and is represented by the > symbol.

Here's an example:

ls -A > list.txt

Try running this and it will generate a list of all the files and folders in your home directory and save it as list.txt. If you then open the created text file using less or cat, you can see the effect for yourself.

Please note, however, that if you use this with an existing file it will be overwritten. However, by using the double redirection sign(>>) you can append the results to the file, so they are added to whatever is there. Try running a variant of the same command:

ls -A >> list.txt

If you've done it right, there will be two entries for each item in list.txt.

You can also use redirection in the opposite direction to send the contents of a file to a command in order to be written to standard output, as in this example:

matthew@trinity:~/Development/Python/PPAB2ED$ grep game < list.txt
game_over_2.py
game_over2.py
game_over.py

In this case I'd already used ls -A > list.txt to provide a list of the files in a directory I'd used for Python programming. This command allows grep to sort through the file and pick out those lines that have the word "game" in them. OK, there are other ways to do this using pipes, such as cat list.txt | grep game, but this demonstrates how redirection works

You can also combine redirection each way within one command. In this case, I use grep to sort through list.txt for lines featuring the word "game", but I then send the output of that to a new file, game_list.txt:

matthew@trinity:~$ grep game < list.txt > game_list.txt
matthew@trinity:~$ cat game_list.txt
game_over_2.py
game_over2.py
game_over.py

I've had to shorten the path to fit this in, but it's still in the same directory. This demonstrates how you can combine redirection in both ways.

It's quite common to see redirection used in shell scripts (programs written in bash) for various purposes, such as capturing errors, editing text files (I've seen it used to edit your /etc/apt/sources.list in order to add new repositories) and other purposes. It's not of use to the average user as often as pipes are, but it's still worth knowing about.

Pipes

One of the original philosophies behind Unix is that each program should do one thing only, and do it well, and by and large most command-line applications in Ubuntu continue that tradition. By sticking to just that one task, they remain efficient and powerful.

But sometimes you may need more complex functionality, and this is where piping comes into its own. Piping allows you to combine commands in a simple but effective way that, if well used, makes them more powerful than the sum of their parts.

The concept is simple. Say it's a hot day and you want to water the plants in your garden. You have a tap on the outside wall of your house, but that's nowhere near the plants. You need to take the output of the tap (water) and use it as input to the plants.

How do you do it? You use a hosepipe, and that's exactly what a pipe is in Linux and Unix. It takes the output of one command and uses it as input for another.

The pipe character looks like this:

|

Where you find it differs on different keyboards, and some really do their best to hide it away! It's often shown as broken in the middle. On my Kubuntu machine it's on the same key as the backslash (\) character. Make a mental note of where it is, as you'll find it very handy!

Here's a simple example. On my Kubuntu laptop, I want to run ls -A, which shows all the files and folders in a directory, including hidden ones, within my /home directory. However, there's quite a lot here so if I display it all, it'll just run off the screen. But if I pipe it into the less command, I can easily scroll up and down through them. Here's how you do it:

ls -A | less

Try it out! This is a common way of making directories like this more readable. Note that less didn't need a file name specified, instead it accepted the output of ls -A as input. Most commands will accept this fine.

Remember I said that grep was more effective when combined with pipes? Well, here's an example where I'm again running ls -A in my /home directory. However, here I want to find files or folders with bash in their names, so I use grep to filter the output for responses that include bash:

matthew@trinity:~$ ls -A | grep bash
.bash_history
.bash_logout
.bashrc

You can use more than one pipe if you wish. To the best of my knowledge, there's no upper limit! Here's an example I used the other day when trying out OpenBSD, another open-source Unix-based operating system similar to Linux. I wasn't sure what device the CD drive was recorded as, so I ran the following:

ls /dev | grep -i cd | less

The first part listed all the files or folders in the /dev directory (in other words, devices on the computer). I then piped the output into grep, which picked out the files or folders which had cd in their names (note the use of -i to make grep case-insensitive). Finally, I piped the results of that into less so I could easily move up and down through the responses. Although this wasn't in Linux, it would work exactly the same in Ubuntu as in OpenBSD.

Pipes are extremely powerful. They let you "glue" commands together to make them more useful. You should already be able to see just how useful pipes can be if used well. Next time we'll deal with a related concept, redirection.

Using grep to find text

The grep command is one of the most useful in Linux. It may not seem that exciting at first, but once you learn about pipes (next time!) you'll begin to see why it's so indispensable.

Put simply, grep searches through one or more text files for a specific word or phrase. For any lines it finds which include the specified word or phrase, it will display them on the screen. It will also work with standard output (such as the output of the cat command, for example), when combined with pipes.

Let's take a simple example: Your name is Eric and you know that somewhere in your /home directory is a report you've written for work. Let's say you know that it has the word competition in it. So you can instruct grep to search through all the files in your home directory like this:

grep -r competition /home/eric/*

Note the use of the * wild card character, meaning it will search through every file in /home/eric. Also, I've used the -r option here, short for recursive (similar to with rm), so it will also search every folder within /home/eric.

However, grep is case sensitive, so if you wanted to make it include results in both upper and lower case, you would use the -i option, like this:

grep -ir competition /home/eric/*

If you're entering a phrase with more than one word, you'll need to put it in quotes, like this:

grep "Hello there" hello.txt

grep is most commonly used just with one file, as in this example, where I'm using to search for the word universe in /etc/apt/sources.list:

matthew@trinity:~$ grep universe /etc/apt/sources.list
## universe WILL NOT receive any review or updates from the Ubuntu security
deb http://gb.archive.ubuntu.com/ubuntu/ hardy universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy universe
deb http://gb.archive.ubuntu.com/ubuntu/ hardy-updates universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-updates universe
# deb http://gb.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
# deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu hardy-security universe
deb-src http://security.ubuntu.com/ubuntu hardy-security universe
matthew@trinity:~$

While grep is a simple utility, and is therefore fairly easy to explain, it's incredibly useful once you get the hang of it. If you want to know more about the various options available, you may want to examine the man page for grep:

man grep

As stated above, its real power comes when you combine it with pipes, which I'll be demonstrating next time.