Using Screen

The bash shell is great, but it can be awkward to use because a task will continue to run until it's completed. It's possible to push a task into the background, but this isn't the best way to do so. What you really need is a way to have more than one shell operating at once.

Now, on a graphical desktop this is no problem, as you can just open a new terminal, or a new tab in the same terminal. However, this is less practical on a command-line only system, or one you've remoted into via SSH. But there is a solution, and it's called Screen.

Screen is a terminal multiplexer - it allows you to access multiple terminal sessions inside a single terminal window or remote terminal session (including SSH). Think of it as being like having a tabbed terminal, but in the command line. You can leave Screen running and detach from a session, then reconnect to it later, so for instance if you had an Ubuntu Server install you could connect to it via SSH, issue a command to update the system, then detach and attach again later once it had finished.

Screen is almost certainly preinstalled on your system, but if not you can install it via apt-get in the usual way (sudo apt-get install screen). Once you have it installed, you can start it by issuing the following command:

screen

You're now running screen! You should be confronted by a standard copyright notice (you can remove this by editing /etc/screenrc and removing the hash at the start of the line that reads #startup_message off).

All of screen's internal commands require you hit Ctrl-a, followed by the appropriate key for what you want to do. Try hitting Ctrl-a, then c, and you'll create a new virtual console. Then hit Ctrl-a, then n to cycle to the next virtual console, or Ctrl-a then p to go to the previous one. Ctrl-a, then " will get you a list of all currently running virtual consoles that you can select from using the cursor keys, while Ctrl-a then d will detach you from your current screen session (it will continue to run in the background, so you could potentially log out and leave the processes to run while you do something else). When you want to reattach, just start screen with screen -r and it will reattach you to the running processes. To stop screen, just exit each of the running consoles with Ctrl-d or the exit command in the usual way.

Screen is of limited use on a graphical desktop, but you can still use it there to get used to the idea. It's most useful when dealing with command-line systems. For more details, check out the man page for Screen, or try searching on Google. It's a great tool that makes a lot of tasks easier, and is indispensable if you have to log into a server via SSH a lot. It's also quite handy if you use a lot of command-line tools because it means you can have everything in one terminal session - for instance, you could have mutt in one screen, irssi in another, lynx in a third, Vim in a fourth, and be compiling code using GCC in a fifth, all at the same time, and in a way that makes it easy to switch between them.

It's also worth checking out byobu, which is an Ubuntu-specific tool that makes Screen a lot easier to use if you're new to it. Byobu makes it easy to set Screen up according to your preferences, without having to edit the configuration files by hand.

SSH - using the shell remotely

If you came to Ubuntu from a Windows background, or are otherwise familiar with Windows (let's face it, not many of us are in a position where they never use Windows at all!), you'll probably be familiar with Windows Remote Desktop, which is often used as a troubleshooting tool. You can do the same kind of thing in Ubuntu, but because the Linux command line is so much more powerful and flexible than its Windows counterpart, it's often much more practical to use a command-line only tool for the same purpose. This is often a better solution because sending a constantly updated image of your desktop to somewhere else is inevitably going to use a lot more bandwidth than a few shell commands, and it's very likely you can resolve the issue from the shell. It can also be useful for connecting to servers, shell accounts, web hosts etc.

Telnet was the original method of connecting to a remote computer, however it has declined in popularity due to security issues. SSH has largely replaced it because it's more secure, and that's what I'm going to tell you about today.

SSH requires two components to work - a client and a server. Think of it like surfing the web - when you visit a web page, you use a client (the web browser) to connect to a server (the web server). Similarly, with SSH you use an SSH client to connect to an SSH server on the machine you want to connect to. The most common SSH client and server on Linux are maintained by the OpenSSH project. Ubuntu ships with the OpenSSH client by default, and if you have a machine you'd like to be able to connect to remotely via SSH, just install the openssh-server package in the usual way.

How do you use it? It's simple. Open a terminal and enter something using the following format:

ssh user@server

Let's go into this in a little more detail. First of all, user is just the user name you want to log in as on the remote server. For instance, if you've installed openssh-server on your desktop and your user account is called eric on that machine, then you should put eric as the username. If you want to connect to a shell account in the name of ebarnes, then it should be ebarnes.

The server can be either a domain name or an IP address. So if you have a shell account on a machine which has the domain name myfreeshellaccount.com and your user name is ebarnes, you'd enter ssh ebarnes@myfreeshellaccount.com. If you want to connect to a machine that doesn't have a domain name associated with it, then you'd need to enter the IP address to connect to. There's nothing stopping you connecting to another machine on your home network via this method, so if you have an old machine running Ubuntu Server, for instance, on your home network, and it's been assigned the IP address 192.168.1.4 by your router, you can just connect like this:

ssh ebarnes@192.168.1.4

If you'd like to be able to use SSH to connect to your home machine from outside your home network, that's a bit more demanding. You'd need to either obtain a static IP address from your ISP, which can be expensive, or use a service like DynDNS, and would also need to set up port forwarding on your router. However, there's all sorts of possibilites that offers, such as managing BitTorrent downloads remotely by using a command-line client like rTorrent, or administering your machine remotely over SSH.

Once you're connected, SSH works just like any other shell session - you can do exactly the same things with SSH as you would in the terminal. When you're finished, just type in exit or hit Ctrl-D as you would normally to close the shell and that will finish your session.

Note that you aren't limited to connecting to an SSH server from Linux or Unix hosts. Although most Linux or Unix-based operating systems ship with an SSH client, you can use PUTTY to connect from a Windows box (so, for instance, if you have a desktop running Ubuntu with an OpenSSH server installed, and you want to remote into it from your laptop running Windows 7, you can do that). There's even SSH clients for a number of smartphones, including the iPhone and Android, so you can remote into your machine from anywhere.

If you want to try it out, and you have an old desktop you don't use, it's quite interesting to install Ubuntu Server on it, connect it to your router and disconnect the monitor, managing it entirely via SSH. It can be a useful resource to have around, and offers a lot of possibilities, such as:

• File server
• uPnP server (allows you to stream content to many electronic devices, including the Sony PlayStation 3)
• BitTorrent box

By managing it entirely via SSH, you don't need the additional overhead of running a desktop, and can devote more system resources to whatever job you want it to do. You can use SSH to install whatever software you want via apt-get to customise it for the job you want it to do.

Alternatively, if you don't have an old machine spare or don't wish to try this, why not try it in a virtual machine? Virtualbox is good for this - just install Ubuntu Server in a VM, and under the network settings, attach it to a bridged adapter. This will mean the virtual machine gets allocated its own IP address by your router, and you can connect to it via SSH.

If you have a web hosting account, you may be able to connect to this via SSH - the host will provide the details if this is an option. Finally, another option is to get a shell account - essentially it's a user account on a remote server. I highly recommend devio.us, who are a free shell account provider running on OpenBSD.

Have fun with SSH! Next time I'll show you a few tricks to make your experience with it better!

Going further with environment variables

First of all, I apologise for how long it's taken to get round to this (merely days away from being a full year) but I've been busy! I was frantically busy at work for much of the last year, and have my own web development studies to deal with too, so I've only just gotten around to doing this post. Hopefully I should be able to maintain a better frequency of posts in future!

Last time I explained a bit about how environment variables work. Now we'll get to use them a little. For instance, say you want to set up a shortcut to a specific folder called mystuff to make it easy to get to. Well, you can easily set up an environment variable to hold the path to that folder. Just enter something like this:

export mystuff=/home/user/my/stuff

This will set mystuff up as a local variable that you can call at any time in the same way you'd refer to it normally, like this:

cd $mystuff

Note the dollar sign is only used when calling it, not when setting it.

If you want to set variables permanently, just add them to the .profile file in your home directory. Then, when you next start up a shell, it will automatically load these settings.

There's a number of different settings you can add to customise your bash shell. For instance, the other day I added this to my .profile so that Vim was set as my text editor when using Subversion:

export SVN_EDITOR=/usr/bin/vim

I also added the following to my shell account at Devio.us (who, incidentally are hands down the best shell account providers I have ever found, and I recommend) so I could use Vim in colour, even over SSH:

export TERM=xterm-256color

There's a huge range of settings you can change to customise your bash shell, so I won't attempt to cover them all. Instead, I'll refer you to the guide in the Ubuntu documentation. A few I will mention, however, are PAGER (typically /usr/bin/less), EDITOR (the default text editor, you can set it to /usr/bin/vim, /usr/bin/emacs, or /usr/bin/nano, or any other CLI editor you want), VISUAL (the visual editor, normally /usr/bin/gedit, although may be different on Kubuntu or Xubuntu), or BROWSER (pretty much what you'd expect).

Now, a related subject to environment variables is aliases. These are handy shortcuts you can write for commonly used commands, and can be a great timesaver. For instance, if you often connect to a specific host via ssh, you can write an alias for it. To set up an alias, just enter something like the following:

alias example="ssh user@server"

Now, just enter example, on its own, into the shell, and it will run the command. But, of course, this is only temporary. To make it permanent, set up a file in your home directory called .bash_aliases and add the alias you want to set up to it. Once you've done that, every time you start a new shell, all the aliases you've defined in the file will be available to you!

I hope this has given you some idea of how to customise your bash shell to your own liking and improve your workflow in the process!

Introducing Environment Variables

What are environment variables? If you're even vaguely familiar with any programming language you'll have heard the term variable, which is a method of storing some information, such as a letter, number, or string of text, in a convenient manner that makes it easy to manipulate. Just as an example, here's a line of Python code (you don't have to know Python, I've just used it because it's the programming language I know best):

name = raw_input("Please enter your name: ")

This should be fairly clear. The variable here is name, because it holds the response to the question.

Now, the shell is in fact a programming language of sorts, so you can do the exact same thing in the shell. However, the syntax (grammar) is different, as it differs between programming languages. Here's an example which sets up the variable example to represent a string of text:

matthew@trinity:~$ example="This is an example"

This should be easy to follow. Please note, however, that the bash shell, at least, is less forgiving than many programming languages - it won't tolerate spaces between the elements of the command. Also, while you can set up a variable using a number or a single word and not use quotes, if you want to put in a string of more than one items, you MUST use either single or double quotes.
Now you've set this, it's easy to view it using the echo command, which for those of you with programming experience is like print in Python or Basic. Note when printing the variable you precede it with a dollar sign:

matthew@trinity:~$ echo $example
This is an example

Environment variables are divided into two types - global and local. Local variables are limited to the current shell session - in other words they're lost once you exit, and if you have two shell sessions and define a local variable in one of them, it's not available in the other. Global variables are present throughout the system, and are often defined for you by the system. By convention, they are usually expressed in uppercase.
You can view the global environment variables with the printenv command:

matthew@trinity:~$ printenv
KDE_MULTIHEAD=false
SSH_AGENT_PID=5704
DM_CONTROL=/var/run/xdmctl
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=efec6dfd9a5f464a8c8a6760490c6b47-1243367179.617199-1707425974
XDM_MANAGED=/var/run/xdmctl/xdmctl-:0,maysd,mayfn,sched,rsvd,method=classic
GTK2_RC_FILES=/home/matthew/.gtkrc-2.0-kde:/home/matthew/.kde/share/config/gtkrc-2.0
GTK_RC_FILES=/etc/gtk/gtkrc:/home/matthew/.gtkrc:/home/matthew/.kde/share/config/gtkrc
GS_LIB=/home/matthew/.fonts
WINDOWID=31468213
KDE_FULL_SESSION=true
USER=matthew
LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.svgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:
SSH_AUTH_SOCK=/tmp/ssh-YTSIDl5600/agent.5600
SESSION_MANAGER=local/trinity:/tmp/.ICE-unix/5751
DCOP_YAKUAKE_SESSION=4
KONSOLE_DCOP=DCOPRef(yakuake,konsole)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
DESKTOP_SESSION=default
KONSOLE_DCOP_SESSION=DCOPRef(yakuake,session-1)
PWD=/home/matthew
KDE_SESSION_UID=
LANG=en_GB.UTF-8
HISTCONTROL=ignoreboth
HOME=/home/matthew
SHLVL=1
XCURSOR_THEME=DMZ-White
LOGNAME=matthew
LESSOPEN=| /usr/bin/lesspipe %s
DISPLAY=:0
LESSCLOSE=/usr/bin/lesspipe %s %s
COLORTERM=
DCOP_YAKUAKE_TERMINAL=4
_=/usr/bin/printenv

These variables can be viewed just like any other. I'll point out three for you in particular. SHELL represents the default shell you have set:

matthew@trinity:~$ echo $SHELL
/bin/bash

HOME represents your home directory:

matthew@trinity:~$ echo $HOME
/home/matthew

And finally, PATH represents where your system looks by default for executable files. The different folders are show, separated by commas:

matthew@trinity:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

This last one is worth mentioning because it can cause problems. Your PATH variable essentially refers to all the places your system will look for an executable. For instance, Firefox on my system is in /usr/bin/firefox. If I type firefox into the terminal and press Enter, the system will search through all the folders in PATH until it finds a file called firefox in one of them, whereupon it executes it.

Note that the current directory (which would be represented by a period/full stop)does not appear. That's a security feature common to most Unix-like operating systems - by not looking in the current directory by default, it makes it harder to run a recently downloaded executable file, and that along with the permissions system is one of the things that makes Linux more generally secure than Windows. This is why if you want to run an executable file in the current directory, you need to precede its name with ./ to show that it's in the current directory, such as ./configure for example, or provide the full path to it, such as /home/matthew/configure.

Local environment variables don't have a specific command to print them. However, you can use the set command to display all environment variables, both global and local. I haven't repeated this here because there was a LOT of it, but if you want to try it just enter the following:

matthew@trinity:~$ set

Next time, we'll take a further look at environment variables, and go into more details about how you can use them.

INX

If you've been following along for a while, you should have picked up a fair bit of knowledge about how to use the command line. You should be comfortable using a non-graphical text editor such as Vim or nano, installing software using apt-get or dpkg, and viewing text files using cat or less. But I strongly suspect you may not be ready to use the command line exclusively!

However, you may be surprised by just how much can be done from the command line. Did you know you can listen to music from the command line? Or watch videos? Read email? Chat via IRC or IM? Even surf the Web (and not just using a text-only browser such as Lynx, I'm talking about with graphics, although you shouldn't expect the same experience as you get in Firefox).

Furthermore, there's a lot you can get from an interactive teaching method, rather than just following along with my blog posts. For that reason, I highly recommend that if you can spare the bandwidth, you download a copy of INX. It's a live CD, based on Ubuntu, that doesn't include either a desktop or an X window server. In other words, it's command line only!

Now, I'm not dreaming of suggesting you install this in place of your regular Ubuntu or Ubuntu-derived distro install, but it's a great teaching tool. It includes the excellent Ceni network manager, which I've encountered before in Sidux, another distro which like Ubuntu is based on Debian. Ceni is surprisingly intuitive to use and isn't appreciably harder to use than Network Manager in Ubuntu.

INX is also a great showcase for GNU Screen, which is included along with a full tutorial on how to use it. There's also a great introduction which shows just how much the command line can do. On top of that, there's a very easy tutorial that teaches you some of the basics of the bash shell, in an interactive and fun way. All the way through you're given menus to navigate, and you'll never be left at a blank prompt with nothing obvious to do (unless you ask to!).

INX is a great way to learn more about the Linux command line, and I strongly recommend everyone who reads this downloads a copy if they can, and either runs it in Virtualbox or burns it to a CD and boots from it. It's based on Ubuntu Hardy so it should work fine on most hardware. Try it out, and even if you always swore you'd never touch the command line you'll soon lose your fear and will begin to appreciate just why so many Linux and Unix users are ardent fans of the command line. It's an interface that in many ways is more powerful and quicker to use than a point-and-click graphical desktop, and INX will help you get the best out of it. And as it's a live CD you don't have to worry about messing anything up!

Redirection

If you've understood pipes reasonably well, redirection should be easy to understand as well. It's a similar concept in that it involves doing something with the output of a command.

Redirection allows you to send the output of a command into a text file, or send the contents of a file as input to a command. It's used in a similar fashion to pipes, and is represented by the > symbol.

Here's an example:

ls -A > list.txt

Try running this and it will generate a list of all the files and folders in your home directory and save it as list.txt. If you then open the created text file using less or cat, you can see the effect for yourself.

Please note, however, that if you use this with an existing file it will be overwritten. However, by using the double redirection sign(>>) you can append the results to the file, so they are added to whatever is there. Try running a variant of the same command:

ls -A >> list.txt

If you've done it right, there will be two entries for each item in list.txt.

You can also use redirection in the opposite direction to send the contents of a file to a command in order to be written to standard output, as in this example:

matthew@trinity:~/Development/Python/PPAB2ED$ grep game < list.txt
game_over_2.py
game_over2.py
game_over.py

In this case I'd already used ls -A > list.txt to provide a list of the files in a directory I'd used for Python programming. This command allows grep to sort through the file and pick out those lines that have the word "game" in them. OK, there are other ways to do this using pipes, such as cat list.txt | grep game, but this demonstrates how redirection works

You can also combine redirection each way within one command. In this case, I use grep to sort through list.txt for lines featuring the word "game", but I then send the output of that to a new file, game_list.txt:

matthew@trinity:~$ grep game < list.txt > game_list.txt
matthew@trinity:~$ cat game_list.txt
game_over_2.py
game_over2.py
game_over.py

I've had to shorten the path to fit this in, but it's still in the same directory. This demonstrates how you can combine redirection in both ways.

It's quite common to see redirection used in shell scripts (programs written in bash) for various purposes, such as capturing errors, editing text files (I've seen it used to edit your /etc/apt/sources.list in order to add new repositories) and other purposes. It's not of use to the average user as often as pipes are, but it's still worth knowing about.

Pipes

One of the original philosophies behind Unix is that each program should do one thing only, and do it well, and by and large most command-line applications in Ubuntu continue that tradition. By sticking to just that one task, they remain efficient and powerful.

But sometimes you may need more complex functionality, and this is where piping comes into its own. Piping allows you to combine commands in a simple but effective way that, if well used, makes them more powerful than the sum of their parts.

The concept is simple. Say it's a hot day and you want to water the plants in your garden. You have a tap on the outside wall of your house, but that's nowhere near the plants. You need to take the output of the tap (water) and use it as input to the plants.

How do you do it? You use a hosepipe, and that's exactly what a pipe is in Linux and Unix. It takes the output of one command and uses it as input for another.

The pipe character looks like this:

|

Where you find it differs on different keyboards, and some really do their best to hide it away! It's often shown as broken in the middle. On my Kubuntu machine it's on the same key as the backslash (\) character. Make a mental note of where it is, as you'll find it very handy!

Here's a simple example. On my Kubuntu laptop, I want to run ls -A, which shows all the files and folders in a directory, including hidden ones, within my /home directory. However, there's quite a lot here so if I display it all, it'll just run off the screen. But if I pipe it into the less command, I can easily scroll up and down through them. Here's how you do it:

ls -A | less

Try it out! This is a common way of making directories like this more readable. Note that less didn't need a file name specified, instead it accepted the output of ls -A as input. Most commands will accept this fine.

Remember I said that grep was more effective when combined with pipes? Well, here's an example where I'm again running ls -A in my /home directory. However, here I want to find files or folders with bash in their names, so I use grep to filter the output for responses that include bash:

matthew@trinity:~$ ls -A | grep bash
.bash_history
.bash_logout
.bashrc

You can use more than one pipe if you wish. To the best of my knowledge, there's no upper limit! Here's an example I used the other day when trying out OpenBSD, another open-source Unix-based operating system similar to Linux. I wasn't sure what device the CD drive was recorded as, so I ran the following:

ls /dev | grep -i cd | less

The first part listed all the files or folders in the /dev directory (in other words, devices on the computer). I then piped the output into grep, which picked out the files or folders which had cd in their names (note the use of -i to make grep case-insensitive). Finally, I piped the results of that into less so I could easily move up and down through the responses. Although this wasn't in Linux, it would work exactly the same in Ubuntu as in OpenBSD.

Pipes are extremely powerful. They let you "glue" commands together to make them more useful. You should already be able to see just how useful pipes can be if used well. Next time we'll deal with a related concept, redirection.

Using grep to find text

The grep command is one of the most useful in Linux. It may not seem that exciting at first, but once you learn about pipes (next time!) you'll begin to see why it's so indispensable.

Put simply, grep searches through one or more text files for a specific word or phrase. For any lines it finds which include the specified word or phrase, it will display them on the screen. It will also work with standard output (such as the output of the cat command, for example), when combined with pipes.

Let's take a simple example: Your name is Eric and you know that somewhere in your /home directory is a report you've written for work. Let's say you know that it has the word competition in it. So you can instruct grep to search through all the files in your home directory like this:

grep -r competition /home/eric/*

Note the use of the * wild card character, meaning it will search through every file in /home/eric. Also, I've used the -r option here, short for recursive (similar to with rm), so it will also search every folder within /home/eric.

However, grep is case sensitive, so if you wanted to make it include results in both upper and lower case, you would use the -i option, like this:

grep -ir competition /home/eric/*

If you're entering a phrase with more than one word, you'll need to put it in quotes, like this:

grep "Hello there" hello.txt

grep is most commonly used just with one file, as in this example, where I'm using to search for the word universe in /etc/apt/sources.list:

matthew@trinity:~$ grep universe /etc/apt/sources.list
## universe WILL NOT receive any review or updates from the Ubuntu security
deb http://gb.archive.ubuntu.com/ubuntu/ hardy universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy universe
deb http://gb.archive.ubuntu.com/ubuntu/ hardy-updates universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-updates universe
# deb http://gb.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
# deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu hardy-security universe
deb-src http://security.ubuntu.com/ubuntu hardy-security universe
matthew@trinity:~$

While grep is a simple utility, and is therefore fairly easy to explain, it's incredibly useful once you get the hang of it. If you want to know more about the various options available, you may want to examine the man page for grep:

man grep

As stated above, its real power comes when you combine it with pipes, which I'll be demonstrating next time.

Viewing text files

One of the central philosophies behind Unix, which Linux has inherited from it, is that configuration files should be plain text. This has the advantage that you can easily edit these files, even if for some reason you can't start the X window server. So an unbootable system can be repaired (hopefully!) by editing a few text files. It also means you can make changes to your system remotely with ease, by logging on via telnet or ssh (don't worry if you don't know what these are!)
For this reason, Linux has a lot of incredibly powerful tools to enable you to manipulate text files. I'm going to go through several that you can use to view text files from the command line.
The most basic of these is cat. This just prints the file as standard output - in other words, it just dumps it into your terminal, as in this example:

matthew@morpheus:~$ cat /etc/apt/sources.list
#deb cdrom:[Ubuntu 8.04 _Hardy Heron_ - Release i386 (20080423)]/ hardy main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://gb.archive.ubuntu.com/ubuntu/ hardy main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://gb.archive.ubuntu.com/ubuntu/ hardy-updates main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## universe WILL NOT receive any review or updates from the Ubuntu security
## team.
deb http://gb.archive.ubuntu.com/ubuntu/ hardy universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy universe
deb http://gb.archive.ubuntu.com/ubuntu/ hardy-updates universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://gb.archive.ubuntu.com/ubuntu/ hardy multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy multiverse
deb http://gb.archive.ubuntu.com/ubuntu/ hardy-updates multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-updates multiverse

## Uncomment the following two lines to add software from the 'backports'
## repository.
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://gb.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
# deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository. This software is not part of Ubuntu, but is
## offered by Canonical and the respective vendors as a service to Ubuntu
## users.
# deb http://archive.canonical.com/ubuntu hardy partner
# deb-src http://archive.canonical.com/ubuntu hardy partner

deb http://security.ubuntu.com/ubuntu hardy-security main restricted
deb-src http://security.ubuntu.com/ubuntu hardy-security main restricted
deb http://security.ubuntu.com/ubuntu hardy-security universe
deb-src http://security.ubuntu.com/ubuntu hardy-security universe
deb http://security.ubuntu.com/ubuntu hardy-security multiverse
deb-src http://security.ubuntu.com/ubuntu hardy-security multiverse

cat is fine for small text files, but for long ones like this example, the text just scrolls off the screen. So for longer files, it's better to use an alternative.
To give you more control when viewing text files, someone came up with the more command, which allows you to scroll through pages of text (for this reason it's known as a pager). The more command has now been largely replaced by the less command, which is more advanced, but does essentially the same thing. However, both are available in Ubuntu.
Try using less to get a feel for it:

less /etc/apt/sources.list

You can scroll up and down using the cursor keys. Alternatively, if you're used to the Vim text editor, you can use j and k to scroll up and down, same as you would in Vim. To exit, press Q.
Finally, two commands that you're unlikely to need unless you're running Ubuntu as a server are head and tail. As the name suggests, these allow you to view the beginning (head) or the end (tail) of a text file, as in this example for head:

matthew@morpheus:~$ head /etc/apt/sources.list
#deb cdrom:[Ubuntu 8.04 _Hardy Heron_ - Release i386 (20080423)]/ hardy main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://gb.archive.ubuntu.com/ubuntu/ hardy main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ hardy main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://gb.archive.ubuntu.com/ubuntu/ hardy-updates main restricted

And this example for tail:

matthew@morpheus:~$ tail /etc/apt/sources.list
## users.
# deb http://archive.canonical.com/ubuntu hardy partner
# deb-src http://archive.canonical.com/ubuntu hardy partner

deb http://security.ubuntu.com/ubuntu hardy-security main restricted
deb-src http://security.ubuntu.com/ubuntu hardy-security main restricted
deb http://security.ubuntu.com/ubuntu hardy-security universe
deb-src http://security.ubuntu.com/ubuntu hardy-security universe
deb http://security.ubuntu.com/ubuntu hardy-security multiverse
deb-src http://security.ubuntu.com/ubuntu hardy-security multiverse

Head and tail are primarily used for log files, which can contain hundreds of lines of text. For example, someone administering a web server would no doubt find tail invaluable to examine the server logs.

By default head and tail display 10 lines of text, but you can change this with the -n modifier, as in this example:

matthew@morpheus:~$ tail -n5 /etc/apt/sources.list
deb-src http://security.ubuntu.com/ubuntu hardy-security main restricted
deb http://security.ubuntu.com/ubuntu hardy-security universe
deb-src http://security.ubuntu.com/ubuntu hardy-security universe
deb http://security.ubuntu.com/ubuntu hardy-security multiverse
deb-src http://security.ubuntu.com/ubuntu hardy-security multiverse

This applies to head as well.
Over the next few sessions, we'll go into some of the other useful tools that are available for manipulating text files in Ubuntu.

Handy shortcuts in the bash shell

You may have noticed that I've pushed the use of the command line (or, to give it the proper name, the shell), over graphical utilities much of the time. This is deliberate - the shell is a fast, flexible and powerful interface. Once you've got a bit of experience, most things can be done a lot faster from the shell. Now that I'm used to apt-get and apt-cache, I almost never use Synaptic to install anything in Ubuntu as it's so much quicker and more efficient to use the shell.

There are several different shells available for Unix-like operating systems, such as the original Bourne shell, the C shell, the Korn shell (used in Sun's Solaris operating system, among others), the Z shell (which was used in early versions of Mac OS X), and others. Linux uses the bash shell (in common with more recent versions of Mac OS X and OpenSolaris), which stands for Bourne Again SHell. This is a free software rewrite of the Bourne shell, but with many of the more advanced features of other shells incorporated into it.

Bash includes a number of very handy shortcuts that really make a difference. Don't you hate having to type out filenames in full, such as package-0.8.1.tar.gz? Me too, but bash has an answer - autocompletion.

To use autocompletion, start typing a command, then press tab. If bash has enough to confirm what you're typing, it will automatically fill in the rest. For example:

sudo apt-get upd

Press tab here, and there's enough for bash to autocomplete to this:

sudo apt-get update

But what if you enter this:

sudo apt-get up

Here, there's more than one option. The final word could be update or upgrade, so bash doesn't know how to complete it. Instead, when you press tab, you'll hear a bell to show that bash can complete it. But press it again and you'll be shown a list of possible options, as in this example:

matthew@morpheus:~$ sudo apt-get up
update upgrade
matthew@morpheus:~$ sudo apt-get up

Note that although the line you were on ends, what you'd entered is copied to the new line so you don't have to retype it.

This works for filenames and commands, so you could use autocompletion in this example with the file package-0.8.1.tar.gz:

tar -xzf pack

Assuming there wasn't another file in the current directory whose first four letters were pack, this would autocomplete fine. You can also use autocompletion to fill in the names of folders. If you had a path to a file like this:

/usr/share/doc

You could type one or two letters for each folder within the path, and press tab to fill in the rest of the folder name.

Autocompletion is a very handy feature in bash that can save you a lot of time, and makes using the shell a lot quicker and easier.

There are some other handy keyboard shortcuts you might want to be aware of. I've mentioned in a previous post that if you've started a process and would like to cancel it, you can press Ctrl-C and the process will stop dead.

Another handy shortcut is Ctrl-D, which acts the same as the exit command ie. it closes down the shell you've been using. However, it's possible to have one shell running inside another so this doesn't necessarily mean it closes the terminal emulator. For instance, if you open the terminal, you have one shell running. If you then enter bash and press enter, this opens a new shell and places you in it (this would be a subshell). By pressing Ctrl-D here, you'd be returned to your original shell. Pressing it again would close the terminal.

Another is Ctrl-L, which clears the terminal. A similar effect can be had by entering the clear command.

If you want to repeat previous commands, you can use the up and down cursor keys to scroll through previously used commands in the current session. For example, if you entered ls, then less text.txt, you could press up twice to get ls again.

Finally, two things you may want to know are how to copy and paste. Copying is Ctrl-Shift-C, while pasting is Ctrl-Shift-V or Shift-Insert.

If you're interested in learning a few more shortcuts, check out this link for more details. I'd recommend you get familiar with at least the commands mentioned above as these are extremely labour-saving. Also, over the next few posts I plan to delve into the shell a little more and you'll probably find these shortcuts invaluable.

I'd also recommend that you find a good command-line editor as these are invaluable for editing configuration files. I highly recommend you take the time to learn Vim using the tutorial mode built into it (just open the shell and enter vimtutor), as although Vim may seem tough at first, it's one of those things like touch-typing - at first it slows you down, but soon enough you'll be faster than you ever were. It's also a great text editor for if you want to program - I'm currently learning Python and I find it incredibly useful. But if you try it and it really doesn't work for you, try nano instead. Even though it's a command-line editor, it's simple to use.

Finally, if you're really finding bash an uphill struggle and would prefer something that really goes out of its way to help beginners, why not try fish? Short for Friendly Interactive SHell, this is a good choice for newbies, and a great learning tool. But don't give up on bash! Use fish to learn the ropes if you like it, but bash is more powerful and flexible in the long term. You can get fish from the repositories:

sudo apt-get install fish

Then, to run it, just enter the following:

fish

This will set fish running as a subshell within bash.

Navigating folders from the command line

If you've followed the posts on this blog for a while, you should be reasonably comfortable using the command line to install software, edit configuration files and so on. You should also be aware of the layout of the Linux filesystem. However, you may not be aware of how to navigate it without using a graphical file manager such as Nautilus or Konqueror. So in this post I'm going to go through basic commands for navigating from the command line.

First of all is the cd command. This is short for "change directory". The default location for you to start is in your home directory, and entering the following:

cd

will take you to your home directory. You can also use the ~ character (known as a tilde) as shorthand for your home directory, which is useful if referring to your home directory from elsewhere. So, entering the following:

cd ~

will give the same results as the previous command.

You can also navigate up one directory as follows:

cd ..

And can go back to the previous directory using this command:

cd -

These commands can be used as part of longer paths. For instance, say you are currently in /var/www but want to go to a folder called Music in your home folder. You can go there with this command:

cd /home/username/Music

Or you can use this:

cd ~/Music

Now, say you want to go from there to another folder in your home directory called Pictures. You can enter this:

cd ../Pictures

Then, to head back to the Music folder, you can enter the following:

cd -

And to head to the root of the filesystem, just enter the following:

cd /

Now, how can you see where you are? Easy, just enter the pwd command (short for Print Working Directory). Here I'm in the Desktop directory:

/home/matthew/Desktop

Got that? I recommend you spend a little time getting comfortable with these commands as they are the first building blocks in becoming proficient at the command line. Next time I'll go into creating and deleting files and folders.

Sudo

What is the sudo command? If you're familiar with Microsoft Windows, you'll no doubt understand what is meant by having administrative access - if you have it you can install new software, make changes to the system and things like that.

Linux works slightly differently to this. Most Linux distributions use what is called a
root account or superuser account to control what users can do. There can be any number of normal user accounts, but there is only ever one root account. To make any change at all to the system (such as installing or removing software), you need to log in as root.

Any operating system based on Unix has this, be it Linux, Mac OS X, FreeBSD or whatever. It's because Unix was designed right from the start to be a multi-user operating system, so it was designed so that many people could use it, but administrative access could be limited to certain individuals. This is why any Unix-based operating system is inherently more secure than Windows, which was designed as a single-user operating system.

However, it does have one or two disadvantages. For one thing, it can be a pain to log in as root, do what you have to do to administer your system, then log out and log back into your normal account. It can be tempting to just think "No, it'll be alright" and remain logged into root and go surfing the web. This can be a bad idea because it means that anything that gets into your computer from the Internet can then get in with administrative access, which could potentially cause problems. Also, it means more than one person may have to know the password for root, which could cause problems - I mean, how many times have you seen offices where people write passwords on Post-It notes so they can remember them? If several people need access to root and they have to write it on a piece of paper so they can remember that as well as their normal password, how bad would it be if someone got hold of that?

That's where sudo (pronounced soo-doo) comes in. It's available for many other distributions to install, but Ubuntu uses it by default (as does Mac OS X). Basically, sudo means that there is no root account. Instead, one or more users can use the sudo command to temporarily gain root access by entering their normal password again.

In use, you preface another command with it to run it as root. For instance, to use the apt-get program to install the game Pingus, here's what you would enter:

sudo apt-get install pingus

This is arguably safer because with sudo, root access expires after a few minutes, so you're safe to go surfing the Internet straight after having made whatever changes you want to. You can set which users can use sudo and which can't. So you can prevent your horrible little brother from installing any new software without your permission.

There are two similar commands for if you want to run graphical programs with root access. For instance, say you want to use the gedit text editor in Ubuntu to edit a text file that's important to the running of your system - let's say this is your /etc/apt/sources.list (you don't need to worry what this is at this stage). You'd enter the following:

gksudo gedit /etc/apt/sources.list

because gksudo is the graphical equivalent to sudo for the Gnome desktop. The KDE desktop has its own equivalent, kdesudo. So, if you wanted to edit the same file using the Kate text editor in Kubuntu, you could enter the following:

kdesudo kate /etc/apt/sources.list

Any application which will always need root access, such as when you open the Add/Remove Programs, will automatically prompt you for your password in this fashion. If you've used Windows Vista, it's very similar to that message that comes up when you want to install or remove anything, only you have to put in your password. So you may encounter kdesudo or gksudo like this even if you never input them on the command line.

Don't forget, it's there to keep your system secure.

Don't fear the command line!

If you've come to Linux from Windows (and probably if you've come to it from a Mac as well), you may not be used to the command line. You may indeed have never used it, and the thought of doing so might well scare you rigid.
Well, don't be scared. The command line is there to help you, and it really doesn't take that much effort to get used to it. When I was eight, I had an Amstrad CPC which had a command-line interface only, and I managed to use it fine, so I'm sure you will too.
Also, you might worry that by using the command line, you'll completely screw up your system. Believe me, it's just as possible to do that from a graphic user interface as it is from the command line.
There is a learning curve, no doubt about it, but it's not that steep. A handful of commands will see you in good stead. I wouldn't call myself a whizz with the command line by anybody's standards and only know a handful of commands, but with those I'm fully able to install any software I wish, add new software repositories, edit configuration files and manipulate files and folders.
It's possible to use Linux without ever using the command line if someone else is administering it for you, but if it's your own computer and you will be doing the administration, you'll probably soon need to use the command line. But DON'T PANIC! There's plenty of resources available to help you learn the basics -  try this link for basic commands in the command line. That should be enough to get you started.
Honestly, the command line is not that hard. And you may not believe me when I say this, but very often it is the quickest and easiest way to get things done. To install a new application (say, Pingus) using Synaptic Package Manager (or any other package manager, for that matter), you have to open the menu, go to the right submenu, click on the application, enter your password, sort through a long list of applications for the one you want, select it, then click a button to install it. Whereas with the command line, you just open a session and type:

sudo apt-get install pingus

Press enter, put in your password, and it does it all for you! It's quicker because the computer doesn't have to spend so much time making it look pretty for you - it can get on with the job at hand!
So dont' worry about it. It will take you no time at all to get used to it, I promise!

Blogged with Flock