The term "Unix" refers to a large family of operating systems. The first version of Unix was created by AT&T Bell Laboratories in the 1970s (the same group which created the C and C++ programming languages).
Today, there are several operating systems in the Unix family which share the same philosophy and design. These include:
A free operating system which runs on every type of computer. It is popular for web servers, super computers and forms the basis for Android phones and tablets.
OSX and iOS
Apple's operating systems for laptops, desktops, phones and tablets are all in the Unix family.
BSD is another free operating system, which is not quite so widely used.
Altogether, Unix runs on most web servers, all super computers, most phones and tablets and some laptop and desktop computers. The only widely used operating system not in the Unix family is Microsoft Windows. Though you can still apply everything you learn in this class on Windows with the help of tools such as Cygwin.
In this course, we will focus on Linux, though nearly everything we do will apply equally to the other systems.
In this course, we will learn to work with programs over the command line. When using the command line, you issue commands to the computer by typing them in, as opposed to using a graphical user interface.
A typical command line terminal.
While it may seem old-fashioned, there are many reasons to learn to use the command line effectively:
Different computer systems have very different graphical user interfaces, but command line interfaces have much less variation. For example, the GUI used by OSX, Android, and different versions of Linux are all completely different, but the command line interfaces are largely identical. If you learn to use the command line effectively, you can use that knowledge on a huge range of systems.
The command line interface in Unix systems does not change very frequently. Commands that were developed in the 1980s still work the same today. While new versions of GUI systems typically work quite differently, new versions of command line programs normally only add options, keeping existing usage intact.
Accessing remote machines is easier and more efficient when done via command line. While there are systems that allow for running GUI programs remotely, they can be difficult to setup and slower to use. Moreover, most server machines and super computers do not have GUI programs installed and can only be accessed through the command line.
Most people can type much faster than they can use a mouse. Many, but not all, tasks can be more efficiently completed by entering commands than by using a graphical interface.
Perhaps the biggest advantage of the command line is that complex or repetitive tasks can be automated with scripts. Command line programs on Unix all work with plain text, so different commands can be easily "chained up" together to build more complex actions. Generally, graphical programs won't work together unless they were designed to. Also, as programmers we can create new commands that work with the others.
As mentioned, a benefit of command-line systems is that they allow remote access. For this class, we will be connecting to remote servers that are part of the Google Cloud platform. This will be done using a protocol called Secure Shell (SSH).
SSH allows you to connect to, and interact with, a remote machine over the command line. SSH consists of a server which runs on the remote machine, and a client which runs on your local machine:
The SSH client sends your commands to the server where they are carried out. The commands are encrypted so that they cannot be read as they are sent over the network.
When you are connected to a machine over SSH, the commands you enter are not run on your own local machine. They are run on the machine you are connected to. Likewise, when you edit files over SSH, those files are not stored on your computer, they are on the server. Using SSH allows you to use a computer any where in the world as if you were sitting right next to it.
SSH is widely used in industry. For instance web developers use it to log in to web servers hosting their sites, data scientists use it to log into super computers to run analyses, and it can also be used to access embedded systems without displays (such as a Raspberry Pi).
The Google Cloud platform is a service run by Google and offers things like data hosting and application support. It also offers virtual machines running Linux which is what we will use it for.
A virtual machine (VM) is a computer system which is run on top of another one. When we create a VM, it is just like having our own personal server. In actuality Google runs many VMs on one physical computer.
The following instructions will guide you through setting up a virtual machine on the Google Cloud:
Navigate to https://console.cloud.google.com. If you are not signed into Google, you will be prompted to do so. If you do not have a Google account, you will need to create one for this course. You should then see a screen like this:
The terms and conditions
Check that you agree to the terms of service and then click "agree and continue". You should then see a screen that looks like this:
The Google Cloud Console
Next, click "Select a project" at the top of the screen, then the "New Project" button. You will then see a screen asking for the details:
Selecting a Project
You can call the project whatever you like, like "CPSC 225". Leave the "Location" part as it is. Then click create. It will take a few moments to set things up, and should then bring you back to the console.
Now, click "Select a project" again, and choose the project which you just created. A "project" is for organizing multiple resources together, we won't really use that, but still have to create one. Now you should see a screen like this:
Notice the Project is selected at the top bar
Now we need to create our virtual machine. The virtual machine will be the Unix computer that we connect to. To create it, click the Navigation menu in the upper left (the three white lines or "hamburger menu"). Then click on "Compute Engine". That should take you to this screen:
Waiting for the Compute Engine
After the Compute Engine is ready, click the "Create" button.
Click the "Create" button. You should now see your machine listed:
Our VM is Ready
Now we are ready to connect to the virtual machine. Do this now by clicking on the "SSH" button under the "Connect" listing. This will open the connection in a new tab. The machine may take a little while to start up, but eventually you should see a prompt like this:
Successfully in the VM
Now we will perform a couple of configuration changes. The first is to allow us to login to the VM with a password. To do this, type the command:
sudo nano /etc/ssh/sshd_config
Hit enter, and then hit the down arrow until you get to a line which says:
Change the "no" on that line to say "yes". When you're done it should look like this:
Allowing us to login with a Password
Next hit Control-O, then Enter to save the file. Then hit Control-X to quit.
Then run the following command:
sudo service ssh restart
We are now able to login with a password, but first we need to set a password. To do this, first observe that your username for the VM is the name of the Google account you used.
Note: Your username is not your display name and can't have any spaces. You should see your username on the command line. For instance mine is "finlaysoni" in the example above, not "Ian Finlayson".
Then run the following command:
sudo passwd username
Of course replace username with your Google account user name.
Note that the command
passwd is not the whole word password.
After entering that command, you will be prompted to enter a password for the VM. When you enter the password it will not show up, not even as dots or stars. This is normal in command line systems. Just enter type your password and hit enter when done. You will be asked to retype it.
Setting our Password
The next configuration change is to give our user administration privileges. To do this, enter the command:
Then scroll down to the bottom of the file, and add the following line:
USER ALL = (ALL) NOPASSWD: ALL
except replace USER with your user name on the VM.
Then hit Control-X to quit. It will ask if you want to save, answer Y for yes. Then hit enter to confirm the file name.
wget http://ianfinlayson.net/class/cpsc225/assignments/submit chmod +x ./submit sudo mv ./submit /usr/bin/
It will be inconvenient to connect to our VM using the method which we have just used each time. It will be quicker and easier to use another SSH client. There are different clients available depending on the operating system of your local machine. You can see instructions for setting up SSH clients either Windows or OSX/Linux below:
No matter which SSH client you use, note that it's possible to open up multiple SSH windows at one time. Rather than do everything in one window, it often is easier to write a program in one window, and run it in another.
The linked instructions above also contain optional instructions on connecting over SSH using keys instead of a password. The advantages of this are that you do not need to type your password to login, but also that keys are generally more secure. Using a key is like using a super-long extra secure password which is entered automatically for you.
SSH keys come in pairs, there is a public key and a private key. The public key can be read by anybody and is placed on the server you want to log into.
The private key must be kept secret. If anybody gets a hold of it, they will be able to login as you and access your files. You should generally only set up SSH keys to login from systems which are secure.
Warning: if you connect from the Google Cloud console again, it will overwrite your SSH keys! I recommend only using your native client once you set this up.
Once you have logged in, you should see something like this:
Logged into our VM.
You are now running a shell on your VM which is ready to take your
commands. The text that you see at the bottom ("
my example) is called a command prompt or just a prompt. The shell will
take commands that you type at the prompt and execute them.
You can now enter commands which will be carried out on the virtual machine server. For example, you can type the 'date' command, and then hit the Enter key to see the current time and date:
passwd commands as we entered them are very
simple, "one word" commands. Many commands take arguments that they
operate on (like parameters to a function), and options (also sometimes
called flags which control how the command works.
For example, the
cal command can be used to display a calendar on
the command line. It can be used without arguments or options in which it will
simply display the current month, with the date highlighted:
finlaysoni@myvm:~$ cal June 2018 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
However, we can also give cal a month and year as an argument. So if I was curious what day of the week I was born on, I could enter the following:
finlaysoni@myvm:~$ cal 10 1984 October 1984 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Here, "10" and "1984" are arguments to the
cal command. For a look
at a historical anomaly, try passing the arguments "9" and "1752" to cal!
cal also accepts several options. Options in Unix commands begin with the
hyphen character. For example, the
-y option tells cal display
the whole year instead of just the current month:
finlaysoni@myvm:~$ cal -y 2018 January February March Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 1 2 3 1 2 3 7 8 9 10 11 12 13 4 5 6 7 8 9 10 4 5 6 7 8 9 10 14 15 16 17 18 19 20 11 12 13 14 15 16 17 11 12 13 14 15 16 17 21 22 23 24 25 26 27 18 19 20 21 22 23 24 18 19 20 21 22 23 24 28 29 30 31 25 26 27 28 25 26 27 28 29 30 31 April May June Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 1 2 3 4 5 1 2 8 9 10 11 12 13 14 6 7 8 9 10 11 12 3 4 5 6 7 8 9 15 16 17 18 19 20 21 13 14 15 16 17 18 19 10 11 12 13 14 15 16 22 23 24 25 26 27 28 20 21 22 23 24 25 26 17 18 19 20 21 22 23 29 30 27 28 29 30 31 24 25 26 27 28 29 30 July August September Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 1 2 3 4 1 8 9 10 11 12 13 14 5 6 7 8 9 10 11 2 3 4 5 6 7 8 15 16 17 18 19 20 21 12 13 14 15 16 17 18 9 10 11 12 13 14 15 22 23 24 25 26 27 28 19 20 21 22 23 24 25 16 17 18 19 20 21 22 29 30 31 26 27 28 29 30 31 23 24 25 26 27 28 29 30 October November December Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 1 2 3 1 7 8 9 10 11 12 13 4 5 6 7 8 9 10 2 3 4 5 6 7 8 14 15 16 17 18 19 20 11 12 13 14 15 16 17 9 10 11 12 13 14 15 21 22 23 24 25 26 27 18 19 20 21 22 23 24 16 17 18 19 20 21 22 28 29 30 31 25 26 27 28 29 30 23 24 25 26 27 28 29 30 31
Some options require arguments themselves. For example the "-m" option to cal requires the month as an argument. So to check the date of Christmas, we could use the following command:
finlaysoni@myvm:~$ cal -m 12 December 2018 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Unix commands let you combine arguments and options which gives the command line so much flexibility.
This section contains some helpful tips for working at the shell.
The shell saves your previous commands, and you can scroll through that history using the "up" and "down" keys on your keyboard. This allows you to repeat commands that you have used in the past. You can also edit the commands before executing them. This is really handy for when you reat a command (like running a program) over and over. Also, if you mistype a command, you can go back and edit it rather than type it all from scratch.
You can have the shell complete
your commands by hitting the "tab" key. For example, rather than type the
passwd command, we could instead have typed
then hit the tab key to complete the command to
passwd. While that
does not make much difference in this case, it make longer commands faster to
Tab completion also lets you find commands more easily. If you remember
passwd command begins with "pas", but cannot remember the
exact spelling, you can type
pas and hit tab. The system will then
either complete the command for you, or tell you all of the commands that start
with "pas", if there are multiple.
Finally, to exit the shell, you can simply enter "Control-D". That is, hold the control key and tap the "D" key on your keyboard.
You can also enter the
exit command to exit the shell.
Copyright © 2022 Ian Finlayson | Licensed under a Creative Commons Attribution 4.0 International License.