The term "Unix" refers to a large family of operating systems. The first version of Unix was created by AT&T Bell Laboratories in the 1970s (the same group which created the C and C++ programming languages).
Today, there are several operating systems in the Unix family which share the same philosophy and design. These include:
A free operating system which runs on every type of computer. It is popular for web servers, super computers and forms the basis for Android phones and tablets.
OSX and iOS
Apple's operating systems for laptops, desktops, phones and tablets are all in the Unix family.
BSD is another free operating system, which is not quite so widely used.
Altogether, Unix runs on most web servers, all super computers, most phones and tablets and some laptop and desktop computers. The only widely used operating system not in the Unix family is Microsoft Windows. Though you can still apply everything you learn in this class on Windows with the help of tools such as Cygwin.
In this course, we will focus on Linux, though nearly everything we do will apply equally to the other systems.
In this course, we will learn to work with programs over the command line. When using the command line, you issue commands to the computer by typing them in, as opposed to using a graphical user interface.
A typical command line terminal.
While it may seem old-fashioned, there are many reasons to learn to use the command line effectively:
Different computer systems have very different graphical user interfaces, but command line interfaces have much less variation. For example, the GUI used by OSX, Android, and different versions of Linux are all completely different, but the command line interfaces are largely identical. If you learn to use the command line effectively, you can use that knowledge on a huge range of systems.
The command line interface in Unix systems does not change very frequently. Commands that were developed in the 1980s still work the same today. While new versions of GUI systems typically work quite differently, new versions of command line programs normally only add options, keeping existing usage intact.
Accessing remote machines is easier and more efficient when done via command line. While there are systems that allow for running GUI programs remotely, they can be difficult to setup and slower to use. Moreover, most server machines and super computers do not have GUI programs installed and can only be accessed through the command line.
Most people can type much faster than they can use a mouse. Many, but not all, tasks can be more efficiently completed by entering commands than by using a graphical interface.
Perhaps the biggest advantage of the command line is that complex or repetitive tasks can be automated with scripts. Command line programs on Unix all work with plain text, so different commands can be easily "chained up" together to build more complex actions. Generally, graphical programs won't work together unless they were designed to. Also, as programmers we can create new commands that work with the others.
As mentioned, a benefit of command-line systems is that they allow remote access. For this class, we will be connecting to remote servers that are part of the Google Cloud platform. This will be done using a protocol called Secure Shell (SSH).
SSH allows you to connect to, and interact with, a remote machine over the command line. SSH consists of a server which runs on the remote machine, and a client which runs on your local machine:
The SSH client sends your commands to the server where they are carried out. The commands are encrypted so that they cannot be read as they are sent over the network.
When you are connected to a machine over SSH, the commands you enter are not run on your own local machine. They are run on the machine you are connected to. Likewise, when you edit files over SSH, those files are not stored on your computer, they are on the server. Using SSH allows you to use a computer any where in the world as if you were sitting right next to it.
SSH is widely used in industry. For instance web developers use it to log in to web servers hosting their sites, data scientists use it to log into super computers to run analyses, and it can also be used to access embedded systems without displays (such as a Raspberry Pi).
The Google Cloud platform is a service run by Google and offers things like data hosting and application support. It also offers virtual machines running Linux which is what we will use it for.
A virtual machine (VM) is a computer system which is run on top of another one. When we create a VM, it is just like having our own personal server. In actuality Google runs many VMs on one physical computer.
The following instructions will guide you through setting up a virtual machine on the Google Cloud:
Navigate to https://console.cloud.google.com. You should then see a screen like this:
The Google Cloud Console
If you are not signed into Google, you will be prompted to do so. If you do not have a Google account, you will need to create one for this course.
Next, click "Select a project" at the top of the screen, then the "New Project" button:
Selecting a Project
You can call the project whatever you like, like "CPSC 225". Leave the "Organization" part blank. Then click create.
Now, click "Select a project" again, and choose the project which you just created. A "project" is for organizing multiple resources together, we won't really use that, but still have to create one. Now you should see a screen like this:
Notice the Project is selected at the top bar
Now we need to create our virtual machine. The virtual machine will be the Unix computer that we connect to. To create it, click the Navigation menu in the upper left (the three white lines or "hamburger menu"). Then click on "Compute Engine". That should take you to this screen:
Waiting for the Compute Engine
After the Compute Engine is ready, click the "Create" button.
In order to ensure that you have created a free VM, and won't use up your credit, you should see a note under the charge which says the first 720 or 744 hours of the month are free:
720 hours is 30 days, and 744 minutes is 31 days, so the VM will be free for the whole month, and won't use your credit at all. If you don't see that, then double-check the settings, or let me know.
Click the "Create" button. You should now see your machine listed:
Our VM is Ready
Now we are ready to connect to the virtual machine. Do this now by clicking on the "SSH" button under the "Connect" listing. This will open the connection in a new tab. The machine may take a little while to start up, but eventually you should see a prompt like this:
Succesfully in the VM
Now we will perform a couple of configuration changes. The first is to allow us to login to the VM with a password. To do this, type the command:
sudo nano /etc/ssh/sshd_config
Then hit the down arrow until you get to a line which says:
Change the "no" on that line to say "yes". When you're done it should look like this:
Allowing us to login with a Password
Next hit Control-O, then Enter to save the file. Then hit Control-X to quit.
Then run the following command:
sudo service ssh restart
We are now able to login with a password, but first we need to set a password. To do this, first observe that your username for the VM is the name of the Google account you used.
Note: Your username is not your display name and can't have any spaces. You should see your username on the command line. For instance mine is "finlaysoni" in the example above, not "Ian Finlayson".
Then run the following command:
sudo passwd username
Of course replace username with your Google account user name. Note that the command passwd is not the whole word password.
After entering that command, you will be prompted to enter a password for the VM. When you enter the password it will not show up, not even as dots or stars. This is normal in command line systems. Just enter type your password and hit enter when done. You will be asked to retype it.
Setting our Password
The last configuration change is to give our user administration privileges. To do this, enter the command:
Then scroll down to the bottom of the file, and add the following line:
USER ALL = (ALL) NOPASSWD: ALL
except replace USER with your user name on the VM.
Then hit Control-X to quit. It will ask if you want to save, answer Y for yes. Then hit enter to confirm the file name.
It will be inconvenient to connect to our VM using the method which we have just used each time. It will be quicker and easier to use another SSH client. There are different clients available depending on the operating system of your local machine. You can see instructions for setting up SSH clients either Windows or OSX/Linux below:
No matter which SSH client you use, note that it's possible to open up multiple SSH windows at one time. Rather than do everything in one window, it often is easier to write a program in one window, and run it in another.
The linked instructions above also contain optional instructions on connecting over SSH using keys instead of a password. The advantages of this are that you do not need to type your password to login, but also that keys are generally more secure. Using a key is like using a super-long extra secure password which is entered automatically for you.
SSH keys come in pairs, there is a public key and a private key. The public key can be read by anybody and is placed on the server you want to log into.
The private key must be kept secret. If anybody gets a hold of it, they will be able to login as you and access your files. You should generally only set up SSH keys to login from systems which are secure.
Warning: if you connect from the Google Cloud console again, it will overwrite your SSH keys! I recommend only using your native client once you set this up.
Once you have logged in, you should see something like this:
Logged into our VM.
You are now running a shell on your VM which is ready to take your commands. The text that you see at the bottom ("finlaysoni@myvm:~$" in my example) is called a command prompt or just a prompt. The shell will take commands that you type at the prompt and execute them.
You can now enter commands which will be carried out on the virtual machine server. For example, you can type the 'date' command, and then hit the Enter key to see the current time and date:
The date and passwd commands as we entered them are very simple, "one word" commands. Many commands take arguments that they operate on (like parameters to a function), and options (also sometimes called flags which control how the command works.
For example, the cal command can be used to display a calendar on the command line. It can be used without arguments or options in which it will simply display the current month, with the date highlighted:
finlaysoni@myvm:~$ cal June 2018 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
However, we can also give cal a month and year as an argument. So if I was curious what day of the week I was born on, I could enter the following:
finlaysoni@myvm:~$ cal 10 1984 October 1984 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Here, "10" and "1984" are arguments to the cal command. For a look at a historical anomaly, try passing the arguments "9" and "1752" to cal!
cal also accepts several options. Options in Unix commands begin with the hyphen character. For example, the -y option tells cal display the whole year instead of just the current month:
finlaysoni@myvm:~$ cal -y 2018 January February March Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 1 2 3 1 2 3 7 8 9 10 11 12 13 4 5 6 7 8 9 10 4 5 6 7 8 9 10 14 15 16 17 18 19 20 11 12 13 14 15 16 17 11 12 13 14 15 16 17 21 22 23 24 25 26 27 18 19 20 21 22 23 24 18 19 20 21 22 23 24 28 29 30 31 25 26 27 28 25 26 27 28 29 30 31 April May June Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 1 2 3 4 5 1 2 8 9 10 11 12 13 14 6 7 8 9 10 11 12 3 4 5 6 7 8 9 15 16 17 18 19 20 21 13 14 15 16 17 18 19 10 11 12 13 14 15 16 22 23 24 25 26 27 28 20 21 22 23 24 25 26 17 18 19 20 21 22 23 29 30 27 28 29 30 31 24 25 26 27 28 29 30 July August September Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 1 2 3 4 1 8 9 10 11 12 13 14 5 6 7 8 9 10 11 2 3 4 5 6 7 8 15 16 17 18 19 20 21 12 13 14 15 16 17 18 9 10 11 12 13 14 15 22 23 24 25 26 27 28 19 20 21 22 23 24 25 16 17 18 19 20 21 22 29 30 31 26 27 28 29 30 31 23 24 25 26 27 28 29 30 October November December Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 1 2 3 1 7 8 9 10 11 12 13 4 5 6 7 8 9 10 2 3 4 5 6 7 8 14 15 16 17 18 19 20 11 12 13 14 15 16 17 9 10 11 12 13 14 15 21 22 23 24 25 26 27 18 19 20 21 22 23 24 16 17 18 19 20 21 22 28 29 30 31 25 26 27 28 29 30 23 24 25 26 27 28 29 30 31
Some options require arguments themselves. For example the "-m" option to cal requires the month as an argument. So to check the date of Christmas, we could use the following command:
finlaysoni@myvm:~$ cal -m 12 December 2018 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Unix commands let you combine arguments and options which gives the command line so much flexibility.
This section contains some helpful tips for working at the shell.
The shell saves your previous commands, and you can scroll through that history using the "up" and "down" keys on your keyboard. This allows you to repeat commands that you have used in the past. You can also edit the commands before executing them. This is really handy for when you reat a command (like running a program) over and over. Also, if you mistype a command, you can go back and edit it rather than type it all from scratch.
You can have the shell complete your commands by hitting the "tab" key. For example, rather than type the entire passwd command, we could instead have typed pass and then hit the tab key to complete the command to passwd. While that does not make much difference in this case, it make longer commands faster to type.
Tab completion also lets you find commands more easily. If you remember that the passwd command begins with "pas", but cannot remember the exact spelling, you can type pas and hit tab. The system will then either complete the command for you, or tell you all of the commands that start with "pas", if there are multiple.
Finally, to exit the shell, you can simply enter "Control-D". That is, hold the control key and tap the "D" key on your keyboard.
You can also enter the exit command to exit the shell.
Copyright © 2019 Ian Finlayson | Licensed under a Creative Commons Attribution 4.0 International License.