This week we will discuss the ways in which files and directories are organized under Unix. We will also talk about commands for interacting with files and directories.
Unix uses the term "directory" while other systems use the term "folder". Here, we will call them directories, but the concept is the same.
The file system is a tree with "/" as the root of the tree. Every other directory or file is under "/" some place. Note that "/" is itself a directory (akin to the "C:" folder on Windows systems.
The following image shows a portion of the file system tree:
The Unix directory structure
Your personal files are stored in your "home directory" which is a sub-directory of /home/. When you login, you automatically start in your home directory.
The directories outside of /home/ contain programs and files used by the system. You will not need to navigate these directories normally.
Some of these are summarized below:
/usr/
The "Unix System Resources" directory contains most programs and that you will run, along with data files used by those programs.
/usr/bin/
Program files are stored here, including most of the commands that you will run.
/usr/share/
Contains most data files used by the programs in /usr/bin/.
/etc/
Contains configuration files for the operating system.
/tmp/
Used for storing temporary files that can be deleted at any time. Unlike the directories above, regular users can create files in this directory.
Most of the time, you will be working in your home directory, but it is good to have a sense of how the system as a whole fits together. When we get to administration tasks, we will talk more about some of these system directories.
When you are first logged in to the cpsc server, you will be in your home
directory, which is a directory under /home/students named the same as your username.
The directory that you are in is called your present working directory,
and you can always find what it is using the pwd command:
ifinlay@cpsc:~$ pwd /home/faculty/ifinlay
In my examples, my username is "ifinlay", and my home directory is in /home/faculty.
You can list the files and directories in your present working directory
with the ls command:
ifinlay@cpsc:~$ ls ifinlay@cpsc:~$
Right now, there is nothing in the home directory because we have not yet created any files!
You can pass ls an argument of another directory and it will tell you the contents of that directory instead of the present working directory:
ifinlay@cpsc:~$ ls / bin home lib64 opt sbin tmp vmlinuz.old boot initrd.img lost+found proc snap usr dev initrd.img.old media root srv var etc lib mnt run sys vmlinuz
ls also supports many options. In fact, of the 26 lowercase letters, only 'e', 'j', and 'y' are not valid options to ls. You will probably never need most of these options, but some helpful ones are listed below:
-a
Show all files, even hidden ones that begin with a '.' character. These hidden files are used for your personal configurations to programs that you use. For instance, you have a hidden ".ssh/" directory within your home directory for SSH configuration files. Normally ls does not display these files, but the -a option tells it to.
-l
Give long file listings including information on files such as the owner, size, date last modified and permissions.
-h
Only useful in conjunction with -l, this gives file sizes in human readable format. By default, ls just gives the number of bytes which is hard for us to read. The -h flag gives more readable listings in units such as kilobytes, megabytes or gigabytes, whichever is appropriate.
-t
Sort by modification time, with the most recently modified files first, instead of alphabetically by filename.
-S
Sorts the files by size, with the largest first, instead of filename.
-r
Reverse the order of the files being listed. With just the "-r" flag, files will be listed alphabetically from z to a. This flag can also be combined with "-t" or "-S".
-R
List files recursively. This means that if there are directories in the listing, all of their contents will be listed as well. If those sub-directories themselves have sub-directories, their contents will be listed. This causes all files any where under the present working directory to be listed.
For instance, we can look at all of the hidden directories in our home directory, with details, and sizes in human readable format with:
ifinlay@cpsc:~$ ls -alh total 44K drwxr-xr-x 6 ifinlay faculty 4.0K Jun 20 18:09 . drwxr-xr-x 4 root root 4.0K Jun 19 16:26 .. -rw------- 1 ifinlay faculty 289 Jun 20 18:09 .bash_history -rw-r--r-- 1 ifinlay faculty 220 Apr 4 18:30 .bash_logout -rw-r--r-- 1 ifinlay faculty 3.7K Apr 4 18:30 .bashrc drwx------ 2 ifinlay faculty 4.0K Jun 19 16:29 .cache drwx------ 3 ifinlay faculty 4.0K Jun 19 16:29 .gnupg drwxrwxr-x 3 ifinlay faculty 4.0K Jun 20 18:05 .local -rw-r--r-- 1 ifinlay faculty 807 Apr 4 18:30 .profile drwxrwxr-x 2 ifinlay faculty 4.0K Jun 20 18:09 .ssh -rw------- 1 ifinlay faculty 1.6K Jun 20 18:05 .viminfo
Note that the options can be given in any order. The above command could
be given as ls -lah or ls -hla or any other order and it
would do the same thing.
The recursive flag is worth looking at, as many other commands share a recursive option. If we list files recursively, it shows what's in the directories here, instead of just listing the directory names:
ifinlay@cpsc:~$ ls -aR .: . .bash_history .bashrc .gnupg .profile .viminfo .. .bash_logout .cache .local .ssh ./.cache: . .. motd.legal-displayed ./.gnupg: . .. private-keys-v1.d ./.gnupg/private-keys-v1.d: . .. ./.local: . .. share ./.local/share: . .. nano ./.local/share/nano: . .. ./.ssh: . .. authorized_keys
Now ls list the files in our home directory first. Then it goes into
directories within here, and shows what's in them, as far down as needed. The
deepest directory is ".local/share/nano" which stores settings for the "nano"
text editor. You will also see that the .ssh directory has an authorized_keys file
(if you set up the SSH keys in week 1).
Essentially, by default ls shows us what things are here directly, whereas the recursive flag shows us everything that is here, no matter how many sub-directories it is in.
It is a good idea to keep the files in your home directory organized into their own directories. This keeps things tidy, makes it easier to find things and prevents you from accidentally changing the wrong files. Generally, making a directory for each project you work on is a good idea.
To create a directory, use the mkdir command which takes the
name of the new directory to create as an argument:
ifinlay@cpsc:~$ mkdir new-directory
Now if we run ls again, we will see our new directory:
ifinlay@cpsc:~$ ls new-directory
Note: it's simpler to avoid spaces in file and directory names in Unix which is why I've used a hyphen to separate the words instead of a space.
By default, mkdir does not allow you to create multiple levels
of directories at one time:
ifinlay@cpsc:~$ mkdir projects/project1 mkdir: cannot create directory 'projects/project1': No such file or directory
Here, the "projects" directory did not exist, so mkdir refused
to create the projects/project1 directory. If we pass mkdir the
"-p" option, it will create both directories:
ifinlay@cpsc:~$ mkdir -p projects/project1
You can than move into that directory using the cd command:
ifinlay@cpsc:~$ cd new-directory ifinlay@cpsc:new-directory$
Now you are no longer in your home directory, but the new directory that was just created:
ifinlay@cpsc:new-directory$ pwd /home/finlayson/new-directory
There are a couple of directory names in Unix which have special significance:
~
The tilde character always refers to your home directory. So if you want
to return to your home directory, you can always enter the command cd ~
no matter which directory you are currently in.
.
The . directory always refers to the current directory. The command
cd . does nothing, but the . directory will be useful in commands
we will learn later on.
..
The .. directory refers to one directory above us in the file system tree.
So if my present working directory is "/home/finlayson/new-directory", and
I enter the command cd .., then I will move to the "/home/finlayson/"
directory. If I enter cd .. again, then I will be in "/home/".
If we want to remove a directory, you can use the rmdir command to
do so:
ifinlay@cpsc:~$ rmdir new-directory
rmdir only allows you to remove empty directories. We will
talk about removing non-empty directories later on.
Creating and moving into directories is fun, but to get real work done, we need files as well.
The touch command creates an empty file. This is not something you
will need to do very often. Later, we will use Vim to create files and put things
in them.
For now, touch will let us create empty files that we can move
and remove without worry. To create an empty file, run touch and pass the file
name as an argument:
ifinlay@cpsc:new-directory$ touch file1 ifinlay@cpsc:new-directory$ ls file1
Note: the other main purpose of the touch command is to update the modified time of a file. If you pass it an existing file, it doesn't change it, but updates the time as if you had.
The command to copy a file is cp. The first argument is the file
to copy. The second argument is the location to copy it to. This can be a new
filename:
ifinlay@cpsc:new-directory$ ls file1 ifinlay@cpsc:new-directory$ cp file1 file2 ifinlay@cpsc:new-directory$ ls file1 file2
In this case, cp has copied 'file1' into 'file2'.
The second argument to cp can also be an existing directory
name. In this case, cp will copy the first argument into the
directory keeping the same name:
ifinlay@cpsc:new-directory$ ls file1 ifinlay@cpsc:new-directory$ mkdir directory ifinlay@cpsc:new-directory$ cp file1 directory ifinlay@cpsc:new-directory$ ls directory file1 ifinlay@cpsc:new-directory$ ls directory file1
Sometimes we may want to copy not just one file, but an entire directory
to a new location. For instance, suppose I have "new-directory" containing
a file called "file1", and I want to make a copy of the directory including
the file as "backup-directory". cp by default does not do this:
ifinlay@cpsc:~$ ls new-directory file1 ifinlay@cpsc:~$ cp new-directory backup-directory cp: omitting directory `new-directory'
To tell cp to copy a whole directory, we need the -R flag
which stands for recursive:
ifinlay@cpsc:~$ ls new-directory file1 ifinlay@cpsc:~$ cp -R new-directory backup-directory ifinlay@cpsc:~$ ls backup-directory file1
Just like before, "recursive" means not just the directory, but everything inside of it as well.
Moving files is done with the mv command which works very similarly
to cp except that it does not keep the original file where it is.
If the second argument to mv is a new name, then mv renames
the first argument to that name:
ifinlay@cpsc:new-directory$ ls file1 ifinlay@cpsc:new-directory$ mv file1 file2 ifinlay@cpsc:new-directory$ ls file2
If the second argument is the name of a directory, then mv will
move the file into that directory:
ifinlay@cpsc:new-directory$ ls file1 ifinlay@cpsc:new-directory$ mkdir directory ifinlay@cpsc:new-directory$ mv file1 directory ifinlay@cpsc:new-directory$ ls directory ifinlay@cpsc:new-directory$ ls directory file1
You have to be somewhat careful with the mv command as it can
remove files if you are not careful.
Say you have two files in a directory, named file1 and file2. Attempting to move file1 to file2 will actually erase the original file2 in the process:
ifinlay@cpsc:directory$ ls file1 file2 ifinlay@cpsc:directory$ mv file1 file2 ifinlay@cpsc:directory$ ls file2
Warning: Unix commands do not warn you when they will overwrite data. Furthermore, the command prompt has no "recycle bin". If you delete something, it is likely not recoverable.
If you want to use mv in "interactive mode", it will prompt you
before overwriting data:
ifinlay@cpsc:directory$ ls file1 file2 ifinlay@cpsc:directory$ mv -i file1 file2 mv: overwrite `file2'? n ifinlay@cpsc:directory$ ls file1 file2
Note that mv now asked if file2 should be overwritten, to
which you can answer 'y' or 'n'.
We have seen how to create, copy and move files. Now we will see how to
remove files with the rm command.
To remove one or more files, pass them as arguments to rm:
ifinlay@cpsc:directory$ ls file1 file2 ifinlay@cpsc:directory$ rm file1 file2 ifinlay@cpsc:directory$ ls
rm also has a -i flag wherein it will prompt you for each
file that is being removed:
ifinlay@cpsc:directory$ ls file1 file2 ifinlay@cpsc:directory$ rm -i file1 file2 rm: remove regular empty file `file1'? y rm: remove regular empty file `file2'? n ifinlay@cpsc:directory$ ls file2
Normally, rm does not remove entire directories:
ifinlay@cpsc:new-directory$ ls directory ifinlay@cpsc:new-directory$ ls directory file1 file2 ifinlay@cpsc:new-directory$ rm directory rm: cannot remove `directory': Is a directory
We can tell rm to remove the directory recursively with
the -R option. Again, in Unix, "recursive" means to apply some operation
not just to one directory, but to every file or directory nested within
that.
ifinlay@cpsc:new-directory$ ls directory ifinlay@cpsc:new-directory$ ls directory file1 file2 ifinlay@cpsc:new-directory$ rm -R directory
You should be careful with the rm command, especially with the
-R flag. Remember that Unix has no "recycle bin", so if you delete a file,
it is usually not possible to get it back!
Copyright © 2024 Ian Finlayson | Licensed under a Creative Commons BY-NC-SA 4.0 License.