The term “shell scripting” gets mentioned often in Linux forums, but many users aren’t familiar with it. Learning this easy and powerful programming method can help you save time, learn the command-line better, and banish tedious file management tasks.
What Is Shell Scripting?
Being a Linux user means you play around with the command-line. Like it or not, there are just some things that are done much more easily via this interface than by pointing and clicking. The more you use and learn the command-line, the more you see its potential. Well, the command-line itself is a program: the shell. Most Linux distros today use Bash, and this is what you’re really entering commands into.
Now, some of you who used Windows before using Linux may remember batch files. These were little text files that you could fill with commands to execute and Windows would run them in turn. It was a clever and neat way to get some things done, like run games in your high school computer lab when you couldn’t open system folders or create shortcuts. Batch files in Windows, while useful, are a cheap imitation of shell scripts.
Shell scripts allow us to program commands in chains and have the system execute them as a scripted event, just like batch files. They also allow for far more useful functions, such as command substitution. You can invoke a command, like date, and use it’s output as part of a file-naming scheme. You can automate backups and each copied file can have the current date appended to the end of its name. Scripts aren’t just invocations of commands, either. They’re programs in their own right. Scripting allows you to use programming functions — such as ‘for’ loops, if/then/else statements, and so forth — directly within your operating system’s interface. And, you don’t have to learn another language because you’re using what you already know: the command-line.
That’s really the power of scripting, I think. You get to program with commands you already know, while learning staples of most major programming languages. Need to do something repetitive and tedious? Script it! Need a shortcut for a really convoluted command? Script it! Want to build a really easy to use command-line interface for something? Script it!
Before You Begin
Before we begin our scripting series, let’s cover some basic information. We’ll be using the bash shell, which most Linux distributions use natively. Bash is available for Mac OS users and Cygwin on Windows, too. Since it’s so universal, you should be able to script regardless of your platform. In addition, so long as all of the commands that are referenced exist, scripts can work on multiple platforms with little to no tweaking required.
Scripting can easily make use of “administrator” or “superuser” privileges, so it’s best to test out scripts before you put them to work. Also use common sense, like making sure you have backups of the files you’re about to run a script on. It’s also really important to use the right options, like –i for the rm command, so that your interaction is required. This can prevent some nasty mistakes. As such, read through scripts you download and be careful with data you have, just in case things go wrong.
At their core, scripts are just plain text files. You can use any text editor to write them: gedit, emacs, vim, nano… This list goes on. Just be sure to save it as plain text, not as rich text, or a Word document. Since I love the ease of use that nano provides, I’ll be using that.
Script Permissions and Names
Scripts are executed like programs, and in order for this to happen they need to have the proper permissions. You can make scripts executable by running the following command on it:
This will allow anyone to run that particular script. If you want to restrict its use to just your user, you can use this instead:
In order to run this script, you would have to cd into the proper directory and then run the script like this:
To make things more convenient, you can place scripts in a “bin” folder in your home directory:
./script1
In many modern distros, this folder no longer is created by default, but you can create it. This is usually where executable files are stored that belong to your user and not to other users. By placing scripts here, you can just run them by typing their name, just like other commands, instead of having to cd around and use the ‘./’ prefix.
Before you name a script, though, you should the following command to check if you have a program installed that uses that name:
A lot of people name their early scripts “test,” and when they try to run it in the command-line, nothing happens. This is because it conflicts with the test command, which does nothing without arguments. Always be sure your script names don’t conflict with commands, otherwise you may find yourself doing things you don’t intend to do!
Scripting Guidelines
As I mentioned before, every script file is essentially plain text. That doesn’t mean you can write what you want all willy-nilly, though. When a text file is attempted to be executed, shells will parse through them for clues as to whether they’re scripts or not, and how to handle everything properly. Because of this, there are a few guidelines you need to know.
Every script should being with “#!/bin/bash” Every new line is a new command Comment lines start with a # Commands are surrounded by ()
The Hash-Bang Hack
When a shell parses through a text file, the most direct way to identify the file as a script is by making your first line:
If you use another shell, substitute its path here. Comment lines start with hashes (#), but adding the bang (!) and the shell path after it is a sort of hack that will bypass this comment rule and will force the script to execute with the shell that this line points to.
New Line = New Command
Every new line should be considered a new command, or a component of a larger system. If/then/else statements, for example, will take over multiple lines, but each component of that system is in a new line. Don’t let a command bleed over into the next line, as this can truncate the previous command and give you an error on the next line. If your text editor is doing that, you should turn off text-wrapping to be on the safe side. You can turn off text wrapping in nano bit hitting ALT+L.
Comment Often with #s
If you start a line with a #, the line is ignored. This turns it into a comment line, where you can remind yourself of what the output of the previous command was, or what the next command will do. Again, turn off text wrapping, or break you comment into multiple lines that all begin with a hash. Using lots of comments is a good practice to keep, as it lets you and other people tweak your scripts more easily. The only exception is the aforementioned Hash-Bang hack, so don’t follow #s with !s. ;-)
Commands Are Surrounded By Parentheses
In older days, command substitutions were done with single tick marks (`, shares the ~ key). We’re not going to be touching on this yet, but as most people go off and explore after learning the basics, it’s probably a good idea to mention that you should use parentheses instead. This is mainly because when you nest — put commands inside other commands — parentheses work better.
Your First Script
Let’s start with a simple script that allows you to copy files and append dates to the end of the filename. Let’s call it “datecp”. First, let’s check to see if that name conflicts with something:
You can see that there’s no output of the which command, so we’re all set to use this name.
Let’s create a blank file in the ~/bin folder:
And, let’s change the permission now, before we forget:
Let’s start building our script then. Open up that file in your text editor of choice. Like I said, I like the simplicity of nano.
And, let’s go ahead and put in the prerequisite first line, and a comment about what this script does.
Next, let’s declare a variable. If you’ve ever taken algebra, you probably know what a that is. A variable allows us to store information and do things with it. Variables can “expand” when referenced elsewhere. That is, instead of displaying their name, they will display their stored contents. You can later tell that same variable to store different information, and any instruction that occurs after that will use the new information. It’s a really fancy placeholder.
What will we put in out variable? Well, let’s store the date and time! To do this, we’ll call upon the date command.
Take a look at the screenshot below for how to build the output of the date command:
You can see that by adding different variables that start with %, you can change the output of the command to what you want. For more information, you can look at the manual page for the date command.
Let’s use that last iteration of the date command, “date +%m_%d_%y-%H.%M.%S”, and use that in our script.
If we were to save this script right now, we could run it and it would give us the output of the date command like we’d expect:
But, let’s do something different. Let’s give a variable name, like date_formatted to this command. The proper syntax for this is as follows:
And for us, we’d build it like this:
This is what we call command substitution. We’re essentially telling bash that whenever the variable “date_formatted” shows up, to run the command inside the parentheses. Then, whatever output the commands gives should be displayed instead of the name of the variable, “date_formatted”.
Here’s an example script and its output:
Note that there are two spaces in the output. The space within the quotes of the echo command and the space in front of the variable are both displayed. Don’t use spaces if you don’t want them to show up. Also note that without this added “echo” line, the script would give absolutely no output.
Let’s get back to our script. Let’s next add in the copying part of the command.
This will invoke the copy command, with the –i and –v options. The former (“interactive”) will ask you for verification before overwriting a file, and the latter (“verbose”) will display on the command line what is being done.
Next, you can see I’ve added the “$1” option. When scripting, a dollar sign ($) followed by a number will denote that numbered argument of the script when it was invoked. For example, in the following command:
The first argument is “Trogdor2.mp3” and the second argument is “ringtone.mp3”.
Looking back at our script, we can see that we’re referencing two arguments:
This means that when we run the script, we’ll need to provide two arguments for the script to run correctly. The first argument, $1, is the file that will be copied, and is substituted as the “cp –iv” command’s first argument.
The second argument, $2, will act as the output file for the same command. But, you can also see that it’s different. We’ve added a period and we’ve referenced the “date_formatted” variable from above. Curious as to what this does?
Here’s what happens when the script is run:
You can see that the output file is listed as whatever I entered for $2, followed by a period, then the output of the date command! Makes sense, right?
Now when I run the datecp command, it will run this script and allow me to copy any file to a new location, and automatically add the date and time to end of the filename. Useful for archiving stuff!
Shell scripting is at the heart of making your OS work for you. You don’t have to learn a new programming language to make it happen, either. Try scripting with some basic commands at home and start thinking of what you can use this for.
Do you script? Have any advice for newbies? Share your thoughts in the comments! There’s more to come in this series!
