Variables and string in the shell

I have not really grasped the tricky variable use and string operations in the shell, and this post is a cheatsheet of the common variable and string operations in the shell.

Variables

Assigning values to variables, note that = immediately follows the variable name and it allows no space.

variable='value'
variable="value"
variable=value

There are several ways of calling a variable, by $variable, ${variable} and "$variable". The curly braces are useful when you need to clearly mark the boundary of the variable name. For example, $variableisvalue will not output the value of $variable since the shell interprets the name of the variable as variableisvalue. To tell the shell where the variable name ends, surround the variable name with curly braces ${variable}isvalue, which is not ambiguous to the shell.

echo $variable
echo ${variable}
echo "$variable"

The arguments to the script can be accessed as common variables. For example, the following lists the common ways of accessing the script arguments.

Apart from the variables that you need to define yourself, the shell stores a set of environment variables, such as $HOME, $PATH, etc. The command printenv lists all these variables.

There are also a bunch of variables called positional parameters, including the following variables.

• $0: the basename of the program
• $1 .. $9: the first nine additional arguments the script was called with
• $@: all arguments, each as a separate string
• $*: all arguments as a string, without space
• $#: the number of arguments
• $?: the exit status of the last run command
• $$: PID (Process IDentifier) of the current shell
• $!: PID (Process IDentifier) of the last run background process
• IFS: Internal Field Separator

$* vs. $@

The difference between $* and $@ is tricky. Both refers to all the arguments, and when they are used without quote, they are equivalent. But when used with double quote, "$*" stands for all the arguments as a single string, but each arguments are treated separately in "$@". The following demonstrates the difference:

# run ./args.sh arg1 "arg2 word" arg3
for a in $*; do
  echo "${a}"
done

# The output is:
# > arg1
# > arg2 word
# > arg3

for a in "$*"; do
  echo "${a}"
done

# The output is:
# > arg1 arg2 word arg3

Single and double quotes

Unlike in many other languages, single and double quotes are treated differently by the shell. The shell treats the characters literally surrounded by single quotes, but within double quotes, $ and certain other signs are evaluated. If $ is used in double quotes with a backslash, or an escape character \, it will not be evaluated. It sounds very hard to understand, but its actually quite clear, as demonstrated below

More generally, in double quotes, parameter expansion, arithmetic expansions, and command substitution can all take place, which are all suppressed in single quotes.

String operations

The following table shows the command with its meaning, and an example for each together with its output.

• ${#variable}: the length of the string
• ${var1}{var2}{var3}: concatenate strings, with optional string in between (same below)
• $var1$var2$var3: concatenate strings
• "$var1""$var2""$var3": concatenate strings
• ${string:position}: substring from $position till the end
• ${string:position:length}: a substring of $length characters from $position
  • position can also be negative, ${string: -3} means the last three characters, note also the space before - is required
• ${string#pattern}: pattern is a wild card pattern, like *.txt. This one removes the first match from the beginning of the string, while the one below removes the longest match
• ${string##pattern}
• ${string%pattern}: replaces the pattern from the end of the string
• ${string/pattern/replace}: replace the matched pattern for the first occurrence
• ${string//pattern/replace}: replace the matched pattern for all occurrences
• ${string/#pattern/replace}: replace the matched pattern at the beginning of the string
• ${string/%pattern/replace}: replace the matched pattern at the end of the string
• $((expression)): arithmetic operations, including +, -, *, /, **, %

String expressions

String expressions can be useful in the if statement.

• string: string is not null
• -n string: the length of string is greater than zero
• -z string: the length of string is zero
• string1 = string2: string1 and string2 are equal
• string1 == string2: string1 and string2 are equal. This is preferred over the previous one with single equal sign
• string1 != string2: string1 and string2 are not equal
• string1 > string2: string1 sorts after string2
• string1 < string2: string1 sorts before string2