Bash Scripts : Programming, Debugging, and Performance

This page serves to guide us through main points related to Programming. We introduce debugging strategies and comment on performance impact of aspects of script design choices..

Refer to the Bash Script Guides for sections that are not yet completed....

Bash Script Programming

Scripts can be done on a command line or within a file.

• command line example
• give a script in myScript.sh, from command line:
  • ./myScript.sh
  • ./myScript.sh &
  • bash ./myScript.sh
• Exit rc
  • <0 implies error
  • 0 implies success
  • >0 implies error with a specific error code

Functions

Tests:

The [] is shorthand for invoking the Bash test. The Bash comparision operators use textual operators for numbers and symbolic operators for strings (which seems backwards!).

• Strings (symbolic operators)
  • =, !=, <, >
  • -n an d-z for not null and is null for numbers
• Numbers (textual operators)
  • -eq, -ne, -lt, -gt,

Variables -

Control flow

• if test-command ; then
  • commands
• elif test-command ; then
  • commands
• else
  • commands
• fi

Example - look at the script numbersAndStrings.sh

Portion of that script - beware of numbers vs strings

• myString1=apple
  myString2=apple
  x=1
  y=1
  #let z=$x+$y #equivalent to next line
  z=$((x+y))

• if [ "$x" -eq 1 ] ; then
  echo "$0:$LINENO: Comparison works."
  else
  echo "$0:$LINENO: Comparison does NOT work"
  fi
• if [ "$x" = "$y" ] ; then
  echo "$0:$LINENO: Comparison works."
  else
  echo "$0:$LINENO: Comparison does NOT work"
  fi
• #both work
  if [ "$z" -eq 2 ] ; then
  #if [ "$z" = "2" ] ; then
  echo "$0:$LINENO: Comparison works. z:$z"
  else
  echo "$0:$LINENO: Comparison does NOT work. z:$z"
  fi
  
  

Loops

Arrays and math

Debugging -

As we have seen, it is very easy to make a coding mistake - and unfortunately shells, including Bash, provide very limited support for finding syntax or runtime errors. Suggestions include:

• You can run Bash with several options - issue 'bash options yourScript.sh'
  • -n : runs your script through the shell preprocessor to check for syntax errors. It does not actually run the commands.
  • -x : forces the shell to print all commands prior to running
  • -v : similar but adds verbosity .
• These commands can be added to always run by :
  • issue the set option
  • add the commands to the sh-bang line: !#//bin/bash -x

Additional items to help with debugging... (I'll add to this as I learn more)

• The eclipse SDE supports Bash
• There are a number of tools - we will add links if we hear of any that are actually helpful!!

Script Design Guidelines

The Unix shell along with the pipe concept was meant to provide an efficient method for processing data. A pipe was specifically intended to avoid the following:

• cmd1 > ./tmpfile.out
• cmd2<./tmpfile.out >output.dat

The pipe avoids the inefficiencies with 1)tmp output 2:sequential execution : cmd1 | cmd2 > output.dat. Clearly, there are only processing benefits if cmd1 does in fact operate on a stream as opposed to a batch mode.

The following example illustrates aspects of the above point. The command line effectively cuts and pastes two blocks of text lines from the file file- lines 6-11more useful example deals with being able to print a block of text lines from a file (called file) to an output file (file1) using just head and tail . The first case breaks the operation into two sequential steps. The second approach is a nice example of how to use a list to control the flow of a stream.

Note: 'head -11' prints the first 11 lines of file and 'tail -6 prints the last 6 lines.

• head -11 file|tail -6 > file1; head -24 file| tail -6 >> file1
• (head -11 file | tail -6; head -24 file | tail -6) > file1

A second example illustrates the performance hit associated with reading a data file line by line versus a more optimized approach.. The script is passed a data file (text file with each line consisting of fields separated by a whitespace) , a start and stop delimeter, and the output file. The script searches each line for a substring starting /stopping with the delimeters, and outputs the lines that match. The slower approach (analyzeData1.sh) and the more efficient approach (analyzeData2.sh). We run the two implementations using the same input data file (ping1.out), running the script with the time invocation.

time ./analyzeData2.sh ./ping1.out ' ' ' ' outfile.out
./analyzeData2.sh:Thu Feb 9 20:08:33 EST 2017: parse: ./ping1.out Begin Del: , End: , output: outfile.out

real 0m0.038s
user 0m0.004s
sys 0m0.004s
jjm@jjm-VirtualBox:~/courses/cpsc424/Spring-2017/bashScripts/analyzeData$ time ./analyzeData1.sh ./ping1.out ' ' ' ' outfile.out
./analyzeData1.sh:Thu Feb 9 20:08:44 EST 2017: parse: ./ping1.out Begin Del: , End: , output: outfile.out

real 0m1.690s
user 0m0.064s
sys 0m0.192s

Several articles that provide some information on design and/or performance related to Bash scripts:

• http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_01_05.html
• http://unix.stackexchange.com/questions/67057/bash-script-optimization-of-processing-speed

Last update: 4/29/2018