bash Shell Examples

August 15, 2020 brezeale

I originally created this on June 23, 2004. These examples primarily use the for loop in the bash shell.

Count the number of lines with the_word in each .txt file

for file in *.txt; do 
	grep the_word $file | wc -l
done

This code creates directories (under the current directory, as written) for any groups whose names begin with dev-.

for GRP in `getent group | cut -d: -f1 | grep ^dev-`; do
		mkdir $GRP
done

Create directories named dev1, dev2, …, dev10

for (( num=1; num <= 10; num++ )); do
		mkdir "dev$num"
done

for num in $(seq 1 10); do
		mkdir dev$num
done

If you need to have a special format, the Unix command seq supports some of the printf standards. For instance, to convert the image files named frame001.jpg, frame002.jpg,…, frame999.jpg to GIFs, I can use

for num in $(seq -f %03g 1 999); do
		convert frame$num.jpg frame$num.gif
done

Change all files with the extension out to lowercase

for file in *.out; do
	mv $file `echo $file | tr '[A-Z]' '[a-z]'`
done

Change filenames with the extension chm that consist of words separated by periods (e.g., this.is.a.long.name.chm) to use underscores instead except for the period that typically exists between the prefix and file extension. Here sed’s ‘-e’ option is used to combine expressions.

for file in *.chm; do 
	mv $file `echo $file | sed -e 's/\./_/g' -e 's/_chm/.chm/'`
done

Find all directories which have files within 2 levels of the current location that have changed in the past day. Use sed to get just the directory name and tar it with the current date in the tar’ed filename.

SRC="/windows_data/data"
DEST="/windows_data/data_backups"
DATE=`date '+%y%m%d'`

cd $SRC

for file in `find . -maxdepth 2 -mtime -1 | sed 's/\.\///' | sed 's/\(.*\)\/.*/\1/' | sort|uniq`
do
  if [ ! -e "$file" ]       # Check if file exists.
  then
    echo "$file does not exist."; echo
    continue                # On to next.
  fi
  if [ ! "$file" == "." ]
  then
    tar -czf $DEST/$file$DATE.tgz $file 
  fi
done

Count the number of lines in each file with the extension dct and divide it by 480.

for file in *.dct; do
	LC=`cat $file | wc -l`
	echo $file
	COUNT=$(echo "scale=3; $LC / 480" | bc)
	echo "  lines = " $LC", ratio = " $COUNT
done

For each line in test.txt, count the number of words.

while read line; do  
	echo $line | wc -w; 
done < test.txt

extract the files for user1010 from the compressed files

for file in *sequence.csv.tgz; do 
	tar tvzf $file | grep user1010 |sed 's/.*\(user1010.*\)/\1/' > tempvar
	tar xzf $file `cat tempvar`
done

combine the contents of all user files (which have names of the form user1234-sequence.csv), with each user’s part preceded by the user name and separated by a blank line

for file in user*sequence.csv; do 
	prefix=`basename $file -sequence.csv`
	echo $prefix >> allusers.txt
	cat $file >> allusers.txt
	echo "" >> allusers.txt
done

rename files of the form test3-q01.c, test3-q02.c,… to test2-q01.c, test2-q02.c,…

for file in test3*.c; do 
    suf=`echo $file | sed 's/test3-//'`
    mv $file test2-$suf
done