Linux Command Line & Shell Scripting

Linux Command Line & Shell Scripting

The command line is the most powerful interface to a computer. Developers who master the terminal work faster, automate repetitive tasks, debug more effectively, and understand their systems more deeply. Almost every server in the world runs Linux; knowing the shell is a professional requirement.

Navigation and File Operations

pwd              # print working directory
ls -la           # list files (l=long format, a=include hidden)
cd /var/log      # change directory (absolute path)
cd ..            # go up one level
cd ~             # go to home directory

mkdir -p /var/app/data     # create directory tree (-p: no error if exists)
cp -r src/ dest/           # copy recursively
mv old_name new_name       # move or rename
rm -rf directory/          # delete recursively (CAUTION: no undo)
ln -s /etc/nginx/nginx.conf nginx.conf  # symbolic link

File Viewing and Text Processing

cat file.txt              # print entire file
head -n 20 file.txt       # first 20 lines
tail -n 100 file.txt      # last 100 lines
tail -f /var/log/app.log  # follow (real-time new lines — invaluable for logs)

grep -r "ERROR" /var/log/  # recursive grep
grep -n "function" src/*.ts  # show line numbers
grep -E "^[0-9]{4}" file     # extended regex

awk '{print $1, $3}' file    # print columns 1 and 3
sed 's/old/new/g' file.txt   # substitute globally
sort | uniq -c | sort -rn    # count occurrences, sort by frequency
wc -l file.txt               # count lines

Pipelines: The Unix Way

The Unix philosophy: programs that do one thing well, composable through pipes. Each command reads stdin, processes, writes stdout. Pipelines are the composition operator.

# Count unique IP addresses in an access log
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -20

# Find the 10 largest files in a directory
du -sh * | sort -rh | head -10

# Extract all email addresses from a file
grep -oE '[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+.[a-zA-Z]{2,}' file.txt | sort -u

# Watch a process's CPU usage
watch -n 1 'ps aux | grep node | grep -v grep'

Process Management

ps aux               # list all processes
top / htop           # interactive process monitor
kill -9 PID          # SIGKILL (force kill)
kill -15 PID         # SIGTERM (graceful shutdown)
killall nginx        # kill all processes named nginx

# Background and foreground
command &            # run in background
jobs                 # list background jobs
fg %1                # bring job 1 to foreground
nohup command &      # run after logout (ignore SIGHUP)

# System resources
free -h              # memory usage
df -h                # disk usage
iostat 1             # I/O statistics every 1 second
netstat -tulpn       # listening ports and their processes
ss -s                # socket statistics

File Permissions

Unix permissions: 9 bits for owner (rwx), group (rwx), and others (rwx).

-rwxr-xr-- 1 deploy www-data 4096 Oct  1 09:00 deploy.sh
 |||||||||||
 |rwx       owner: read, write, execute
 |   r-x    group: read, no write, execute
 |      r-- others: read only

chmod 755 script.sh       # rwxr-xr-x (octal)
chmod +x script.sh        # add execute for all
chmod -w file.txt         # remove write for all
chown user:group file.txt # change owner and group
umask 022                 # default permissions mask (new files = 644)

Shell Scripting

#!/bin/bash
# Deployment script example

set -e           # exit on any error
set -u           # error on undefined variables
set -o pipefail  # pipeline fails if any command fails

APP_DIR="/var/app"
DEPLOY_USER="deploy"

echo "Deploying to $APP_DIR..."

# Variables and command substitution
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="$APP_DIR/backups/$DATE"

# Conditionals
if [ -d "$BACKUP_DIR" ]; then
  echo "Backup directory exists"
else
  mkdir -p "$BACKUP_DIR"
fi

# Loops
for file in *.js; do
  echo "Processing: $file"
  node "$file"
done

# Functions
check_dependency() {
  if ! command -v "$1" &> /dev/null; then
    echo "Error: $1 is not installed"
    exit 1
  fi
}

check_dependency node
check_dependency git

echo "Deploy complete"

Environment Variables and Configuration

export DATABASE_URL="postgres://localhost/mydb"  # set in current and child processes
echo $DATABASE_URL                               # access variable
unset DATABASE_URL                               # remove

env                  # list all environment variables
printenv HOME        # print specific variable

# .env file (common pattern for apps)
# source .env         # load into current shell
# env $(cat .env | xargs) command   # pass to one command

SSH and Remote Access

ssh user@host                    # connect to remote host
ssh -i ~/.ssh/key.pem user@host  # connect with specific key
ssh -L 5432:localhost:5432 user@host  # port forward (tunnel local 5432 to remote 5432)

scp local_file user@host:/remote/path   # copy to remote
rsync -avz ./src/ user@host:/app/src/  # sync directories efficiently

# SSH config (~/.ssh/config)
Host production
  HostName prod.example.com
  User deploy
  IdentityFile ~/.ssh/prod_key

Cron and Scheduled Tasks

crontab -e        # edit current user's crontab
crontab -l        # list current crontab

# Crontab syntax: minute hour day month weekday command
# 0 2 * * *   run at 2:00am daily
# */5 * * * * run every 5 minutes
# 0 9 * * 1   run at 9am on Mondays
# 0 0 1 * *   run at midnight on the 1st of each month

0 2 * * * /scripts/backup.sh >> /var/log/backup.log 2>&1
*/5 * * * * /scripts/health_check.sh