Linux Commands for Intermediate Users: Processes, Pipes & Networking
Go beyond navigation — master pipes, text processing with sed and awk, process management, SSH, cron, and essential networking commands used daily in production environments.
Before you begin
- Comfortable with beginner Linux commands (ls, cd, mkdir, grep, find)
- Understand what a file and directory are
- A Linux terminal or SSH access to a remote server
Linux Commands for Intermediate Users: Processes, Pipes & Networking
The beginner commands get you moving around the filesystem. Intermediate Linux is where you start doing real engineering work — wiring commands together with pipes, processing structured text, watching and killing processes, managing scheduled tasks, and diagnosing network problems.
These are the commands you'll use daily when working with Kubernetes nodes, CI/CD pipelines, and cloud VMs.
1. Pipes and Redirection
The pipe | takes the output of one command and feeds it as input to the next. This is the core of shell composition.
1# Find all ERROR lines in a log, count them
2grep "ERROR" app.log | wc -l
3
4# Show the 10 most recently modified files
5ls -lt | head -10
6
7# Find processes listening on port 8080
8ss -tlnp | grep 8080Redirection operators
command > file.txt # Write stdout to file (overwrites)
command >> file.txt # Append stdout to file
command 2> errors.txt # Write stderr to file
command &> all.txt # Write both stdout and stderr to file
command < input.txt # Feed file as stdin to command
command | tee file.txt # Write to file AND still print to terminaltee is especially useful when you want to capture output while still watching it live:
./deploy.sh 2>&1 | tee deploy-$(date +%Y%m%d).log2. Text Processing: grep, sed, awk, cut, sort, uniq, wc
Most log analysis and config manipulation in production is done with these tools chained together.
Extended grep
grep -E "ERROR|WARN" app.log # Multiple patterns (extended regex)
grep -c "ERROR" app.log # Count matching lines
grep -A 3 "panic" app.log # 3 lines After each match (context)
grep -B 2 "timeout" app.log # 2 lines Before each match
grep -rn "DB_PASSWORD" ./ # Recursive search with line numberssed — stream editor
sed edits text line by line. The most common use is substitution:
sed 's/old/new/' file.txt # Replace first occurrence per line
sed 's/old/new/g' file.txt # Replace all occurrences (global)
sed -i 's/localhost/db.internal/g' config.yaml # Edit file in-place
sed '/^#/d' nginx.conf # Delete comment lines
sed -n '10,20p' app.log # Print only lines 10–20Tip: always test without
-ifirst.sed -imodifies the file directly.
awk — column-based processing
awk treats each line as a row of fields separated by whitespace (or a delimiter you specify). $1, $2, etc. refer to columns.
1# Print the 11th column of ps output (COMMAND)
2# ps aux columns: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
3ps aux | awk '{print $11}'
4
5# Print processes where RSS memory (column 6, in KB) is over 1GB
6ps aux | awk '$6 > 1000000 {print $1, $11, $6}'
7
8# Sum the sizes of all .log files
9ls -l *.log | awk '{sum += $5} END {print sum " bytes"}'
10
11# Use a custom delimiter
12awk -F: '{print $1}' /etc/passwd # Print all usernamescut, sort, uniq, wc
1cut -d',' -f2 data.csv # Extract column 2 from CSV
2cut -d':' -f1,3 /etc/passwd # Fields 1 and 3 (username, UID)
3
4sort file.txt # Sort alphabetically
5sort -n numbers.txt # Sort numerically
6sort -rn numbers.txt # Sort numerically, reverse
7
8sort access.log | uniq -c | sort -rn # Count occurrences, sort by frequency
9uniq -d file.txt # Show only duplicate lines
10
11wc -l app.log # Line count
12wc -w file.txt # Word count
13wc -c binary.bin # Byte countA practical example — find the top 10 most frequent IP addresses in an nginx access log:
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -103. Process Management
Viewing processes
ps aux # All running processes
ps aux | grep nginx # Find nginx processes
top # Live process monitor (q to quit)
htop # Better top (if installed)In ps aux output:
USER— who owns the processPID— process ID (use this to kill or inspect)%CPU,%MEM— resource usageCOMMAND— the actual command
Killing processes
kill 1234 # Send SIGTERM (graceful shutdown)
kill -9 1234 # Send SIGKILL (force kill, no cleanup)
pkill nginx # Kill by process name
pkill -f "python server.py" # Kill by full command string
killall node # Kill all processes named nodeAlways try kill (SIGTERM) before kill -9. SIGTERM lets the process clean up; SIGKILL does not.
Background and foreground jobs
./long-running-script.sh & # Run in background
jobs # List background jobs
fg %1 # Bring job 1 to foreground
bg %1 # Resume job 1 in background
Ctrl+Z # Suspend current foreground job
nohup ./script.sh & # Run and keep running after logoutnohup is essential when starting a process over SSH that needs to survive after you disconnect.
4. Disk and Memory
1df -h # Disk usage per filesystem (human-readable)
2df -h / # Disk usage of root filesystem only
3du -sh ./ # Total size of current directory
4du -sh */ | sort -rh | head -10 # Top 10 largest subdirectories
5du --max-depth=1 -h /var # Sizes of /var subdirectories
6
7free -h # Memory and swap usage
8vmstat 1 5 # System stats every 1 second, 5 timesCommon workflow: disk alert fires → df -h to confirm → du -sh */ to find the culprit directory → du --max-depth=1 to drill down.
5. Networking Basics
Connectivity
ping google.com # Test connectivity
ping -c 4 google.com # 4 packets then stop
curl https://api.example.com/health # HTTP request
curl -I https://example.com # Headers only
curl -o file.zip https://example.com/file.zip # Download to file
wget https://example.com/file.tar.gz # Download fileOpen ports and connections
ss -tlnp # TCP listening ports + process names
ss -s # Socket statistics summary
netstat -tlnp # Same (older systems)
lsof -i :8080 # What's using port 8080DNS lookups
dig codingprotocols.com # Full DNS query output
dig codingprotocols.com A # A record only
dig +short codingprotocols.com # Just the IP
nslookup codingprotocols.com # Simpler DNS lookup6. Archives
1# Create
2tar -czf archive.tar.gz dir/ # gzip-compressed archive
3tar -cjf archive.tar.bz2 dir/ # bzip2-compressed archive
4zip -r archive.zip dir/ # ZIP archive
5
6# Extract
7tar -xzf archive.tar.gz # Extract gzip archive
8tar -xzf archive.tar.gz -C /tmp/ # Extract to /tmp/
9unzip archive.zip
10
11# List contents without extracting
12tar -tzf archive.tar.gz
13unzip -l archive.zipMemory aid: tar -czf = Create, gZip, File. tar -xzf = eXtract.
7. SSH Essentials
ssh user@host # Connect to remote host
ssh -p 2222 user@host # Custom port
ssh -i ~/.ssh/id_rsa user@host # Specific key
ssh -L 5432:db.internal:5432 user@jump # Local port forwarding (tunnel)Key management
ssh-keygen -t ed25519 -C "ajeet@codingprotocols.com" # Generate key pair
ssh-copy-id user@host # Copy public key to remote authorized_keys~/.ssh/config — stop typing long commands
Host prod-bastion
HostName 203.0.113.10
User ec2-user
IdentityFile ~/.ssh/prod.pem
ServerAliveInterval 60
Host db-tunnel
HostName 203.0.113.10
User ec2-user
LocalForward 5432 db.internal:5432
After this you can just run ssh prod-bastion instead of the full command.
Copying files
scp file.txt user@host:/remote/path/ # Copy local → remote
scp user@host:/remote/file.txt ./ # Copy remote → local
rsync -avz ./src/ user@host:/app/src/ # Sync directories (incremental)
rsync -avz --delete ./dist/ user@host:/var/www/ # Mirror (delete extras)rsync is almost always better than scp — it only transfers changed files.
8. Environment Variables
export DB_HOST=localhost # Set variable for current session + child processes
echo $DB_HOST # Read a variable
env # List all environment variables
printenv PATH # Print a specific variable
unset DB_HOST # Remove a variable$PATH
$PATH is a colon-separated list of directories where the shell looks for executables.
echo $PATH
# /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
# Add a directory to PATH (add to ~/.bashrc to make permanent)
export PATH="$HOME/.local/bin:$PATH".bashrc vs .bash_profile
.bash_profile— runs on login shells (SSH sessions, terminal apps that start fresh).bashrc— runs on interactive non-login shells (new terminal tab, runningbash)
In practice: put your export statements and aliases in ~/.bashrc, then source it from ~/.bash_profile with source ~/.bashrc. Most distributions handle this for you.
source ~/.bashrc # Reload config without opening a new shell
alias k='kubectl' # Aliases go in ~/.bashrc9. Scheduled Tasks — cron
cron runs commands on a schedule. Edit your crontab with:
crontab -e # Edit your cron jobs
crontab -l # List current cron jobs
crontab -r # Remove all cron jobs (careful!)Cron syntax
┌─────────────── minute (0–59)
│ ┌───────────── hour (0–23)
│ │ ┌─────────── day of month (1–31)
│ │ │ ┌───────── month (1–12)
│ │ │ │ ┌─────── day of week (0=Sunday, 6=Saturday)
│ │ │ │ │
* * * * * command
# Every day at 2:30 AM
30 2 * * * /opt/scripts/backup.sh
# Every 5 minutes
*/5 * * * * /opt/scripts/healthcheck.sh
# Every Monday at 9 AM
0 9 * * 1 /opt/scripts/weekly-report.sh
# First day of every month at midnight
0 0 1 * * /opt/scripts/monthly-cleanup.sh
Always use absolute paths in cron jobs — cron runs with a minimal $PATH.
Putting It Together — A Real Workflow
Here's how these tools combine for a typical debugging session:
1# 1. Find processes consuming the most memory
2ps aux --sort=-%mem | head -10
3
4# 2. Check what ports the suspicious process has open
5ss -tlnp | grep <pid>
6
7# 3. Look at its recent log output
8tail -f /var/log/app/app.log | grep -E "ERROR|WARN"
9
10# 4. Count error rate over the last 1000 lines
11tail -1000 /var/log/app/app.log | grep -c "ERROR"
12
13# 5. Find which endpoints are erroring
14grep "ERROR" /var/log/app/app.log | awk '{print $7}' | sort | uniq -c | sort -rnWhat's Next
- Linux Commands for Advanced Engineers —
strace,lsof,tcpdump,systemdunits, cgroup primitives, and kernel parameter tuning - Linux Commands Beginner — Revisit the foundations if anything here felt unfamiliar
These tutorials are part of the Linux Foundations learning path, which maps to Stage 1 of the Platform Engineering Roadmap. Every stage in that roadmap — containers, Kubernetes, CI/CD — assumes this intermediate Linux knowledge.
Next in Linux Foundations
Linux Commands for Advanced Engineers: Debugging, systemd & Kernel Internals
We built Podscape to simplify Kubernetes workflows like this — logs, events, and cluster state in one interface, without switching tools.
Struggling with this in production?
We help teams fix these exact issues. Our engineers have deployed these patterns across production environments at scale.