Linux File Viewing & Text Stream Manipulation

Inspecting, slicing, filtering, and manipulating text streams are essential daily tasks in Linux environments. This guide covers file viewers (cat, tac, nl, less, head, tail) and text transformation utilities (cut, paste, tr, sort, uniq, wc, tee).

⚡ Quick Dive

Viewing & Stream Processing Cheat Sheet

Quick Start Pipeline Examples

# Count top 5 most frequent IP addresses in an access log
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -n 5

# View compressed logs without unzipping to disk
zless /var/log/nginx/access.log.2.gz

📖 Extended Guide

1. Sequential File Viewers: cat, tac, and nl

cat (Concatenate & Print)

• -n: Number all output lines.
• -b: Number non-empty output lines only.
• -s: Squeeze multiple adjacent blank lines into a single blank line.
• -A (-vET): Show all non-printable characters (tabs as ^I, end of line as $).

# Squeeze blank lines and show non-printable characters
cat -s -A input.txt

# Merge multiple files into a new file
cat header.txt body.txt footer.txt > document.txt

tac (Reverse cat)

Reads lines from end to beginning:

# View newest log entries at the top of output
tac /var/log/auth.log | head -n 15

nl (Number Lines)

Advanced line numbering with formatting styles:

# Number only lines matching body format
nl -b a script.py

2. Interactive Pagers: less and more

less is the standard terminal pager. It loads files on demand without reading the whole file into RAM, making it fast even for multi-gigabyte files.

Key Command-Line Flags

• -N: Display line numbers on each row.
• -S: Truncate long lines rather than wrapping (use Left/Right arrows to scroll horizontally).
• -i: Case-insensitive search unless search pattern contains uppercase characters.
• +F: Enter live follow mode (press Ctrl+C to return to interactive navigation).

Interactive Keybindings Cheat Sheet

3. Slicing Content: head and tail

head

• -n <N>: Print first N lines (default 10).
• -n -<N>: Print all lines except the last N lines.
• -c <N>: Print first N bytes.

# Print all lines except the last 5
head -n -5 dataset.csv

tail

• -n <N>: Print last N lines.
• -n +<N>: Print starting from line N to the end of the file.
• -f: Follow changes by file descriptor.
• -F: Follow changes by filename (essential for production logs managed by logrotate).

# Start reading from line 100 onwards
tail -n +100 dump.sql

# Monitor multiple log files simultaneously
tail -f /var/log/nginx/access.log /var/log/nginx/error.log

4. Text Extraction & Slicing: cut and paste

cut (Column / Field Slicing)

• -d '<delimiter>': Set field delimiter (default is TAB).
• -f <list>: Select fields (e.g., -f 1,3 or -f 2-4).
• -c <range>: Select specific byte/character positions.

# Extract usernames and login shells from /etc/passwd
cut -d: -f1,7 /etc/passwd

# Extract columns 1 through 10 of fixed-width records
cut -c1-10 records.txt

paste (Merge Lines Side-by-Side)

Merges lines of files side-by-side using delimiters:

# Combine two lists side by side separated by a comma
paste -d, names.txt emails.txt

5. Stream Transformation: tr, sort, and uniq

tr (Translate / Delete Characters)

Operates directly on standard input:

• tr 'a-z' 'A-Z': Convert lowercase to uppercase.
• -d '<chars>': Delete specified characters.
• -s '<chars>': Squeeze repeated consecutive characters into one.

# Convert DOS CRLF line endings to UNIX LF
tr -d '\r' < dos_file.txt > unix_file.txt

# Replace colons with tabs
cat /etc/passwd | tr ':' '\t'

sort (Ordering Text Data)

• -n: Numerical sort (treats 10 as greater than 2).
• -r: Reverse order.
• -k <pos>: Sort based on a specific key/column position.
• -t '<delim>': Set delimiter for column keys.
• -h: Human-numeric sort (understands 2K, 5M, 1G).

# Sort disk usage output by size in descending order
du -h --max-depth=1 | sort -hr

uniq (Deduplicating Adjacent Lines)

[!IMPORTANT] uniq only detects adjacent duplicate lines. Always run sort before uniq unless lines are already ordered.

• -c: Prefix lines with the number of occurrences.
• -d: Only print duplicate lines.
• -u: Only print unique lines.

# Find unique failed login attempts and frequency
grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -nr

6. Pipeline Splitting with tee

The tee command duplicates standard input to both standard output and one or more files.

# Write to a root-owned file in a non-root pipeline
echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.d/99-custom.conf

# Log build output to disk while watching it live in console
make build 2>&1 | tee build.log

7. Formatted Terminal Output: echo vs printf

# echo with escape interpretation (-e)
echo -e "Status:\t\e[32mOK\e[0m"

# printf (portable and predictable across all POSIX shells)
printf "%-15s: %s\n" "Host" "srv01.prod"
printf "%-15s: %d MB\n" "Memory" 2048