Functions, Pipes, grep & Text

What is it?

Pipes send one command's output into the next (cat | grep | awk); functions name reusable blocks; grep/sed/awk filter and reshape text. Together they process logs and data at speed.

Why it matters

Linux is a text-processing machine. The pipe from grep to awk is how an admin turns a million-line log into a two-line answer in seconds.

Where you see it

`grep failed auth.log | awk '{print $NF}' | sort | uniq -c` counts failures by source; a function `log(){ echo "[$(date)] $*"; }` standardizes output.

What normal looks like

Short, composable pipelines and small functions; each stage does one thing and the data flows left to right.

What suspicious looks like

A giant one-liner no one can read, a regex that matches too much, or parsing structured data (JSON) with grep instead of a real parser (jq) — fragile results.

How analysts investigate

Build a pipeline one stage at a time, checking the output at each step, and reach for jq/csv tools when data is structured rather than forcing grep.

Common beginner mistakes

Parsing JSON/CSV with grep/awk instead of jq or a CSV-aware tool.
Writing an unreadable mega-pipeline instead of stages you can verify.

Text in, answer out

  grep 'Failed password' /var/log/auth.log \
    | awk '{print $(NF-3)}' \   (the source IP field)
    | sort | uniq -c | sort -rn | head
     512 203.0.113.66   ← the brute-force source, in one pipeline
  build it stage by stage; use jq for JSON, not grep

Pipelines compose grep/awk/sort/uniq into fast log answers; use structured tools for structured data.

Quick check

You need one field from a JSON API response. Best tool?

A quick self-check — it doesn't affect your XP or progress.