▶Prefer to watch? This short video runs this page's main commands on a real Ubuntu 24.04 machine, with on-screen captions and no voiceover. Click to play, or watch it on YouTube.
python3 [options] [script.py] [arguments...]
python3 --version
Python is a readable, general-purpose language that is very popular for automation, scripting, web services, data work and
tooling. For sysadmins it is the natural next step when a bash script grows too big: real data structures, proper error
handling, JSON and API support, and a huge library collection. This sheet covers Python 3 (3.10 and newer, with notes on
newer features). Python 2 is dead: always use python3. See the Bash sheet for shell scripting and the AI-Assisted
Shell Scripting guide for reviewing generated code safely.
Install, run and explore
python3 --version # check what you have
sudo apt install python3 python3-pip python3-venv # Debian/Ubuntu (dnf install python3 python3-pip on Fedora/RHEL family)
brew install python # macOS winget install Python.Python.3.12 Windows
python3 script.py arg1 arg2 # run a script
python3 -c 'print("hello")' # run a one-liner
python3 -m module # run a module as a script (for example: python3 -m http.server)
python3 -i script.py # run a script, then stay in the interactive prompt with its variables
python3 # start the REPL (interactive prompt). Exit with exit() or Ctrl+D
python3 -m pip install requests # always use "python3 -m pip" so it matches the interpreter you run
python3 -X dev script.py # development mode: extra warnings
python3 -B script.py # do not write .pyc files
python3 -u script.py # unbuffered output (good for logs and pipes)
python3 -W error script.py # turn warnings into errors
In the REPL: help(len) dir(obj) type(x) _ (the last result) import this (the Zen of Python)
Virtual environments and packages (do this for every project)
python3 -m venv .venv # create an isolated environment in .venv
source .venv/bin/activate # activate it (Windows: .venv\Scripts\activate)
deactivate # leave it
pip install requests # install into the active environment
pip install "requests>=2.31,<3" # version limits
pip install -r requirements.txt # install a list
pip freeze > requirements.txt # save exact versions
pip list pip list --outdated pip show requests
pip install --upgrade pip pip install -U package pip uninstall package
pip install -e . # editable install of your own project (uses pyproject.toml)
pip check # find broken dependencies
pipx install httpie # install a command line APPLICATION in its own hidden environment (sudo apt install pipx)
uv venv && uv pip install requests # uv is a very fast modern alternative to pip and venv (astral.sh/uv)
Never run "sudo pip install". On Debian 12, Ubuntu 23.04 and newer, pip refuses system-wide installs:
"error: externally-managed-environment" This protects your operating system's Python. Use a venv, pipx, or the apt package (python3-NAME)
Add .venv/ to .gitignore. Do not copy a venv to another machine: recreate it from requirements.txt.
Basics
name = "Alice" # no type declaration, no semicolon. Indentation (4 spaces) defines blocks
age, city = 30, "Paris" # multiple assignment a, b = b, a swap
x = y = 0 # same value to both
PI = 3.14159 # constants are just a naming habit (UPPER_CASE)
print("Hello", name, sep=", ", end="!\n") # print to the screen
type(x) isinstance(x, int) # what is it?
# a comment """a docstring: documentation for a module, function or class"""
Types: int, float, str, bool (True, False), None, list, tuple, dict, set, bytes, complex
Truthiness: these are False: False, None, 0, 0.0, "", [], (), {}, set(). Everything else is True.
Operators: + - * / (float divide) // (floor divide) % (remainder) ** (power) == != < > <= >= and or not in not in is is not
10 / 4 = 2.5 10 // 4 = 2 10 % 4 = 2 2 ** 8 = 256 1_000_000 (underscores for readability) 0xFF 0o17 0b101
Compare with None using "is": if x is None: Chained comparisons work: 0 < x < 10
Conversion: int("42") float("3.5") str(42) bool(0) list("abc") tuple([1,2]) set([1,1,2]) int("ff", 16) round(3.14159, 2) abs(-5) divmod(17, 5)
Strings
s = "Hello, World"
len(s) s[0] s[-1] s[0:5] s[7:] s[::-1] # index, last, slice, from 7, reversed. Strings are immutable
f"{name} is {age} years old" # f-string: put expressions in braces
f"{price:.2f} {n:05d} {n:,} {ratio:.1%} {name:>10} {name:<10} {name:^10}" # formatting: decimals, zero pad, thousands, percent, align
f"{x = }" # debug form: prints x = 5
s.upper() s.lower() s.title() s.capitalize() s.swapcase()
s.strip() s.lstrip() s.rstrip() s.strip("-") # trim whitespace or given characters
s.split(",") s.split() s.splitlines() ",".join(["a", "b"]) # split and join
s.replace("l", "L") s.startswith("He") s.endswith("ld") s.find("o") s.index("o") s.count("l")
"World" in s # substring test
s.isdigit() s.isalpha() s.isalnum() s.isspace()
s.zfill(5) s.center(20, "*") s.ljust(10) s.rjust(10)
"a\tb\n" r"raw \n string (backslashes stay)" b"bytes" """multi-line
string"""
text.encode("utf-8") data.decode("utf-8") # strings <-> bytes
"%s is %d" % (name, age) "{} is {}".format(name, age) # older styles (use f-strings)
Remove a prefix or suffix (3.9+): s.removeprefix("Hello, ") s.removesuffix(".txt")
Lists, tuples, sets and dicts
nums = [3, 1, 4, 1, 5] # list: ordered, changeable
nums.append(9) nums.extend([2, 6]) nums.insert(0, 7) nums.remove(1) nums.pop() nums.pop(0) del nums[0]
nums.sort() nums.sort(reverse=True) nums.reverse() nums.copy() nums.clear() nums.index(4) nums.count(1)
sorted(nums) reversed(nums) len(nums) min(nums) max(nums) sum(nums) any(nums) all(nums)
nums[1:3] nums[::2] nums[-1] nums + [1] nums * 2 4 in nums
t = (1, 2, 3) # tuple: ordered, NOT changeable. a, b, c = t unpacking first, *rest = t
s = {1, 2, 3} # set: unique, unordered. s.add(4) s.discard(2) a | b a & b a - b a ^ b (union, intersection, difference, symmetric)
d = {"name": "Alice", "age": 30} # dict: key -> value (keeps insertion order)
d["name"] d.get("email") d.get("email", "n/a") # get() avoids a KeyError
d["city"] = "Paris" d.update({"a": 1}) d.pop("age") del d["name"] d.setdefault("k", [])
d.keys() d.values() d.items() "name" in d len(d)
for k, v in d.items(): print(k, v)
merged = d1 | d2 # merge dicts (3.9+) {**d1, **d2} also works
list(d) dict(zip(keys, values)) dict.fromkeys(["a", "b"], 0)
Copy: shallow b = a.copy() or list(a); deep import copy; copy.deepcopy(a) "b = a" does NOT copy (both names point to one object)
Comprehensions and generators
squares = [x**2 for x in range(10)] # list comprehension
evens = [x for x in nums if x % 2 == 0] # with a filter
pairs = [(a, b) for a in "ab" for b in (1, 2)] # nested loops
names = {u["name"]: u for u in users} # dict comprehension
unique = {x.lower() for x in words} # set comprehension
total = sum(x * x for x in range(1000)) # generator expression: lazy, uses little memory
def count_up(n): # a generator function: yields one value at a time
for i in range(n):
yield i
Use a comprehension for simple transforms. Use a normal loop when it gets hard to read.
Control flow
if x > 10:
print("big")
elif x > 5:
print("medium")
else:
print("small")
label = "adult" if age >= 18 else "minor" # conditional expression
for i in range(5): # 0..4 range(2, 10) range(0, 10, 2)
print(i)
for i, item in enumerate(items, start=1): ... # index and value
for a, b in zip(list1, list2): ... # walk two lists together
for key in d: ... # dict keys
while count < 5:
count += 1
if count == 3: continue # skip to the next round
if count == 4: break # leave the loop
else:
print("loop finished without break") # runs only if no break happened
match command: # structural pattern matching (3.10+)
case "start": ...
case "stop" | "quit": ...
case ["go", direction]: ...
case {"type": "error", "code": code}: ...
case _: print("unknown")
if (n := len(items)) > 10: print(n) # walrus operator: assign inside an expression
pass # do nothing (placeholder)
Functions
def greet(name, greeting="Hello"): # default value
"""Return a greeting."""
return f"{greeting}, {name}!"
greet("Alice") greet("Bob", greeting="Hi") # positional and keyword arguments
def total(*numbers, **options): # *args = tuple of extras, **kwargs = dict of named extras
return sum(numbers)
total(1, 2, 3) total(*[1, 2, 3]) func(**{"a": 1}) # unpack when calling
def area(w: float, h: float = 1.0) -> float: # type hints (not enforced at run time, used by editors and mypy)
return w * h
def only_keywords(*, a, b): ... # arguments after * must be named
def positional_only(a, b, /): ... # arguments before / must be positional
square = lambda x: x * x # a small anonymous function
sorted(items, key=lambda i: i["age"])
Do NOT use a mutable default: def f(x, items=[]) (the list is shared between calls). Use items=None then if items is None: items = []
Scope: variables assigned in a function are local. Use "global" or "nonlocal" only when you must.
Decorators wrap a function:
import functools, time
def timed(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
print(f"{func.__name__} took {time.perf_counter() - start:.3f}s")
return result
return wrapper
@timed
def work(): ...
@functools.lru_cache(maxsize=None) # cache results of a pure function
def fib(n): return n if n < 2 else fib(n - 1) + fib(n - 2)
Classes and dataclasses
class Server:
count = 0 # class attribute (shared)
def __init__(self, name, port=22): # the constructor
self.name = name # instance attributes
self.port = port
def __repr__(self): # how it prints in the REPL and logs
return f"Server({self.name!r}, {self.port})"
def url(self):
return f"ssh://{self.name}:{self.port}"
@property
def is_default(self): return self.port == 22
@classmethod
def from_string(cls, text): host, port = text.split(":"); return cls(host, int(port))
@staticmethod
def valid_port(p): return 0 < p < 65536
class Web(Server): # inheritance
def __init__(self, name): super().__init__(name, 80)
from dataclasses import dataclass, field
@dataclass
class Host: # generates __init__, __repr__ and __eq__ for you
name: str
port: int = 22
tags: list[str] = field(default_factory=list)
@dataclass(frozen=True) # immutable
class Point: x: int; y: int
Other useful dunder methods: __str__, __len__, __eq__, __lt__, __iter__, __getitem__, __enter__ / __exit__, __call__.
Errors and exceptions
try:
value = int(text)
except ValueError as e: # catch specific errors
print(f"bad number: {e}")
except (KeyError, IndexError):
...
else:
print("no error happened") # runs only if nothing was raised
finally:
cleanup() # always runs
raise ValueError("port must be 1-65535") # raise your own
raise RuntimeError("failed") from original_error # keep the original cause
class ConfigError(Exception): pass # your own exception type
with suppress(FileNotFoundError): os.remove(path) # from contextlib import suppress
Never write a bare "except:" (it hides Ctrl+C and real bugs). Catch the specific errors you can handle.
Common built-in errors: ValueError, TypeError, KeyError, IndexError, AttributeError, NameError, FileNotFoundError, PermissionError, OSError, ImportError, ZeroDivisionError, TimeoutError
assert x > 0, "x must be positive" # for internal checks and tests. Do not use for input validation (python -O removes asserts)
import traceback; traceback.print_exc() # print the full error trace
Files and paths
with open("file.txt") as f: # "with" closes the file for you, even on errors
text = f.read() # whole file f.readline() one line
for line in f: # line by line (memory friendly)
print(line.rstrip())
open("out.txt", "w") "a" (append) "r+" (read and write) "rb" / "wb" (binary) open(p, encoding="utf-8") # be explicit about encoding
with open("out.txt", "w", encoding="utf-8") as f:
f.write("line\n"); f.writelines(lines)
from pathlib import Path # the modern way to handle paths
p = Path("/var/log") / "syslog" # join with /
p.exists() p.is_file() p.is_dir() p.name p.stem p.suffix p.parent p.resolve() p.stat().st_size
p.read_text() p.write_text("data") p.read_bytes()
list(Path(".").glob("*.py")) list(Path(".").rglob("*.py")) # rglob = recursive
Path("a/b/c").mkdir(parents=True, exist_ok=True) p.unlink() p.rename("new") Path.home() Path.cwd()
import shutil
shutil.copy("a", "b") shutil.copy2("a", "b") shutil.copytree("src", "dst") shutil.move("a", "b") shutil.rmtree("dir") shutil.which("git")
shutil.disk_usage("/") # (total, used, free) in bytes
shutil.make_archive("backup", "gztar", "folder") # create a tar.gz
import os
os.getcwd() os.listdir(".") os.walk("dir") os.environ.get("HOME") os.getenv("DEBUG", "0") os.chmod(p, 0o644) os.getpid()
import tempfile
with tempfile.TemporaryDirectory() as tmp: ... # a temp folder that is deleted afterward
tempfile.NamedTemporaryFile("w", delete=False)
Write files safely: write to a temp file, then replace the real one: Path("f.tmp").write_text(data); Path("f.tmp").replace("f")
JSON, CSV and config files
import json
data = json.loads('{"a": 1}') text = json.dumps(data, indent=2, sort_keys=True) # strings
with open("x.json") as f: data = json.load(f) # files
with open("x.json", "w") as f: json.dump(data, f, indent=2)
import csv
with open("x.csv", newline="") as f:
for row in csv.DictReader(f): print(row["name"]) # each row is a dict keyed by the header
with open("out.csv", "w", newline="") as f:
w = csv.DictWriter(f, fieldnames=["name", "age"]); w.writeheader(); w.writerow({"name": "A", "age": 1})
import tomllib # read TOML (3.11+)
with open("pyproject.toml", "rb") as f: cfg = tomllib.load(f)
import configparser # INI files
cp = configparser.ConfigParser(); cp.read("app.ini"); cp["section"]["key"]
YAML needs a package: pip install pyyaml then import yaml; yaml.safe_load(text) # always safe_load, never yaml.load on untrusted input
Command line: python3 -m json.tool file.json (pretty-print and validate JSON)
Regular expressions
import re
re.search(r"\d+", text) # first match anywhere (or None). Use raw strings r"..."
re.match(r"\d+", text) re.fullmatch(r"\d+", text) # match at the start / match the whole string
re.findall(r"\b\w+@\w+\.\w+\b", text) # list of all matches
re.finditer(pattern, text) # iterate over match objects
re.sub(r"\s+", " ", text) # replace
re.split(r"[,;]\s*", text)
m = re.search(r"(?P<user>\w+)@(?P<host>[\w.]+)", text)
if m: m.group(0) m.group("user") m.groups() m.start() m.end()
pat = re.compile(r"^ERROR (\d+)", re.MULTILINE | re.IGNORECASE) # compile once and reuse; flags: I, M, S (dot matches newline), X (verbose)
Patterns: . \d \w \s (digit, word, space) \D \W \S (not) ^ $ * + ? {2,4} (a|b) [abc] [^abc] (?:non-capturing) *? +? (lazy) (?=ahead) (?<=behind)
Dates and times
from datetime import datetime, date, timedelta, timezone
now = datetime.now() utc = datetime.now(timezone.utc) today = date.today()
now.strftime("%Y-%m-%d %H:%M:%S") # format: %Y year %m month %d day %H hour %M minute %S second %A weekday %b month name %j day of year
datetime.strptime("2026-09-26", "%Y-%m-%d") # parse
datetime.fromisoformat("2026-09-26T10:30:00") now.isoformat()
tomorrow = now + timedelta(days=1, hours=2) (later - now).total_seconds() (later - now).days
datetime.fromtimestamp(1700000000) now.timestamp() # to and from Unix time
from zoneinfo import ZoneInfo # time zones (3.9+)
datetime.now(ZoneInfo("Europe/Paris"))
import time
time.time() time.sleep(2) time.perf_counter() # Unix time, pause, precise timer
Store and compare times in UTC. Convert to local time only when displaying.
Running commands: subprocess
import subprocess
r = subprocess.run(["ls", "-l", "/tmp"], capture_output=True, text=True, check=True)
print(r.stdout) r.stderr r.returncode
subprocess.run(["systemctl", "restart", "nginx"], check=True) # check=True raises CalledProcessError on failure
subprocess.run(["grep", "-c", "error", "app.log"], capture_output=True, text=True, timeout=30)
subprocess.run("df -h | head -3", shell=True) # a shell string: ONLY with trusted, fixed text
out = subprocess.check_output(["hostname"], text=True).strip()
p = subprocess.Popen(["tail", "-f", "log"], stdout=subprocess.PIPE, text=True) # for streaming or long-running processes
for line in p.stdout: print(line, end="")
Always pass a LIST of arguments, not one string, and avoid shell=True. Never build a command by pasting user input into a string (command injection).
import shlex; shlex.quote(user_text) shlex.split("a 'b c'") # if you must deal with shell-style text
Add timeout= to anything that could hang, and handle subprocess.TimeoutExpired and CalledProcessError.
Command line arguments: argparse
import argparse
parser = argparse.ArgumentParser(description="Clean old files")
parser.add_argument("path", help="folder to scan") # positional
parser.add_argument("-d", "--days", type=int, default=30, help="age in days")
parser.add_argument("-n", "--dry-run", action="store_true") # a flag
parser.add_argument("--mode", choices=["fast", "safe"], default="safe")
parser.add_argument("files", nargs="*") # zero or more
args = parser.parse_args()
print(args.path, args.days, args.dry_run)
Run: python3 clean.py /var/tmp -d 7 --dry-run Help is automatic: python3 clean.py -h
Subcommands: sub = parser.add_subparsers(dest="cmd"); sub.add_parser("start")
Simple case: import sys; sys.argv (list of arguments; sys.argv[0] is the script). sys.exit(1) ends with a status. Write errors to sys.stderr.
Other options: click and typer (nicer command line tools, pip install).
Logging
import logging
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s")
log = logging.getLogger(__name__)
log.debug("detail") log.info("started") log.warning("careful") log.error("failed") log.exception("with traceback") # exception() inside an except block
logging.basicConfig(filename="app.log", level=logging.DEBUG) # log to a file
from logging.handlers import RotatingFileHandler # rotate files (or use logrotate)
Use logging instead of print in anything that runs unattended. Do not log passwords or tokens.
Environment variables and secrets
import os
db_url = os.environ["DATABASE_URL"] # KeyError if missing (good for required settings)
debug = os.getenv("DEBUG", "0") == "1" # optional, with a default
pip install python-dotenv then from dotenv import load_dotenv; load_dotenv() # load a .env file in development (keep it out of git)
import secrets, getpass
secrets.token_hex(16) secrets.token_urlsafe(32) secrets.choice(chars) # secure random values (not the random module)
getpass.getpass("Password: ") # read a password without echoing it
Never hardcode passwords, tokens or keys in code. Read them from the environment or a secrets manager.
HTTP and APIs
pip install requests
import requests
r = requests.get("https://api.example.com/items", params={"q": "x"}, headers={"Authorization": f"Bearer {token}"}, timeout=10)
r.raise_for_status() # raise an error for 4xx/5xx
data = r.json() r.status_code r.text r.headers
requests.post(url, json={"a": 1}, timeout=10) # send JSON data={...} for a form
s = requests.Session(); s.headers.update({"User-Agent": "mytool/1.0"}) # reuse connections and settings
Always set timeout=. Without it a request can hang forever.
No install needed (standard library):
from urllib.request import urlopen
with urlopen("https://example.com", timeout=10) as resp: html = resp.read().decode()
Async and modern alternatives: httpx, aiohttp. A quick web server for a folder: python3 -m http.server 8000
Useful standard library modules
collections Counter(words).most_common(3) defaultdict(list) deque(maxlen=10) namedtuple OrderedDict ChainMap
itertools chain, product, permutations, combinations, groupby, islice, cycle, count, batched (3.12)
functools lru_cache, partial, reduce, wraps, cache
operator itemgetter, attrgetter sorted(rows, key=itemgetter("age"))
pathlib, shutil, os, glob, tempfile, fnmatch files and paths
json, csv, tomllib, configparser, xml.etree.ElementTree, sqlite3 data formats and a built-in database
datetime, zoneinfo, time, calendar dates and times
re, string, textwrap, difflib text
subprocess, shlex, argparse, logging, sys, os scripts and tools
hashlib hashlib.sha256(data).hexdigest() hmac, secrets, base64, uuid (uuid.uuid4())
socket, ssl, http.client, urllib, smtplib, email, ipaddress, selectors networking (ipaddress.ip_network("10.0.0.0/24").hosts())
threading, multiprocessing, concurrent.futures, asyncio, queue, sched concurrency
gzip, zipfile, tarfile, bz2, lzma, shutil.make_archive compression
unittest, doctest, pdb, timeit, cProfile, tracemalloc testing, debugging and profiling
platform platform.platform(), platform.node() getpass, pwd, grp, signal, resource system information
sqlite3 a database in one file, no server: conn = sqlite3.connect("app.db"); conn.execute("CREATE TABLE ..."); conn.commit()
Databases (parameterized queries only)
import sqlite3
conn = sqlite3.connect("app.db")
conn.execute("INSERT INTO users (name) VALUES (?)", (name,)) # the ? placeholder protects against SQL injection
rows = conn.execute("SELECT id, name FROM users WHERE age > ?", (18,)).fetchall()
conn.commit(); conn.close()
Never build SQL with f-strings or +. MySQL/PostgreSQL: pip install pymysql or psycopg (%s placeholders). ORMs: SQLAlchemy.
Concurrency (pick the right tool)
threading Waiting on network or disk (I/O-bound). Threads share memory
multiprocessing Heavy CPU work. Uses separate processes (the GIL limits threads to one CPU at a time in standard Python)
asyncio Thousands of simultaneous network connections with async/await
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor, as_completed
with ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(fetch, urls)) # run many at once, keep the order
futures = [pool.submit(fetch, u) for u in urls]
for fut in as_completed(futures): print(fut.result())
import asyncio
async def main():
await asyncio.sleep(1)
results = await asyncio.gather(fetch(1), fetch(2)) # run coroutines together
asyncio.run(main())
Use a lock for shared data: lock = threading.Lock(); with lock: ...
Sorting, counting and grouping
sorted(users, key=lambda u: u["age"], reverse=True) sorted(words, key=str.lower) sorted(rows, key=lambda r: (r["team"], -r["score"]))
from collections import Counter, defaultdict
Counter(words).most_common(5) # top 5 words
groups = defaultdict(list)
for u in users: groups[u["team"]].append(u)
from itertools import groupby # the data must be sorted by the key first
max(items, key=len) min(d, key=d.get) sum(1 for x in xs if x.ok)
Type hints and checking
def parse(line: str) -> dict[str, int]: ... # built-in generics (3.9+)
x: int | None = None # union syntax (3.10+) older: from typing import Optional, Union
from typing import Any, Callable, Iterable, Literal, TypedDict, Protocol
pip install mypy mypy script.py # static type check pyright is a fast alternative
Type hints are optional and not enforced at run time, but they make big scripts far easier to maintain.
Testing
pip install pytest
# test_math.py
def add(a, b): return a + b
def test_add(): assert add(2, 3) == 5
import pytest
def test_error():
with pytest.raises(ZeroDivisionError): 1 / 0
@pytest.mark.parametrize("a,b,expected", [(1, 2, 3), (0, 0, 0)])
def test_many(a, b, expected): assert add(a, b) == expected
pytest pytest -v pytest -x (stop at first failure) pytest -k "add" pytest tests/test_math.py::test_add pytest --cov=mypkg
Fixtures give tests set-up data: @pytest.fixture def tmp_config(tmp_path): ... (tmp_path is a built-in temp directory)
Built in alternatives: python3 -m unittest python3 -m doctest file.py
Debugging and profiling
breakpoint() # drop into the debugger at this line (pdb)
python3 -m pdb script.py # start under the debugger
pdb commands: n (next) s (step into) c (continue) l (list code) p expr (print) pp expr b 42 (breakpoint at line 42) w (stack) q (quit)
print(f"{var = }") # quick debugging import pprint; pprint.pprint(data)
python3 -X dev script.py # extra checks python3 -W error warnings become errors
python3 -m timeit "sum(range(1000))" # time a snippet
python3 -m cProfile -s cumtime script.py | head -20 # where does the time go?
import tracemalloc # find memory hogs
Read tracebacks from the BOTTOM: the last line is the error, and the frames above it show how it was reached.
Code quality tools
pip install ruff black mypy pytest
ruff check . ruff check --fix . ruff format . # very fast linter and formatter (replaces flake8, isort and more)
black . # a popular auto-formatter
mypy . # type checker
pre-commit install # run these automatically before each git commit
Follow PEP 8 (4-space indents, snake_case names, CapWords for classes, 79 to 100 character lines). A formatter settles the rest.
Packaging your own project (pyproject.toml)
my_tool/
pyproject.toml
src/my_tool/__init__.py
src/my_tool/cli.py
tests/test_cli.py
# pyproject.toml
[project]
name = "my-tool"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["requests>=2.31"]
[project.scripts]
my-tool = "my_tool.cli:main" # creates a "my-tool" command
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
pip install -e . # develop with live edits
python3 -m build # build a wheel (pip install build)
Run a package's module: python3 -m my_tool.cli Include a main guard in every script (see the skeleton below).
A solid script skeleton
#!/usr/bin/env python3
"""Delete old files in a folder. Use --dry-run first."""
import argparse
import logging
import sys
import time
from pathlib import Path
log = logging.getLogger(__name__)
def parse_args(argv=None):
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("folder", type=Path)
p.add_argument("-d", "--days", type=int, default=30)
p.add_argument("-n", "--dry-run", action="store_true")
p.add_argument("-v", "--verbose", action="store_true")
return p.parse_args(argv)
def old_files(folder: Path, days: int):
cutoff = time.time() - days * 86400
for p in folder.iterdir():
if p.is_file() and p.stat().st_mtime < cutoff:
yield p
def main(argv=None) -> int:
args = parse_args(argv)
logging.basicConfig(level=logging.DEBUG if args.verbose else logging.INFO,
format="%(asctime)s %(levelname)s %(message)s")
if not args.folder.is_dir():
log.error("Not a folder: %s", args.folder)
return 2
for path in old_files(args.folder, args.days):
if args.dry_run:
log.info("Would delete %s", path)
else:
log.info("Deleting %s", path)
path.unlink()
return 0
if __name__ == "__main__": # runs only when executed, not when imported
sys.exit(main())
Make it runnable: chmod +x clean.py then ./clean.py /var/tmp -n The shebang #!/usr/bin/env python3 finds python3 on the PATH.
Sysadmin recipes
# Disk usage
import shutil; t, u, f = shutil.disk_usage("/"); print(f"{u / t:.0%} used")
# Find the 10 biggest files under a folder
from pathlib import Path
files = sorted((p for p in Path("/var").rglob("*") if p.is_file()), key=lambda p: p.stat().st_size, reverse=True)[:10]
# Count the most common IP addresses in a log
import re; from collections import Counter
ips = re.findall(r"\b\d{1,3}(?:\.\d{1,3}){3}\b", Path("access.log").read_text())
print(Counter(ips).most_common(10))
# Check if a port is open
import socket
with socket.socket() as s:
s.settimeout(2); print(s.connect_ex(("example.com", 443)) == 0)
# Send a simple email through an SMTP relay (see the Azure SMTP relay guide for the settings)
import smtplib; from email.message import EmailMessage
msg = EmailMessage(); msg["From"] = "noreply@example.com"; msg["To"] = "you@example.com"; msg["Subject"] = "Alert"; msg.set_content("Disk is 90% full")
with smtplib.SMTP("smtp.example.com", 587) as s: s.starttls(); s.login(user, password); s.send_message(msg)
# System stats (pip install psutil)
import psutil; psutil.cpu_percent(1); psutil.virtual_memory().percent; psutil.disk_usage("/").percent; [p.info for p in psutil.process_iter(["pid", "name"])]
# Run a command on many servers over SSH (pip install paramiko or fabric)
import paramiko
c = paramiko.SSHClient(); c.load_system_host_keys(); c.connect("host", username="me", key_filename="/home/me/.ssh/id_ed25519")
stdin, stdout, stderr = c.exec_command("uptime"); print(stdout.read().decode()); c.close()
# Work with IP networks
import ipaddress
net = ipaddress.ip_network("192.168.1.0/24"); list(net.hosts())[:3]; ipaddress.ip_address("192.168.1.5") in net
# Parse JSON from a command
import json, subprocess
data = json.loads(subprocess.run(["ip", "-j", "addr"], capture_output=True, text=True, check=True).stdout)
# Retry with backoff
import time
for attempt in range(5):
try: do_it(); break
except OSError: time.sleep(2 ** attempt)
# Watch a file for new lines (like tail -f)
with open("app.log") as f:
f.seek(0, 2)
while True:
line = f.readline()
print(line, end="") if line else time.sleep(0.5)
Scheduling your scripts
crontab -e */15 * * * * /usr/bin/python3 /opt/scripts/check.py >> /var/log/check.log 2>&1
Use full paths (cron has a tiny PATH), the virtualenv's interpreter (/opt/app/.venv/bin/python), and log the output.
A systemd timer gives better logging and control (see the systemctl sheet). Use flock to avoid overlapping runs.
Handy one-liners
python3 -m http.server 8000 # share the current folder on port 8000 (local testing only)
python3 -m json.tool data.json # pretty-print and validate JSON cat x.json | python3 -m json.tool
python3 -c "import sys, json; print(json.load(sys.stdin)['key'])" < x.json # pull one value out of JSON
python3 -m venv .venv && . .venv/bin/activate # new environment
python3 -c "import uuid; print(uuid.uuid4())" # a random ID
python3 -c "import secrets; print(secrets.token_urlsafe(32))" # a strong random token
python3 -c "import base64; print(base64.b64encode(b'hello').decode())"
python3 -c "import hashlib; print(hashlib.sha256(b'hello').hexdigest())"
python3 -c "import time; print(time.time())" # current Unix time
python3 -c "import platform; print(platform.platform())"
python3 -c "print(sum(range(1, 101)))"
python3 -m timeit "'-'.join(str(n) for n in range(100))"
python3 -m calendar 2026 # print a calendar
python3 -m zipfile -l archive.zip # list a zip python3 -m tarfile -l archive.tar.gz
python3 -m pip list --outdated
python3 -m compileall . # byte-compile everything (a quick syntax check)
python3 -m pyflakes script.py # a light linter (pip install pyflakes)
Common errors
ModuleNotFoundError: No module named 'x' Not installed in THIS environment. Activate your venv and pip install x. Check which python3 and pip
error: externally-managed-environment Debian/Ubuntu protect the system Python. Use a venv, pipx, or apt install python3-NAME
IndentationError / TabError Mixed tabs and spaces, or wrong indentation. Use 4 spaces everywhere. Configure your editor
SyntaxError: invalid syntax A missing colon, bracket or quote. The error line is often just AFTER the real mistake. Also check for Python 2 print "x"
NameError: name 'x' is not defined A typo, or used before assignment, or forgot an import
TypeError: 'NoneType' object is not subscriptable A function returned None (often it forgot to "return"), or you assigned the result of list.sort() (which returns None)
TypeError: can only concatenate str (not "int") Convert first: str(n), or use an f-string
KeyError: 'name' The dict has no such key. Use d.get("name") or check "name" in d
IndexError: list index out of range The index is past the end. Check len(), and remember indexes start at 0
AttributeError: 'X' object has no attribute 'y' Wrong type, or a typo. Check with type(x) and dir(x)
ValueError: invalid literal for int() The text is not a number. Validate first or catch ValueError
UnicodeDecodeError Wrong encoding. Open with encoding="utf-8" (or errors="replace"), or read in binary mode
FileNotFoundError / PermissionError Wrong path or working directory (print Path.cwd()), or missing permissions
RecursionError A function calls itself forever. Add a base case, or use a loop
ImportError: cannot import name / circular import Two modules import each other. Move shared code to a third module
"python: command not found" Use python3 (or install the python-is-python3 package)
Script edits are ignored You are running an old copy, another environment, or a stale .pyc. Check python3 -c "import mod; print(mod.__file__)"
Changes to a list inside a function surprise you Lists and dicts are passed by reference. Copy first if you must keep the original
The same list appears in every object A mutable default argument, or a class-level list. Create it in __init__ or use default_factory
Slow loops Use built-ins, comprehensions, sets/dicts for lookups, generators for big data, or a profiler to find the hot spot
Security habits
- Never use eval() or exec() on data you did not write, and never pickle.load() untrusted files (both can run arbitrary code)
- Pass argument LISTS to subprocess, avoid shell=True, and never build SQL, shell or file paths by pasting user input
- Use the secrets module (not random) for tokens and passwords, and hashlib.scrypt or a library like argon2/bcrypt to hash passwords
- Load secrets from the environment or a secrets manager, never from source code or git
- Use yaml.safe_load, defusedxml for untrusted XML, and validate input (types, ranges, allowed characters)
- Keep dependencies pinned and updated. Check them with pip-audit. Install only packages you trust and review names carefully (typosquatting is real)
- Run scripts with the least privilege they need, and set timeouts on network calls and subprocesses
- Do not log secrets. Do not commit .env files
Tips
Use python3 and a virtual environment for every project. Never run sudo pip install.
Use f-strings, pathlib, with-statements and dataclasses. They make code shorter and safer.
Always pass timeout= to network calls and subprocess.run, and check=True (or handle the return code) for commands.
Catch specific exceptions, log with logging.exception, and let unexpected errors fail loudly instead of hiding them.
Put a main() function and an "if __name__ == '__main__':" guard in every script, so it can also be imported and tested.
Use argparse (or click and typer) for options, and add --dry-run to anything that changes or deletes things.
Read tracebacks from the bottom up, and use breakpoint() when print statements are not enough.
Format with ruff or black, check types with mypy, and write a few pytest tests for anything you rely on.
Reach for Python when a bash script needs real data structures, JSON, APIs or error handling. Keep bash for gluing simple commands together.
Never trust user input, and never eval it. Treat files, environment variables and network data as untrusted until you validate them.
