subprocess

subprocess

Here’s a DevOps/SRE-focused guide for Python’s subprocess module — with real-world examples for each key method, ideal for scripting system tasks, command execution, and automation pipelines.

---

✅ subprocess — Python Module for Shell & Process Handling

First, import it:

import subprocess

---

🔹 subprocess.run() — Safely Run Shell Command

result = subprocess.run(["systemctl", "status", "nginx"], capture_output=True, text=True)
print("Nginx Status:\n", result.stdout)

✅ Use this for safe and simple execution of shell commands. Use check=True to raise exceptions on failure.

---

🔹 subprocess.Popen() — Advanced Process Control (Stream Logs)

process = subprocess.Popen(["tail", "-f", "/var/log/syslog"], stdout=subprocess.PIPE)

for line in process.stdout:
    print(line.decode(), end="")

✅ Use for non-blocking or long-running processes like streaming logs.

---

🔹 subprocess.call() — Run Command & Get Return Code

exit_code = subprocess.call(["ping", "-c", "3", "google.com"])
print("Ping exit code:", exit_code)

✅ Simple command check, returns exit status.

---

🔹 subprocess.check_output() — Capture Output or Fail

output = subprocess.check_output(["hostname", "-f"], text=True)
print("Fully qualified hostname:", output.strip())

✅ Use when you want command output, but want to fail on error.

---

🔹 subprocess.check_call() — Run and Enforce Success

subprocess.check_call(["kubectl", "apply", "-f", "deployment.yaml"])
print("K8s deployment applied successfully.")

✅ Like run(..., check=True) — good for critical infra steps.

---

🔹 subprocess.DEVNULL — Suppress Output

subprocess.run(["ls", "/root"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)

✅ Used to run quietly — no output/no logs.

---

🔹 stdout=subprocess.PIPE / stderr=subprocess.PIPE — Capture Logs

result = subprocess.run(["df", "-h"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print("Disk Usage:\n", result.stdout)

✅ Use for capturing and logging system status in tools.

---

🔹 shell=True — Use Shell Commands (e.g., pipes)

result = subprocess.run("cat file.txt | grep error", shell=True, capture_output=True, text=True)
print("Errors found:\n", result.stdout)

⚠️ Only use shell=True when absolutely necessary, and never with untrusted input.

---

🔹 timeout= — Prevent Stuck Commands

try:
    subprocess.run(["sleep", "10"], timeout=3)
except subprocess.TimeoutExpired:
    print("Command timed out!")

✅ Critical for scripts running in CI/CD or cron.

---

🔹 env= — Run Command with Custom Env

subprocess.run(["printenv", "STAGE"], env={"STAGE": "dev"}, shell=True)

✅ Useful for setting temporary environment variables.

---

🔹 cwd= — Run Command from Specific Directory

subprocess.run(["ls"], cwd="/var/log")

📌 Used in deploy tools, build systems, or for repo management.

---

🔹 subprocess.CompletedProcess — Returned by run()

result = subprocess.run(["uname", "-a"], capture_output=True, text=True)
print("Exit code:", result.returncode)
print("Output:", result.stdout)

✅ Use attributes like .stdout, .stderr, .returncode

---

✅ Summary Table

Method / Argument	DevOps/SRE Use Case
subprocess.run()	Safe, high-level command execution
subprocess.Popen()	Stream logs, background jobs
subprocess.call()	Run command and get status code
subprocess.check_output()	Get command output, raise on failure
subprocess.check_call()	Run and fail on non-zero exit
subprocess.DEVNULL	Suppress stdout/stderr completely
stdout=PIPE, stderr=PIPE	Capture command logs for monitoring
shell=True	Run shell scripts/pipes (careful with security)
timeout=	Prevent hanging scripts
env=	Run with custom environment (e.g., STAGE, AWS_PROFILE)
cwd=	Run in a specific directory
CompletedProcess object	Inspect result of subprocess (code, stdout, stderr)

---

Would you like a combined PDF, image cheat sheet, or interactive Jupyter notebook with all modules (requests, boto3, yaml, json, os, subprocess) for quick DevOps scripting reference?