Lab 3.2: File I/O and Serialization

Overview

Network automation requires reading configurations, storing data, and exchanging information between systems. This lab covers basic file operations and common serialization formats used in network automation.

What you'll learn:

Basic file I/O (read, write, append)
JSON for configuration and API data
YAML for human-readable configs
XML for legacy systems
Environment variables with dotenv
Pickle for Python objects

Part 1: Basic File I/O

Step 1: Writing Files

# Write to a file (overwrites if exists)
with open("network.txt", "w") as f:
    f.write("Router-1: 192.168.1.1\n")
    f.write("Router-2: 192.168.1.2\n")

# Append to a file
with open("network.txt", "a") as f:
    f.write("Router-3: 192.168.1.3\n")

# Write multiple lines at once
lines = [
    "Switch-1: 192.168.2.1\n",
    "Switch-2: 192.168.2.2\n",
]
with open("switches.txt", "w") as f:
    f.writelines(lines)

Key point: Always use with open() - it automatically closes the file.

Step 2: Reading Files

# Read entire file
with open("network.txt", "r") as f:
    content = f.read()
    print(content)

# Read line by line
with open("network.txt", "r") as f:
    for line in f:
        print(line.strip())  # strip() removes \n

# Read all lines into a list
with open("network.txt", "r") as f:
    lines = f.readlines()
    print(lines)

Step 3: File Modes

# Common modes:
# "r"  - Read (default)
# "w"  - Write (overwrites)
# "a"  - Append
# "r+" - Read and write
# "rb" - Read binary
# "wb" - Write binary

Part 2: JSON (JavaScript Object Notation)

JSON is the most common format for APIs and configuration files. Python dicts map directly to JSON.

Step 1: Writing JSON

import json

# Create a dictionary (configuration data)
network_config = {
    "hostname": "Router-1",
    "ip_address": "192.168.1.1",
    "interfaces": [
        {"name": "GigabitEthernet0/0", "ip": "10.0.0.1", "status": "up"},
        {"name": "GigabitEthernet0/1", "ip": "10.0.1.1", "status": "down"}
    ],
    "vlans": [10, 20, 30],
    "enabled": True
}

# Write to JSON file
with open("router_config.json", "w") as f:
    json.dump(network_config, f, indent=2)

# Convert to JSON string
json_string = json.dumps(network_config, indent=2)
print(json_string)

Step 2: Reading JSON

import json

# Read from JSON file
with open("router_config.json", "r") as f:
    config = json.load(f)

print(config["hostname"])
print(config["interfaces"][0]["name"])

# Parse JSON string
json_data = '{"device": "switch", "ports": 48}'
data = json.loads(json_data)
print(data["device"])

Step 3: Handling JSON Errors

import json

# Bad JSON will raise exception
try:
    with open("config.json", "r") as f:
        data = json.load(f)
except FileNotFoundError:
    print("File not found")
except json.JSONDecodeError as e:
    print(f"Invalid JSON: {e}")

When to use JSON:

REST API requests/responses
Configuration files (modern tools)
Data exchange between systems
Storing structured data

Part 3: YAML (Human-Readable Configuration)

YAML is popular for configuration files (Ansible, Docker Compose, Kubernetes).

Step 1: Install PyYAML

pip install pyyaml

Step 2: Writing YAML

import yaml

# Same dictionary as before
network_config = {
    "hostname": "Router-1",
    "ip_address": "192.168.1.1",
    "interfaces": [
        {"name": "GigabitEthernet0/0", "ip": "10.0.0.1", "status": "up"},
        {"name": "GigabitEthernet0/1", "ip": "10.0.1.1", "status": "down"}
    ],
    "vlans": [10, 20, 30],
    "enabled": True
}

# Write to YAML file
with open("router_config.yaml", "w") as f:
    yaml.dump(network_config, f, default_flow_style=False)

# Convert to YAML string
yaml_string = yaml.dump(network_config, default_flow_style=False)
print(yaml_string)

Output looks like:

hostname: Router-1
ip_address: 192.168.1.1
interfaces:
- name: GigabitEthernet0/0
  ip: 10.0.0.1
  status: up
- name: GigabitEthernet0/1
  ip: 10.0.1.1
  status: down
vlans:
- 10
- 20
- 30
enabled: true

Step 3: Reading YAML

import yaml

# Read from YAML file
with open("router_config.yaml", "r") as f:
    config = yaml.safe_load(f)

print(config["hostname"])
print(config["interfaces"][0]["name"])

# safe_load() is safer than load() - prevents code execution

When to use YAML:

Configuration files (Ansible playbooks, Docker Compose)
More readable than JSON for humans
Supports comments (JSON doesn't)

Part 4: XML (Legacy Systems)

XML is older but still used in network devices (NETCONF, some APIs).

Step 1: Writing XML with ElementTree

import xml.etree.ElementTree as ET

# Create XML structure
root = ET.Element("device")
hostname = ET.SubElement(root, "hostname")
hostname.text = "Router-1"

ip_address = ET.SubElement(root, "ip_address")
ip_address.text = "192.168.1.1"

interfaces = ET.SubElement(root, "interfaces")
interface1 = ET.SubElement(interfaces, "interface")
ET.SubElement(interface1, "name").text = "GigabitEthernet0/0"
ET.SubElement(interface1, "ip").text = "10.0.0.1"
ET.SubElement(interface1, "status").text = "up"

# Write to file
tree = ET.ElementTree(root)
ET.indent(tree, space="  ")  # Python 3.9+
tree.write("router_config.xml", encoding="utf-8", xml_declaration=True)

# Print to string
xml_string = ET.tostring(root, encoding="unicode")
print(xml_string)

Output looks like:

<?xml version='1.0' encoding='utf-8'?>
<device>
  <hostname>Router-1</hostname>
  <ip_address>192.168.1.1</ip_address>
  <interfaces>
    <interface>
      <name>GigabitEthernet0/0</name>
      <ip>10.0.0.1</ip>
      <status>up</status>
    </interface>
  </interfaces>
</device>

Step 2: Reading XML

import xml.etree.ElementTree as ET

# Parse XML file
tree = ET.parse("router_config.xml")
root = tree.getroot()

# Access elements
hostname = root.find("hostname").text
print(f"Hostname: {hostname}")

# Iterate through interfaces
interfaces = root.find("interfaces")
for interface in interfaces.findall("interface"):
    name = interface.find("name").text
    ip = interface.find("ip").text
    print(f"{name}: {ip}")

# Parse XML string
xml_data = "<device><name>Switch-1</name></device>"
root = ET.fromstring(xml_data)
print(root.find("name").text)

When to use XML:

NETCONF protocol (Cisco, Juniper devices)
Legacy APIs
Some vendor-specific tools

Part 5: Environment Variables with dotenv

Store secrets and configuration outside your code.

Step 1: Install python-dotenv

pip install python-dotenv

Step 2: Create .env File

Create a file named .env:

# Network device credentials
DEVICE_USERNAME=admin
DEVICE_PASSWORD=secretpass123
DEVICE_IP=192.168.1.1

# API keys
API_KEY=abc123xyz456
API_URL=https://api.example.com

Important: Add .env to .gitignore - never commit secrets!

Step 3: Load Environment Variables

import os
from dotenv import load_dotenv

# Load variables from .env file
load_dotenv()

# Access environment variables
username = os.getenv("DEVICE_USERNAME")
password = os.getenv("DEVICE_PASSWORD")
api_key = os.getenv("API_KEY")

print(f"Connecting as: {username}")

# Provide default value if not found
debug_mode = os.getenv("DEBUG_MODE", "False")

When to use dotenv:

Storing credentials (usernames, passwords, API keys)
Environment-specific configuration (dev vs prod)
Keeping secrets out of source code

Part 6: Pickle (Python Objects)

Pickle serializes Python objects to binary format. Warning: Only unpickle trusted data!

Step 1: Pickling Objects

import pickle

# Create complex Python objects
network_devices = [
    {"hostname": "Router-1", "ip": "192.168.1.1", "ports": [1, 2, 3]},
    {"hostname": "Switch-1", "ip": "192.168.2.1", "ports": [10, 20, 30]},
]

# Save to pickle file (binary)
with open("devices.pkl", "wb") as f:
    pickle.dump(network_devices, f)

# Can pickle almost any Python object
class Router:
    def __init__(self, name, ip):
        self.name = name
        self.ip = ip

router = Router("Router-1", "192.168.1.1")

with open("router.pkl", "wb") as f:
    pickle.dump(router, f)

Step 2: Unpickling Objects

import pickle

# Load from pickle file
with open("devices.pkl", "rb") as f:
    devices = pickle.load(f)

print(devices[0]["hostname"])

# Load custom object
with open("router.pkl", "rb") as f:
    router = pickle.load(f)

print(f"{router.name}: {router.ip}")

Step 3: When to Use Pickle

Use pickle for:

Caching Python objects
Saving program state
Inter-process communication (on same machine)

Don't use pickle for:

❌ Data exchange with other languages (use JSON)
❌ Long-term storage (pickle format can change)
❌ Untrusted data (security risk - can execute code!)
❌ Human-readable configs (use YAML/JSON)

Part 7: Quick Comparison

Format	Best For	Human Readable	Cross-Language
JSON	APIs, configs	✓	✓
YAML	Configs, Ansible	✓✓	✓
XML	Legacy, NETCONF	✓	✓
dotenv	Secrets, env vars	✓	✓
Pickle	Python objects	✗	✗

Part 8: Hands-On Challenge

Create a network inventory system:

Requirements:

Read device list from CSV:

hostname,ip,device_type
Router-1,192.168.1.1,cisco_ios
Switch-1,192.168.2.1,cisco_ios
Firewall-1,10.0.0.1,palo_alto

Convert to dictionary and save as:
- JSON (inventory.json)
- YAML (inventory.yaml)
- Pickle (inventory.pkl)
Create .env file with:
- Default username
- Default password
- SSH port
Load all formats and verify they contain the same data
Bonus: Create simple XML output for one device

Starter code:

import csv
import json
import yaml
import pickle
from dotenv import load_dotenv
import os

# Read CSV
devices = []
with open("devices.csv", "r") as f:
    reader = csv.DictReader(f)
    for row in reader:
        devices.append(row)

# TODO: Convert to different formats
# TODO: Load and verify

Success Criteria

You've completed this lab when you can:

Read and write text files with open()
Use with statements for file handling
Convert Python dicts to JSON and back
Convert Python dicts to YAML and back
Create and parse basic XML with ElementTree
Load environment variables with dotenv
Serialize Python objects with pickle
Choose the right format for the job

Key Takeaways

File I/O:

Always use with open() for automatic cleanup
"r" read, "w" write (overwrite), "a" append

JSON:

json.dump() / json.load() for files
json.dumps() / json.loads() for strings
Best for APIs and modern configs

YAML:

yaml.dump() / yaml.safe_load()
More readable than JSON
Popular for Ansible, Docker, K8s

XML:

xml.etree.ElementTree (standard library)
Used in NETCONF and legacy systems
More verbose than JSON/YAML

dotenv:

Never commit secrets to git
Use .env for local development
os.getenv() to access variables

Pickle:

Fast Python-specific serialization
Binary format (not human-readable)
Security risk - only unpickle trusted data
Don't use for data exchange or long-term storage

Real-world usage in network automation:

JSON: REST API responses, configuration files
YAML: Ansible playbooks, device configs
XML: NETCONF, legacy device APIs
dotenv: Credentials, API keys
Pickle: Caching, temporary state

Additional Resources

Congratulations! You now know how to read/write files and work with common serialization formats used in network automation!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Lab 3.2: File I/O and Serialization

Overview

Part 1: Basic File I/O

Step 1: Writing Files

Step 2: Reading Files

Step 3: File Modes

Part 2: JSON (JavaScript Object Notation)

Step 1: Writing JSON

Step 2: Reading JSON

Step 3: Handling JSON Errors

Part 3: YAML (Human-Readable Configuration)

Step 1: Install PyYAML

Step 2: Writing YAML

Step 3: Reading YAML

Part 4: XML (Legacy Systems)

Step 1: Writing XML with ElementTree

Step 2: Reading XML

Part 5: Environment Variables with dotenv

Step 1: Install python-dotenv

Step 2: Create .env File

Step 3: Load Environment Variables

Part 6: Pickle (Python Objects)

Step 1: Pickling Objects

Step 2: Unpickling Objects

Step 3: When to Use Pickle

Part 7: Quick Comparison

Part 8: Hands-On Challenge

Success Criteria

Key Takeaways

Additional Resources

Uh oh!

FilesExpand file tree

3.2-python-IO-Serialization.md

Latest commit

History

3.2-python-IO-Serialization.md

File metadata and controls

Lab 3.2: File I/O and Serialization

Overview

Part 1: Basic File I/O

Step 1: Writing Files

Step 2: Reading Files

Step 3: File Modes

Part 2: JSON (JavaScript Object Notation)

Step 1: Writing JSON

Step 2: Reading JSON

Step 3: Handling JSON Errors

Part 3: YAML (Human-Readable Configuration)

Step 1: Install PyYAML

Step 2: Writing YAML

Step 3: Reading YAML

Part 4: XML (Legacy Systems)

Step 1: Writing XML with ElementTree

Step 2: Reading XML

Part 5: Environment Variables with dotenv

Step 1: Install python-dotenv

Step 2: Create .env File

Step 3: Load Environment Variables

Part 6: Pickle (Python Objects)

Step 1: Pickling Objects

Step 2: Unpickling Objects

Step 3: When to Use Pickle

Part 7: Quick Comparison

Part 8: Hands-On Challenge

Success Criteria

Key Takeaways

Additional Resources