Day 15 Task: Python Libraries for DevOps

Reading JSON and YAML in Python

DevOps engineer mostly work with configuration files and Data in various format like JSON: JavaScript Object Notation and YAML: Yet Another Markup Language.These formats are mostly used for to store configuration settings, data exchange, specifying infrastructure as a code.
For JSON use json library in python to prase and manipulate JSON data.
For YAML use PyYAML library in python to read and process YAML files.

As a DevOps Engineer you should be able to parse files, be it txt, json, yaml, etc.

DevOps engineer mostly deal with variety types of file such as Plain texttxt, JSONjason,YAMLyaml and other files.Parsing file involves reading, interpreting, extracting meaning ful information from the files.This skills is essitional for tasks like automation scripts, configuration management and working with infrastructure as a code.

You should know what all libraries one should use in Pythonfor DevOps.

DevOps task are mostly interact with the operating system, executing shell command,manipulating files and working with other data various formats.Python offers a rich ecosystem of libraries that facilitate this operations.Some of the key libraries are:
- os and sys interact with operating system.
- json handling the JSON data.
- yaml processing the YAML data.
- And other libraries like subprocess , requests ,paramiko (for SSH) , docker-py (for Docker interaction) , boto3 (for AWS interaction) depending on specific use cases.

Python has numerous libraries like`os`,`sys`,`json`,`yaml`etc that a DevOps Engineer uses in day to day tasks.

These libraries provide important feature for day-to-day DevOps tasks.Such as:
- os and sys managing operating system and system level interaction.
- json and yaml handling and processing the data of JSON and YAML format.
- subprocess allows to running external process and shell commands.
- A variety of other libraries allows to specific needs to interact with containers(Docker),cloud service(boot3 for AWS) or making HTTPS requests.

Tasks

Create a Dictionary in Python and write it to a json File.

import json

# Creating a dictionary
cloud_services = {
    'aws': 'ec2',
    'azure': 'VM',
    'gcp': 'compute engine'
}

# Writing the dictionary to a JSON file
with open('cloud_services.json', 'w') as json_file:
    json.dump(cloud_services, json_file)

print("Task 1 completed: Dictionary written to 'cloud_services.json'")

Explanation of syntax:

import json:
- This line imports the json module in Python, which provides methods for working with JSON data.
Creating a Dictionary:
- The cloud_services dictionary is created with three key-value pairs. This dictionary represents different cloud service providers and their associated services.
Writing the Dictionary to a JSON File:
- with open('cloud_services.json', 'w') as json_file: opens a file named 'cloud_services.json' in write mode ('w'). The with statement is used to ensure that the file is properly closed after writing.
- json.dump(cloud_services, json_file) writes the content of the cloud_services dictionary to the specified JSON file (json_file).
print("Task 1 completed: Dictionary written to 'cloud_services.json'"):
- This line prints a message indicating that Task 1 is completed, specifically mentioning that the dictionary has been successfully written to the 'cloud_services.json' file.

OutPut:

Read a json file`services.json`kept in this folder and print the service names of every cloud service provider.

import json

# Reading the JSON file
with open('services.json', 'r') as json_file:
    services_data = json.load(json_file)

# Printing service names for each cloud service provider
for provider, service in services_data.items():
    print(f"{provider} : {service}")

Explanation of the syntax:

import json:
- This line imports the json module in Python, which provides methods for working with JSON data.
Reading the JSON File:
- with open('services.json', 'r') as json_file: opens the 'services.json' file in read mode ('r'). The with statement is used to ensure that the file is properly closed after reading.
- services_data = json.load(json_file) reads the content of the JSON file and parses it into a Python dictionary. The resulting dictionary is stored in the variable services_data.
Printing service names for each cloud service provider:
- for provider, service in services_data.items(): iterates through the key-value pairs in the services_data dictionary.
- print(f"{provider} : {service}") prints each key-value pair, representing a cloud service provider and the associated service. The f"{provider} : {service}" syntax is a formatted string, where {provider} and {service} are replaced with the corresponding values during printing.

OutPut:

Read YAML file using python, file`services.yaml`and read the contents to convert yaml to json

First, you need to install the PyYAML library. Open your terminal and run:

pip install pyyaml

Now, you can use the following Python script:
Python script is designed to read both JSON and YAML files and print their contents. Before running the script, make sure you have the services.json and services.yaml files in the same directory with the specified content.

import json
import yaml

json_file = "services.json"
yaml_file = "services.yaml"

with open(json_file, 'r', encoding='utf-8') as f:
    json_data = json.loads(f.read())

print("JSON:\n",json_data)

with open(yaml_file, "r") as stream:
    try:
        yaml_data = yaml.safe_load(stream)
    except yaml.YAMLError as exc:
        print(exc)


print("YAML:\n",yaml_data)

Now, to execute this script in VSCode, follow these steps:

1 . Create a new Python file, for example, pars.py in VScode.

2 . Copy and paste the provided script into the file.

3 . Save the file.

4 . Make sure you have the services.json and services.yaml files in the same directory.

#service.json
{
    "services": {
      "debug": "on",
      "aws": {
        "name": "EC2",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      },
      "azure": {
        "name": "VM",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      },
      "gcp": {
        "name": "Compute Engine",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      }
    }
  }

#service.yaml
---
services:
  debug: 'on'
  aws:
    name: EC2
    type: pay per hour
    instances: 500
    count: 500
  azure:
    name: VM
    type: pay per hour
    instances: 500
    count: 500
  gcp:
    name: Compute Engine
    type: pay per hour
    instances: 500
    count: 500

4 . Open a terminal in VSCode and navigate to the directory where the Python file is located.

5 . Run the script using the following command:

python .\parser.py

Output:

This script will read the contents of both the JSON and YAML files and print them in the terminal. The expected output should include the JSON and YAML data as dictionaries.

Happy Learning

Thanks For Reading! :)

-DevOpsParthu💝💥

Day 15 Task: Python Libraries for DevOps

Table of contents