Implement flask-ops task

[SmirnovsIgor]

Introduction.

In this pull request I've implemented CI/CD pipeline for the flask-ops task using CircleCI and Heroku.
This documentation outlines the steps I took to deploy, and automate the deployment of, the flask-ops application.
The beginning of this documentation outlines the steps I took to understand the project and the environment and architectural decisions I've made.

Understanding of the project. Architectural decisions.

My first step was to start this application and to see how it works on my local machine. Then I started reading the code, thinking in parallel about the environment and pipeline steps, which I'd like to implement. First of all, I decided to change the code based on the PEP8 knowledge. Immediately I came across a secret variable and hid it to prevent security issues, e.g. if this app will be used in generating some tokens like csrf-token or others. Then I analyzed that there were no unit/integration tests. As I know, one of the most important DevOps methodology's goals are:

1. Lower failure rate of new releases
2. Providing continuous delivery with high software quality on each step of SDLC.

Due to this goals, I have made a decision of writing some unit and integration tests for this app, because, in my opinion, the main thing that can show that the product works well is testing.

Unit tests.

The main logic functions I've highlighted were encrypt_string() and generate() func's. I hadn't found the solution of testing the second one, so I decided to write unit tests for sha256 encrypting function.

Integration tests.

The main goals of these tests are to see if the app is working, i.e, it generates unique tokens and the server response is 200 OK.

While writing tests, I faced the problem of non reusable code. The generate() func was tied to a SECRET_SALT_KEY variable and could not be used in other parts of the program, e.g. in creating app mock object for tests.

Also I've edited the start.sh script file based on the work of Heroku itself. Heroku generates PORT env variable on which app will be running on the prod.
So I've decided to make this app running on 5000 by default and on PORT env var if it is exported. This allows us to enter the port on which we want to run this application.

In the next section I'll try to describe my thoughts on the pipeline I've implemented.

Pipeline steps.

1. Linter. I've added this step, because the python is based on the purity and readability of the code. Python has it's own style guides, so if devs ignore these style guides, they will make the code less readable, even for someone who is used to reading code that follows these guides. In my opinion this step will improve code quality both now and in the future (further development of this application).
2. Unit testing. These tests help to find errors in the units of the program on the earliest stage of the pipeline.
3. Build docker image.
4. Integration testing. Testing the interaction between integrated units in the image.
5. Push the image to the registry. Only If all previous steps work fine, we push image to the registry, otherwise we will have a lot of unnecessary images that doesn't work and take up a lot of space.
6. Deployment. We release new version of our app.

Installation.

Flask-ops requires Python v3.6+, pip v20.0.0+, Docker

Clone the repository.
Install the dependencies and start the server.

$ cd flask-ops
$ pip install -r requirements.txt
$ chmod +x ./start.sh && ./start.sh

Server will be available at 0.0.0.0:5000 by default
Also before launching the script you can also export the PORT var, on which you'd like to start the server.

Tests.

Unit tests.

To run unit tests run following command

$ python tests.py

Integration tests.

To run integration tests run following command

$ pytest -vv

Deployment.

Make sure you've installed requirements.txt
Export all necessary env variables before launching the script below.

#!/bin/bash

# Login to heroku and docker registry
bash .circleci/setup-heroku.sh
echo $HEROKU_API_KEY | docker login --username=_ --password-stdin registry.heroku.com

# Build and deploy
docker build --build-arg secret_salt=${SECRET_SALT} -t registry.heroku.com/${HEROKU_APP_NAME}/web .
docker push registry.heroku.com/${HEROKU_APP_NAME}/web
heroku container:release web --app ${HEROKU_APP_NAME}

Monitoring and maintenance

Heroku provides its own observability and monitoring tools. You can access them logging in the Heroku Dashboard.

[jaygeeseman]

Hi @SmirnovsIgor , the code changes you've made, as well as the pipeline setup look pretty good. Especially the python work you did is excellent, great work! However, as a devops/infra engineer, a large part of the job is clearly communicating with others.

• Unlike traditional software engineering, there are no product or project managers to help us conceive what we build - rather, we talk and plan directly with the people who will use the systems,
• We build systems that developers use extensively, and they need to know how to use them,
• Other infra developers need to know how to maintain them, and
• We build so many different things, that often things go untouched for years. By the time something needs work again, often no one remembers anything about it.

For these reasons, we need everything we build to be very clearly explained.

Following this theme, I have two requests:

1. Could you add documentation to this project? Currently, nothing explains how to set it up, how to run tests, how to deploy, how to monitor, or how to maintain it.
2. We're interested in why you made the architectural decisions you made. If you had more time, would you have done anything differently? Feel free to answer this question in the documentation.

Of course if you have any questions about any this, we can discuss in this PR here on GitHub.

[SmirnovsIgor]

Hi @SmirnovsIgor , the code changes you've made, as well as the pipeline setup look pretty good. Especially the python work you did is excellent, great work! However, as a devops/infra engineer, a large part of the job is clearly communicating with others.

• Unlike traditional software engineering, there are no product or project managers to help us conceive what we build - rather, we talk and plan directly with the people who will use the systems,
• We build systems that developers use extensively, and they need to know how to use them,
• Other infra developers need to know how to maintain them, and
• We build so many different things, that often things go untouched for years. By the time something needs work again, often no one remembers anything about it.

For these reasons, we need everything we build to be very clearly explained.

Following this theme, I have two requests:

1. Could you add documentation to this project? Currently, nothing explains how to set it up, how to run tests, how to deploy, how to monitor, or how to maintain it.
2. We're interested in why you made the architectural decisions you made. If you had more time, would you have done anything differently? Feel free to answer this question in the documentation.

Of course if you have any questions about any this, we can discuss in this PR here on GitHub.

Definitely, there must be documentation for this pull request. That's a good point. Now I'm at the stage of writing documentation, but I have one or two questions for you.

• For whom this documentation should be written, i.e., should I write it for users or for devs/infra in more technical way?
• Are there any style guides I need to follow? Could you please provide template, by which you would like to see the documentation?

[DragonStuff]

Hello @SmirnovsIgor,
So sorry we haven't been able to reply to this. Regarding documentation, ideally both the reasoning behind your changes to code and a reasonable written description of your thought process and how you have gone about the project.

No style guides required from our side.

Please use your best understanding of DevOps practices for writing documentation/communicate for when you make changes to a developer's code, and have to explain the infrastructure decisions to other developers/team members. For some inspiration, here's some documentation I wrote for a test project a few years ago: https://github.com/DragonStuff/saltstack-test/blob/master/index.html.md

[SmirnovsIgor]

Hello @DragonStuff, I've wrote the documentation for this PR. Could you please check it out?