I’m a details person, so I wanted a way to log the total number of days I worked, when I worked remotely, and when I took holiday or vacation days. I created a command-line Python application that saved data to a CSV file and used it to log my days. While this worked for my needs, my manager suggested that I modify my application and scale it to work on the cloud as a web application. This blog post details how I researched, planned and implemented this web application.
The planning phase: Ask the right questions
Before I could containerize the application using Docker and deploy the app to the cloud, I needed to create a working, local version of my application. Since Python is the foundation of this application, I wanted to build on top of that with web-based API management. I began to research and understand how Flask, a micro web framework, works and how I could make it work with my command-line application.
As I began to tweak parts of my existing application to use with Flask, it prompted some questions:
- Using CSV to store data is not appropriate on the cloud. How will I store data on the cloud?
- What API endpoints should I create? How should a user be able to interact with this application?
With these questions in hand, I decided to meet with my manager to get another opinion and see what insight he could offer.
Choosing the right data storage
For data storage, my manager suggested I look into Cloudant, MongoDB, and MySQL and explore which storage would best support what I was trying to achieve. I quickly removed Cloudant from the running because I couldn’t find any documentation that showed it was supported by Python.
With my choices down to MongoDB and MySQL, I had to determine whether a relational database was something that was necessary for this application. The data related to the number of remote days worked helped me make my decision.
The way I stored the data in the CSV file included three pieces of data:
- the total number of days that I worked remotely
- each location where I worked
- the number of days worked remotely in each location
Since the locations would differ for each user, I couldn’t come up with a schema in a relational database to handle all locations appropriately. This is where I saw the value and flexibility of a non-relational database. With that, I decided to use MongoDB. And with no prior experience using MongoDB, I began my research into learning how to use and incorporate it with my application.
Broadening the scope of my work
While I know how I use this application, I asked myself how other users may potentially use this application and added a space to record office days worked and sick days used.
I began to write an API.md file that outlined the API endpoints my application would have and the responses from these API calls. All of the endpoints were for one user to interact with the application, similarly to how I use the application. When I showed my manager my progress, he suggested that I expand and scale the application so that multiple users on a team could use it. The best way to do that would be through a cloud-based application.
I went back and reviewed my application, database structure, and API.md file and tried to see how I could expand and scale what I have worked on to include this capability. I then ran into this problem: Each user should only be able to access data associated with themselves. How can I make the application secure and capable of achieving this?
Creating user account information
In order to limit a user’s access to only his or her data, I decided that users should have accounts for logging into the application. To achieve this, I created another database with user account information and updated the API.md file with how I wanted to modify the application to handle this.
I then began researching how I could incorporate logging in and user sessions. I first looked into sessions from Flask. I ran into some blockers with getting sessions to hold user data so the program would know the user was logged in. I switched gears and decided to incorporate session info in MongoDB which ended up working as intended. With the application now having the capability to handle multiple users, a new question now came to light: I can easily create new users in MongoDB, but how can I easily allow others to create new users?
I have the luxury of creating new users myself in MongoDB, but actual users of this cloud-based application won’t have this luxury. In fact, when an instance of this application is created, there won’t be any users at all. I needed to figure out how to handle initializing this application and onboarding of new users. I decided to create an endpoint to allow for creating new users.
Adding security and encryption
Once I finished the user end points, I moved on to adding a layer of security by encrypting the passwords when storing them in the database. After including this layer, I was satisfied with the application for the time being, so I started the process of moving the application to Docker containers and Kubernetes.
Next steps after deployment
After successfully containerizing the application in Docker, adding the containers to Kubernetes, and moving the application to IBM Cloud, I finally had my local application running on the cloud. It was definitely a rewarding feeling making successful API calls to the cloud.
Now that I had a working version of my application on the cloud, I started asking myself questions about next steps:
- Now that this application is running successfully, how should I test this application? What kind of a test bed should I create?
- How can I improve this application? What new features should I add?
- This application currently returns JSON objects, what kind of a web GUI should I build on top of this?
Testing the application
For testing this application, I decided on creating Python unit tests where I would be testing each API end point. I researched how to automate the test bed. Eventually, I was able to automate the initializing of a test client for the Flask application and a Docker container for the Mongo database. I then had a working automated test bed.
A tweak to the file structure
Once I had my application and test bed where I wanted them, I asked for feedback so I could improve my application. One person suggested that I reorganize the application file structure so each file would have its own “microservice” rather than having all the files in one main file. This suggestion made sense, because typically applications are not just in one file and are spread out. This change also makes the code less intimidating and easier to follow and understand. In addition to making the application easier to understand, I also included a Swagger file for those who use this application to know how the APIs are supposed to work.
Designing the GUI
Once my restructuring was completed and the application was running where it was supposed to, I started to design the GUI. I wanted to implement design thinking principles into the process of creating the GUI.
I went to my manager to help me with this process for a couple reasons: He has more experience than me with UI/UX design, and I wanted someone else’s opinion and perspective on the UI and functionality of the application. During our design thinking sessions, we both realized that this application had a lot of potential and there were so many directions where we could take this application. At the end of the day, we both realized that we needed to focus on the original goal, which was to get a working GUI for the existing functionality. As a result, we then designed wire frames for the GUI.
With all parts of the application now successfully running on the IBM Cloud, I demoed the application to my manager and colleagues. The working GUI help them better see and understand the application and my work. It was mentioned that my database schema could be modified to allow for individual entries associated with a specific date to the work log rather than increasing the totals for each day type for a given year. This would allow for more flexibility, better logging of work log data, and also incorporate a calendar component to the front end. I decided it would be best to modify the database schema to incorporate these changes. I then made all of the other resulting necessary changes to the rest of the application.
With all of the updates now working as intended, I conducted usability testing with some users and recorded their feedback. The feedback was in respect to the design, functionality, and ideas the users had for future improvements and additions.
With all of this feedback in hand, I made improvements to the GUI and functionality of the UI. Having some time to spare, I decided to implement a few of the smaller ideas that some of the users recommended which included Slack integration.
Throughout this process I not only had to learn how to use Flask, MongoDB, Kubernetes, React, and other various components of how to make and structure an application, but also how to get all of the different technology to work together. This was definitely a rewarding experience, and I hope you enjoyed reading this and want to use this application!