Getting too old

For hacking out Software features

Photo by Fotis Fotopoulos on Unsplash

Is it possible to say that one is getting too old for hacking out Software features? Is Software Engineering a young person’s game only? We might not have the answers to these questions but we will explain in this article some steps involved in adding a new feature to an existing Web application.

First, we prepared a small illustration of the feature under discussion. Next we will provide a summary of the work involved and close with some thoughts about getting old.

A log statistics viewer

The recorded demo shows the initial coding of the ‘Web Log Viewer’ we have been working on for this past week. The user requests our landing page and signs in. We arrive at the Vue.js application and proceed to the Reports Menu, selecting the Log.

A recording of a new feature under construction. https://www.mlexperience.org

Our new Log feature is being built to provide a Dashboard so that our users can see the amount of web traffic being handled by our site. We provide a visual using Vue-Vega and the Refresh button makes an api call to get the latest log data from MongoDB. Next we will reflect on the work so far.

Technical work in adding a feature

In our previous article ‘ Making sense of Web Logs , Log analysis with Python for fun! ’ we created a new backlog job that manages the analysis of the Web Log. In closing out that article we set out an agenda for future work:-

• Add the Python script to the Cron scheduler on an Ubuntu VM over in AWS land.
• Add a MongoDB instance and write the parsed out values to a Database.
• Create an interactive Dashboard using Plotly or Vega and publish the Dashboard as part of the core application.

This article continues the technical work.

Python Script

Our previous batch job required some revisions so that the output of the job would be to our MongoDB database. We also added an IP address lookup and that allowed us to have more features and information about our visitors. The revised script took a few hours and included console based testing.

Interactive Dashboard using Vega

Earlier we introduced the Dashboard as the new feature and we reviewed the design. The technical work involved:-

• Our Back-end Server is through Python Flask. All database services are through a DB services class. There was a change to the DB services class to provide a new method. That new method fetches the Web Log data from the MongoDB instance.
• With the Back-end server we added a new API end point (‘/api/stats’) and we added code to service calls to the end point from the client side.
• At the client side we added a new Vue component called Log.vue. Log.vue requires an API call, and a template to render the Vega visual.
• We also had to change our Nav.vue component which offers the Menu as a service component. Nav.vue had the ‘log’ option added under the Reports menu.
• We had to change our router.js so that the Vue router could map the menu option ‘report->log’ to the Log component.

I bet that sounds easy and you might ask what would the big deal be. It is a big deal and things drove us mad.

Going mad

Going mad at the front

Linting errors drive me mad.

These linting messages are warnings but behind every warning is a large problem waiting. For a considerable time our Refresh button wasn’t working at all. There was no obvious reason. The linting errors had hidden a problem. One of the closing curly braces was in the wrong place. The effect of putting a curly brace in the wrong place was that our getStats() function vanished from the run-time context. We got t.getStats() is unknown.

Closing curly brace nightmares

Next the eternal problem of the Div. Did you close out all your Div’s? We didn’t again! Our button, a column and a row seemed to collide.

A Vue.js template with a lot of <div> and </div> that need to correct.

Gone mad at the back

Adding to and testing a Flask application, with Gunicorn, and the Nginx setup using SSL/TLS is not straightforward. We also connect to MongoDB and Memcached servers for the database services.

Our batch job adds the records to the MongoDB. Our Database class then reads the data, posted by the batch, and provides that to the Flask Application. The Flask application converts the Python data structure (dictionary) to a json object and sends that object back to the client side.

There are some moving parts here and as usual we had some issues where the record written by the batch job differed from what the database class expected and the Flask guy wanted sugar with that. What the Flask guy sent to the client side looked like yesterday’s eaten lunch and the Log.vue component just plum refused to eat Flask’s lunch at all.

Yes, going mad at the back end is extra special after a visit to the client-side lunatic asylum of linting and curly brace hell. How many times was that famous expression heard ‘I’m getting too old for this shit’?

Too old for this shit

Creating a new Software feature in a modern web application is a lot of work. Nobody minds hard work but it is time consuming and the simple things just kill hours of time. Why is the button not working? Doh! a curly brace is in the wrong place. Yeah, I’m getting too old for this shit!