This is an update on things related to this blog and my work. This month post is a bit longer than usually. This post is a log over the last 4 months.
I have not written much recently. I have been busy with lots of things and there has not been enough time left for writing.
Some time I ago wrote an article about the European Business Number. As the letters are sent in different countries I get a peak in the number of readers. This peak is from Greece. It occurred in February but since then there have been corresponding peaks from multiple countries, including Estonia.
I had a large list of article topics I wanted to write about. I deleted it as lots of it had become obsolete or irrelevant for me. I now have a new list.
Due to construction noise at my home I was forced to rent an office at the beginning of February. I was not able to find a good place in my home town. All places that were offered were either too large (100+ square meters, too noisy, required long contract, or had no central heating). One of the places asked me to provide services for a room for which I did not have enough time. I finally found a nearest good place in Tartu, 25km from home. The building is managed by Kaanon Kinnisvara and they have lots of office rooms here.
The room had a soviet-era table and a newer chair. I found the chair too soft for sitting for a longer time and I brought my own. In the room, there is a closet for clothes and some shelves as well. I do not plan to stay long so I have not got more furniture.
My largest project was put on hold again in January. The reasons were raising from the results of the recent US presidental elections and its effect on minorities that makes living in US impossible for some people.
Electron and Vue.js
The Electron-based desktop application I wrote in December saw further development. It was my first Vue.js application and going back allowed to evaluate it further. Electron was upgraded to Node.js 7.6 with native async/await and I took benefits from it by rewriting my promise-based code to employ the new control structures. In my opinion it makes Node.js and anything based on it a superior platform for IO-heavy applications. Vue.js parts of the application were refactored into components which made the codebase easier to maintain and extend. I have a plan to write more about my experience with Vue.js and compare it to React and KnockoutJS. I have used KnockoutJS so far but I consider it too hard to be safely used in large applications. I have recently revisited React and its exellent tool support and much simpler working principles are worth considering.
My main project during March and April was an electronic voting system for the Estonian Free Party. We built the main part of the system a year ago but it was not possible to use it due to legal reasons. The law does not prohibit electronic voting but neither gives any useful directions how to properly implement it. This was finally solved and we proceeded to implement it. Part of the solution were electronic signatures given by using the ID-card and Mobiil-ID. Up to this point we only had implemented authentication with them. The application was built onto Node.js and the async parts of the code were refactored to use the new async/await language constructs. This made the code cleaner and improvements easier. The voting process itself lasted couple of days. The biggest trouble was solving technical ID-card issues that some users were having. There are lots of cases where things do not work correctly and we are unable to report exact error because things are not under our control (browser plugins, hardware to read cards, etc.).
In one of the older projects I had used UUID's as primary keys. This project was not suitable for them: lots of secondary indexes. The main table had over 500k rows and indexes took 3 times more space than the amount of data in the whole database. In MySQL InnoDB engine, an index points to primary keys and thus all primary key values have to be stored for each index. Furthermore, I had used
CHAR(36) datatype for UUID's. This datatype takes 3*36 bytes when the column character set is
utf8mb4. In this project I was able to replace UUID primary keys with good old
auto_increment integer keys.
In another project I had to replace Knex with a simpler solution: SQL queries in files. Knex is pretty OK for CRUD queries but gets in the way once queries join multiple tables, some of which are derived tables from further subqueries. The query-in-file approach was recommended by Gajus.
Toolset: Sublime Text
I did improve the text editor a bit by finally finding a good set of plugins. I use Sublime Text and the list of plugins:
- Sublime Linter (a framework, required by ESLint plugin)
- DocBlockr (helps to write block comments)
- Theme Soda Light
- Prolog (for editing Prolog files)
- Less (for editing Bootstrap 3 setups)
The plugins are all installable using the Package Control. Besides these plugins I also use a simple highlighter for EJS.
I tried to convert my Redmine installation text formatting from Textile into Markdown but it did not work well as Redmine Markdown does not support embedded HTML. This script is a good start but would have required too much tweaking for me as I had used lots of advanced Textile. I hope that Redmine adds project-specific text formatting option which would allow me to use Markdown for all the new projects and keep current Textile-based text formatting intact. There is a plugin to do it but I usually try to avoid installing plugins to avoid complicated upgrade when a new version of Redmine comes out.
I set up file syncronization for important parts of my home directory between different computers. I cannot use rsync for this as it is impossible to sync two-way deletes without keeping metadata. The Unison application seems to be an excellent tool for this. In my setup I syncronize each device with a central server to distribute changes between the devices.
At the moment I'm preparing for one new project. It is a continuation of one of my previous projects.
In May I will be on a vacation and will not work on commercial projects.
I have been thinking about the types of the projects that I work with. Startup-like projects are not anymore my favorite. These types of projects tend to have too many issues with budget and scope. One project having schedule issues means issues for anything else that was queued and scheduled to happen after the project. This is not sustainable and screws up lots of deals. I would be happy to work on a large uncertain project if it was my only one. This is not a situation I'm in today and I do not see myself working in this setting in the near future.
Some recent offerings have been typical enterprise projects to reduce the amount of manual work and boost productivity. This is what I originally wanted to do in 2010 except back then I lacked experience to get potential jobs and close deals. Some of my last projects have been fixed-price deals. I feel that I have enough experience now to make safe fixed-price estimates. I have used a stable development platform (Node.js and some SWI-prolog) for almost 5 years up to this point. Back then I was still experimenting with multiple different platforms and was not sure how much time will programming something take or whether it's practically useful or doable at all.
In the last 2 months everything has been migrated away from the server that was running at my home. The migration took some work as surprisingly many things were dependent on the server and my IP address. I sold the server disks and used money to set up a desktop machine at the new office.
I also bought a new laptop. It's an HP 250 G4. It came with a Windows 10 installation which I replaced with Debian 8 (MATE desktop). The laptop has excellent compatibility with Linux, at least on the model with an Intel wireless interface.
My main desktop at home is also running Debian 8. The installation has a tricky part to get UEFI+NVMe working although it was considerably easier than the previous setup with Slackware. The installation process took about 3 hours. I'm running dual boot with Windows 10. The MSI boot selector works well with multiple UEFI boot disks.
I'm now running Debian 8 on all my systems. Together with this I have a dozen of my client servers all running Debian. This lets me to focus on a single Linux distribution.
My backup scripts were gone too complex for shell scripts and had become hard to maintain. I rewrote them in Node.js and put the source on GitHub. The script does not embed keys and passwords anymore and is configured with a JSON file instead.
I decided to take over and fix the Node.js interface to the SWI-Prolog. I found a dozen of forks of it to make the codebase buildable on different Node.js versions. So I forked it again, fixed most of the issues and published a new package. Attempts to contact the original authors were unfruitful and I had to publish it under a new name.
Unfortunately I hit multiple restrictions with the bindings. I'm not an expert on native C++ code, especially on 3 different platforms. Instead, I wrote a new version that separates the Node.js and SWI-Prolog processes and communicates them over stdio. This turned out to work much better and I was able to solve the remaining issues (most important ones for me were support for unicode and dict data structures). However, all this comes with the cost of an additional serialization into line-terminated JSON that I use for transmitting the data over stdio.
One of my new projects needs a good PDF output support. I decided to play around with PDFKit a bit and made a simple Markdown to PDF converter. It supports a very limited subset of Markdown but I consider it good enough for producing text documents. The source of the application can be found on GitHub.
A new bicycle
I have finished building a new bicycle. It is put together from new and used parts, sourced from all over the EU. I had plans for building a new racing bike for the last 2 years but did not work on it actively until I acquired an high-end frame. This happened in the last September and since then I have gathered parts and put them together.
A custom part
The headset required a small spacer below the upper headset cap that covers the upper bearing. I did not find a suitable spacer so I had to make my own. It was kinda OK to use 2 BB30 bottom bracket spacers for testing but cap's inner bevel edge would have put too much stress on the spacer's edge and cause issues in the long term. I made a suitable spacer from an aluminum alloy workpiece and accounted for the bevel.
The bike is currently ready for racing which I intend to do during my vacation in May. I will write more about the bike build details in a separate post.
The construction work to rebuild homes (including mine) is proceeding fast. 4 buildings out of 5 already have the new outer heat insulation installed. Air ventilation shafts and new water pipes have been installed.
Noise and parts of work inside the apartments have caused some stress (moving 102 families into temporary homes would have been very costly) but I hope that the rest of the work will be done soon and normal living conditions are restored.