View Posts for Software Engineering
-
Svelte/Sapper is F*cking Amazing
Updated at June 24, 2019 at the 03th hour
Categories
Alright alright, so I dumped React as my frontend framework for this website and that's primarily because Googlebot was being all weird in rendering my pages for search results (this was after I fixed my "mobile usability problems" with simple html!). Same reason why I dumped Aurelia, but I had Server Side Rendering working for React. Mind you, I was an Aurelia fan boi 😂. Well, I needed a new framework to use that had SSR working out of the box, but not with React or Angular style heavy bundle.
I considered just writing my own framework that just pushed html/css/js, but then I remembered Svelte v3 from a couple months ago and checked it out. Boy oh boy, I was blown away by the tutorial. I'm looking at it saying this is what Aurelia was trying to do! I was excited, but more cautious this time around with switching to it. I documented what I needed for this site and other internal UIs I created to verify what I'd be losing if I did switch. Once I verified that I liked it and actually gained, I took the plunge to convert the site. Honestly, I was fighting the decision ...
-
Building a ZeroMQ based Service Discovery application
Updated at May 24, 2019 at the 18th hour
Categories
Introduction
Service Discovery, the phonebook of applications, is a foundation for services oriented architecture. It answers the question of how does one service dynamically discover the endpoints of another service.
It is not a new thing that came with microservices. Zookeeper is a famous application for implementing service discovery on regardless of requirements. I give it good praise for the things you can do with it. If you don't have a lot of time to spend I recommend it.
Use Case
Building on top of my last post for Building a low latency Async RPC library, I needed a way for my applications to at runtime whether on laptop or datacenter environment find the registered running services quickly. From the get go, you know that there has to be a way to enter and query for entries. The application would have to be able to provide the ip:host and the protocol of which I label as the service provider. Protocol is http, rpc server and rpc publisher. It would also need to provide health checking to make sure the ip:host:protocol are still valid. Finally a way to be notified of any changes.
Building it
Just have to say, it ...
-
Been Working on a High Performance Async Low Latency RPC Library
Updated at March 03, 2019 at the 19th hour
Categories
Lately I've taken an interest in facilitating Remote Procedure Calls (RPC). Spawned from service management (mgmt) application I've been building in Rust. Along with that service management system, I built an in memory db for it in Rust as well. Very interesting stuff for me espcially Rust. More on that some other time.
What is an RPC?
RPCs are used to communicate across applications. Like if I wanted to get a user's email then I would submit a GetUserEmailRPC Request and get back a response at some point (or timeout if too long). If you think about HTTP, it is what browsers use to communicate with a server in order to get the data that makes up what you see here.
My Use case
In my case my service mgmt application will use RPCs to communicate with any application that connects to it. Most of the RPCs should be a PingRPC just to make sure that the other application is still communicating other wise it is considered down. To expand the use, I always wanted a way to connect to any of my applications and draw some data out of them (pub/sub). Namely, metrics and maybe logging in a linux ...
-
Building Java 9 and 10 Modules with Gradle
Updated at September 24, 2018 at the 02th hour
Categories
Introduction
Java 9 introduced the Java Platform Module System a.k.a. JMS, JPMS and Project Jigsaw. With JMS, classloading and visibility are much different than before with modules being the way to import and export classes and resources. There are articles out that that explain the changes. Mark Reinhold made a post about JMS. What are the benefits? Segmentation, smaller build sizes. Let's talk bout Gradle.
Gradle is a step above Maven as a build system. With incremental parallel builds, it is super quick! Gradle has performance analysis features such as build scan and profile reports to help with figuring out what is taking so long. The one thing about Gradle though is that it is a little obtuse sometimes. Take the time to learn Groovy since it is a superset of Java.
So, how do we build a Java module for a single project?
Let's say our project root starts from /Project. In the root is src and build. src contains main and etc. the usual directory structure.
We need to declare our module with the exports and imports.
Module File Path: /Project/src/main/java/module-info.java
module com.newcompany.project {
requires java.sql
exports com.newcompany.project.common;
}Let's configure gradle: /Project/build.gradle
Gradle, under the hood, will ...
-
Storing Creation and Updated Times Metadata in MySQL
Updated at March 17, 2018 at the 15th hour
Categories
A common use case: Let's assume you have an accounts table and you want to know when an account was created or last edited. or you want a log table that stores created time.
First thing I would think of is what fields and what data type do I need? created_on BIGINT and updated_at BIGINT.Â
Well, why not use TIMESTAMP instead?
- The year 2038 problem! Only 20 years t o go. A MySQL task is create d to track expanding the field. I'm sure it will get solved via database update, but until then... yeah.Â
- I store milliseconds in database by default and TIMESTAMP doesn't support that as far as I know, call it cargo culting!
- I like comparing numbers rather than relying on a timestamp data type.
There is nothing wrong with using TIMESTAMP and it comes down to personal preference. It makes absolute sense to use TIMESTAMP as you can take advantage of MySQL's automatic initalization feature, which means the database manages these two fields and it is a happy day for a developer.
Back to BIGINT
So we have two fields that are bigint. Now I need to make the decision on whether this will be ...
-
I Deployed Today and Recommend react-draft-wysiwyg as a text editor
Updated at March 11, 2018 at the 18th hour
Categories
Alright, so I deployed a new version of the blog to support canonical URLs, new text editor for the backend, and fixed how lists get shown in a post.
Fixed list numbered elements going out of bounds.
Very small update, but lists' numbered elements were showing out of the post boundaries and all that needed to happen was modify the list css to push in the numbered elements back into the boundaries. Took a google search to find the solution unfortunately.
Canonical URLs (SEO)
I've learned that canonical urls are important in telling a search engine which page is the authoritative page. It makes sense to push it to the website owner considering a search engine would have to randomly pick a page out of "duplicates". Duplicates cause your search engine rank to drop or pages will be de-indexed all together. For some reason, a url ending in / and without are seen as different. As a human, I say it makes no difference. It makes a big difference path-wise. / could lead to an index page while without the '/' it would be a folder. The best way to deal with that is to just redirect from / to ...
-
Why I Think Flutter Looks Amazing!
Updated at March 03, 2018 at the 17th hour
Categories
It has been a while for me to get excited about a language. PHP was the first language I learned, but I wasn't excited about it. It wasn't until I discovered PHP 5 and OOP that I felt an epiphany! C++ and Java really take the cake for amazing languages. Most recent has been NodeJS with async await! A new contender has come on to the field!
Flutter, a cross platform framework built on the Dart language. It is used to build iOS and Android apps! Usually, when I see this cross platform claim, I say it is too good to be true, but I looked at the architecture of the framework and I like how easy it is to define UI components and styles, that I can't resist trying this!
You can read about why Flutter used Dart (https://hackernoon.com/why-flutter-uses-dart-dd635a054ebf) to gain some reasoning behind their choice.
I remember Cordova, which I think was ok for some apps, but generally crappy looking and because of that you could tell the difference between a native and Cordova app. I think because I had a crappy experience with Cordova that the idea of compiling to JS put me off. There is React ...
-
Generate SQL queries to update collations for all tables in a DB
Updated at March 03, 2018 at the 15th hour
Categories
Found this gem on another website:
SELECT CONCAT('ALTER TABLE ', tbl.TABLE_SCHEMA, '.', tbl.TABLE_NAME, ' CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;') FROM information_schema.TABLES tbl WHERE tbl.TABLE_SCHEMA = '<TABLENAME>';
This query will generate sql statements to alter all tables in a MySQL db to utf8mb4 collation.
-
Data Collection, Monitoring and Visualization
Updated at February 17, 2018 at the 04th hour
Categories
For the past two years, I've been focused on the problem of moving data from sensors on networked equipment to data sinks for aggregation, monitoring and visualization purposes. The process is what I call a pipeline that achieves the goals of monitoring and visualization.
Some terminology I use are measurement which can be substituted with metric. Observations which are a collection of measurements.Data Collection
Collecting observations is done by agents. Agents can setup a receiving server (stream), poll data sources or subscribe to a data source (stream). Observations may be processed/transformed here a bit to standardize/normalize measurements since other agents may be collecting the same measurements from different sources.
Once you have the data, you have a good amount of choices on what to do with the data, so here are two: you can transfer immediately to the data sinks or store into a cache where a curator service will push to data sinks.
Before transferring, some more data processing in the transmission process may be needed for formatting purposes.
If data is transferred immediately, then you have to worry about questions such as backfilling data if the sink goes down!
If data is in a cache, how much ... -
I'm surprised at the lack of Media Asset Management software
Updated at February 17, 2018 at the 04th hour
Categories
Few resources on the topic
-Â http://www.dpci.com/blog/making-case-digital-asset-management
It is odd that not many seem to be doing this. I would think a centralized repository of assets and way to upload things would be in order. Instead of having some weird asset pipeline in nodejs, I could submit all of my assets into a repo with some versioning scheme to follow. I could update assets and/or push to any CDN via a UI. Sharing assets between projects. Asset Discovery. etc. I have notes on this and what I desire. Really would like to make the development environment experience easier with regards to assets.
Seems like a lot of benefits over any potential downsides. I'm looking into making my own software, but I'm likely not to open source it as I will not have any free time to cover other's use cases.
I would say go use Cloudinary if you can afford it. Seems to be the closest thing to asset management with CDN push.So Why not a CMS like Drupal or Wordpress as the DAM?
Resources:
-Â https://www.isovera.com/blog/digital-asset-management-systems-vs-drupal-whats-dam-difference
It is a good question given some of the perceived and actual benefits. You get media, auth, content serving in ... -
Performant and Lock-free do not mean what you think
Updated at February 17, 2018 at the 04th hour
Categories
I think of words that have a lack of definition like performant as an indicator of whether an author knows what they are talking about. "Performant" ain't a word and even if it was what does it mean? High performance? By what measure?
Avoid using "Performant": https://english.stackexchange.com/questions/38945/what-is-wrong-with-the-word-performant
Along those lines, lock-free != wait-free and I would/will forgive people for this. Lock-free sounds nice, but the definition does not imply non-blocking or wait-free. Wait-free is not typically used to describe any algorithms, because guaranteeing wait-free anything is hard.
The differences are noted here: http://concurrencyfreaks.blogspot.com/2013/05/lock-free-and-wait-free-definition-and.html -
Efficient LRU Cache in Java
Updated at February 17, 2018 at the 04th hour
Categories
Fun fun stuff to do in lintcode. I mean we can always incur a linear traversal using a LinkedList if one wanted a simple way to do LRU, but why not challenge myself.public class LRUCache {
public class LRUNode {
Integer key;
Integer data;
LRUNode prev;
LRUNode next;
public LRUNode(int key, int data) {
this.key = key;
this.data = data;
}
}
LRUNode head;
LRUNode tail;
Map nodeMap = new HashMap();
int maxCap;
/*
* @param capacity: An integer
*/public LRUCache(int capacity) {
// do intialization if necessary
this.maxCap = capacity;
}
/*
* @param key: An integer
* @return: An integer
*/
public int get(int key) {
// write your code here
LRUNode node = nodeMap.get(key);
if(node != null) {
this.remove(node);
this.push(node);
return node.data;
}
return -1;
}
/*
* @param key: An integer
* @param value: An integer
* @return: nothing
*/
public void set(int key, int value) {
// write your code here
if(nodeMap.containsKey(key)) {
LRUNode oldNode = this.nodeMap.get(key);
this.remove(oldNode);
}
LRUNode node = new LRUNode(key, value);
this.push(node);
if(nodeMap.size() > this.maxCap) {
this.removeOld();
}
}
private void push(LRUNode newHead) {
if(this.head != null) {
newHead.prev = null;
newHead.next = this.head;
this.head.prev = newHead;
}
if(this.tail == ...