Apr 9, 2017 - Flatbuffers tables in tables

Forgot to push publish.

Besides having some family over to the house for dinner, today’s coding was even more Flatbuffers coding.

I’m in the trough of disillusionment with FlatBuffers.

For FlatBuffers, one writes a schema description of records. The tool then generates serialization and deserialization code for the target language. The coolness comes from the the streamlined deserialization which minimizes data reformatting and copying.

But the programming interface to these serialization routines is a tad clunky. Firstly, you never know what type of serialization interface it is going to generate. Sometimes it will generate a single create call to build a structure and sometimes it generates a ‘start’ and ‘end’ function with ‘add’ functions for the fields. It took me a while to realize the interface difference comes between ‘struct’ and ‘table’ definitions. Another factoid that is not explained in the documentation.

Another thing I’m learning is that the serialization interface gets complex if one defines a protocol with ‘table’s inside ‘table’s. Using the schema, it is easy to define a record that includes several tables. The problem comes in the complexity of the record generation code – table generation code embedded in table generation code embedded in table generation code. The make-a-record subroutine becomes long and complex. Additionally, if a field is ever added to a table, code in several different places will need modification.

It would be nice if the code generator created some higher level code.

Comments

Apr 8, 2017 - Coding Flatbuffers

Not much new running code to report today.

There are those days when you just code infrastructure and new running features don’t come about. They will soon.

The previous version of the browser version of Basil had all its pieces integrated into one module. To effect the distributed Basil architecture, I need to create the messaging system that goes between the modules. That means improving the Basil protocol and coding all the Flatbuffers message construction and decoding code.

Comments

Apr 7, 2017 - FlatBuffer Documentation

Today’s work (besides wrestling with email setup) has been defining the Basil protocols using FlatBuffers. Mainly because someone in the project I’m working on has discovered them as a (in theory) faster way of serializing and de-serializing protocol messages.

FlatBuffers were created by some engineers at Google as a faster alternative to Protocol Buffers. FlatBuffers have a much simpler schema and the serialization format lays down the data in a data natural way so, at the receiving end, the program does not need to copy the data but have just be passed a pointer to the data in the message buffer. All this while still having optional fields and downward compatibility.

The largest problem I’ve found so far is a lack of documentation. For instance, there is a ‘union’ of tables definition in the schema. This defines a field that can be one of a selection of ‘tables’ (the schema defines ‘table’s and ‘struct’s with the latter being quicker and more compact but not extensible). The receiver of a message that includes a ‘union’ must know what was put in the field so it can parse the correct table contents. Well, there is an enumeration automatically generated and a special function that is generated so the receiver can know that. But where is the documentation? No where that I found.

In the next day or two I’ll post some code examples so people Google’ing will find some working definitions and code.

Comments

Apr 6, 2017 - Focus on Producing

I’ve been enjoying retirement but I have come to feel that I am missing some organization. So, while spending a lot of time enjoying life, I have set the goal of spending half-a-day producing something.

Most anything, really.

I have some home business to take care of but, other project-wise, I have lots of ideas for virtual world/augmented-reality work which I will putter along with.

So, I have two things to focus on: 1) producing something new every day (could be research or code), and 2) writing a blog entry everyday about what I learned or coded.

Onward and upward.

Comments

Jan 13, 2017 - Browser require

I need to create a web page for Basil’s WebGL version so I started looking at the JavaScript world. For a humorous and insightful take on what I’m getting into, read How It Feels to Learn JavaScript in 2016.

I want to include modularized files and, since I had experience with NodeJS modules, I figured I would just need to add a few require()s to my program.

That’s not the case.

While NodeJS has a (relatively) simple require() for appliction module inclusion, the reigning solution on the browser side is RequireJS. Coming from NodeJS’s implementation, RequireJS is a complex and confusing wall of specifications. Using RequireJS means a different wrapper for modules, a config specification for paths, shims, and whatever.

Sheesh. What’s all this stuff I have to learn? Why all the complexity? Why is there a whole new framework I have to learn in order to modularize my web page?

RequireJS does have a way of handling CommonJS modules but it requires manual conversion of all the modules. That’s not going to fly.

Another solution is Browserify which reads all the JavaScript files, interprets the require()s and creates a single bundle to include in the web page. This does solve part of the async loading problem but introduces a step that would be different between developement and production.

The main difference between NodeJS’s require() and RequireJS is that the former uses JavaScript language features to do its thing while packages like RequireJS introduce a new level of abstraction and complexity. Do I really need to learn a new framework in order to get asynchronous loading? It seems so unnecessary.

Other people have wanted a simple require() (my unhappiness is shared) and they have created some alternatives. Some discussion in Relation Between CommonJS and RequireJS (http://stackoverflow.com/questions/16521471/relation-between-commonjs-amd-and-requirejs), Some simple solutions are given in Node Style Require for Browser JavaScript.

I am giving NodularJS and Smoothie a try.

Others:

And don’t get me started on the crush of JavaScript package managers (Npm, Bower, Grunt, Gulp, Broccoli, Mimosa, webpack, rollup and it goes on and on). A nice summary of choices.

Comments