Not much new running code to report today.
There are those days when you just code infrastructure and new running features don’t come about. They will soon.
The previous version of the browser version of Basil had all its pieces integrated into one module. To effect the distributed Basil architecture, I need to create the messaging system that goes between the modules. That means improving the Basil protocol and coding all the Flatbuffers message construction and decoding code.
Today’s work (besides wrestling with email setup) has been defining the Basil protocols using FlatBuffers. Mainly because someone in the project I’m working on has discovered them as a (in theory) faster way of serializing and de-serializing protocol messages.
FlatBuffers were created by some engineers at Google as a faster alternative to Protocol Buffers. FlatBuffers have a much simpler schema and the serialization format lays down the data in a data natural way so, at the receiving end, the program does not need to copy the data but have just be passed a pointer to the data in the message buffer. All this while still having optional fields and downward compatibility.
The largest problem I’ve found so far is a lack of documentation. For instance, there is a ‘union’ of tables definition in the schema. This defines a field that can be one of a selection of ‘tables’ (the schema defines ‘table’s and ‘struct’s with the latter being quicker and more compact but not extensible). The receiver of a message that includes a ‘union’ must know what was put in the field so it can parse the correct table contents. Well, there is an enumeration automatically generated and a special function that is generated so the receiver can know that. But where is the documentation? No where that I found.
In the next day or two I’ll post some code examples so people Google’ing will find some working definitions and code.
I’ve been enjoying retirement but I have come to feel that I am missing some organization. So, while spending a lot of time enjoying life, I have set the goal of spending half-a-day producing something.
Most anything, really.
I have some home business to take care of but, other project-wise, I have lots of ideas for virtual world/augmented-reality work which I will putter along with.
So, I have two things to focus on: 1) producing something new every day (could be research or code), and 2) writing a blog entry everyday about what I learned or coded.
Onward and upward.
I need to create a web page for Basil’s WebGL version so I started looking at the JavaScript world. For a humorous and insightful take on what I’m getting into, read How It Feels to Learn JavaScript in 2016.
I want to include modularized files and, since I had experience
with NodeJS modules, I figured I would just need to add a few require()s
to my program.
That’s not the case.
While NodeJS has a (relatively) simple require() for appliction module
inclusion, the reigning solution on the browser side is RequireJS.
Coming from NodeJS’s implementation, RequireJS is a complex and confusing
wall of specifications. Using RequireJS means a different wrapper for
modules, a config specification for paths, shims, and whatever.
Sheesh. What’s all this stuff I have to learn? Why all the complexity? Why is there a whole new framework I have to learn in order to modularize my web page?
RequireJS does have a way of handling CommonJS modules but it requires manual conversion of all the modules. That’s not going to fly.
Another solution is Browserify which reads all the JavaScript files, interprets the
require()s and creates a single bundle to include in the web page.
This does solve part of the async loading problem but introduces a step that
would be different between developement and production.
The main difference between NodeJS’s require() and RequireJS is that the
former uses JavaScript language features
to do its thing while packages like RequireJS introduce a new level of abstraction and
complexity. Do I really need to learn a new framework in order to get asynchronous
loading? It seems so unnecessary.
Other people have wanted a simple require() (my unhappiness is shared) and
they have created some alternatives.
Some discussion in Relation Between CommonJS and RequireJS
(http://stackoverflow.com/questions/16521471/relation-between-commonjs-amd-and-requirejs),
Some simple solutions are given in Node Style Require for Browser JavaScript.
I am giving NodularJS and Smoothie a try.
Others:
And don’t get me started on the crush of JavaScript package managers (Npm, Bower, Grunt, Gulp, Broccoli, Mimosa, webpack, rollup and it goes on and on). A nice summary of choices.
A while back I sent an email describing some of the features of BulletSim – the physics engine I did for OpenSimulator. This is useful information so it should be on the web somewhere:
The C# part of BulletSim can be in addin-modules – it doesn’t need to be’in core’ but needs to be built with core so it can be an addin module.
There is a separate OpenSimulator source tree… opensim-libs at “git://opensimulator.org/git/opensim-libs” that has a bunch of the non-core parts of OpenSimulator (http server, old and ancient other tries at he physics engine, …). The C++ portion of BulletSim is in ‘opensim-libs/trunk/unmanaged/BulletSim’ and there are the instructions for fetching the Bullet sources, patching same, and then building with the interface to the C# code). The C++ wrapper mostly deals with passing the structures back and forth between the C# and C++ code (pinned memory for the position updates and collisions, copying meshes in arrays of floats, …)
The BulletSim design is around making a simulation step be only one transition between C# and C++. So, under normal running conditions, there is only one transition per simulation step and the data (position updates and collisions) are passed in pinned memory so there is no copy. 98% of the C# code deals with doing and adapting Bullet to what OpenSimulator required (link sets (ugh!), …). The C# -> C++ interface for BulletSim is rather large… physics engines seem to have lots of calls for all their features Bullet, for instance, has what seems like zillions of methods of changing constraint parameters I made those appear in the interface to C#. If I had it to do over again, I’d probably go more with a functional design where there is a “call a named function with parameter blob” design so the C#/C++ interface was smaller and new function could be added without changing the binding of the DLL then use some fancy reflection to build the binding on both sides
The .NET C#/C++ binding is pretty good except that ints and booleans change size between 32 and 64bits… if you look at the BulletSim interface you’ll see I use floats and arrays of floats everywhere because they are always 32 bit.
I recently played with building “BulletThrift”… a version of BulletSim that uses Thrift to call a remote process physics engine (experiment in distributed physics). It didn’t get finished mainly because the existing interface to the C++ module is so large. BulletSim actually has a HAL to access the physics engine and there are two physics engines: the C++ Bullet and a C# port of Bullet. The latter was last used by Nebadon to run OpenSimulator on a Raspberry PI. But this also means it is easy to add a link to a remote Bullet. That’s where I was going to add BulletThrift that would call across the network to a remote Bullet server. My main reason for doing this was to be able to run Bullet in a pure C++ environment where debugging wouldn’t be complicated by the managed/unmanaged environment.
If you distributed the physics engine, operationally, I’d expect you’d see some of the things that happen when running BulletSim on its own thread like jitter caused when there is a ‘beat’ between the physics simulation time and the simulator heartbeat. BulletSim running on its own thread means that the physics engine is called on its own thread and the passing back of collisions and position updates happens when the simulator heartbeat thread calls into the physics engine.