I’ve been suffering from analysis paralysis. What I envision for Basil and ultimately the Herbal3D system is a huge project with many components. These days, with the Internet and all the collaborations and projects happening, there are innumerable technologies to choose from. What languages to use? What IDE to use? What messaging library? And on and on and on and on and on. Argh! There are so many to choose from!!

Previous blog posts (Pesto to Python, Cassandra and Docker, Looking for a message bus, Thirft vs ProtoBuff) have all been about analysing various software libraries and packages. The end effect is that nothing has gotten done.

Nothing.

Well, some little experiments and some documentation but really no useful code or results.

The next step is to do something. The best thing to do is the prim baking code and the comparison of display frame rates between a browser based viewer and an Unreal Engine viewer. This experiment will create some of the required basic functionality and verify some of my conjectures about the basic Basil architecture. It will also be visual and will hopefully spark interest and thus build a community of developers.

So, rather than worry about transport and APIs, over the next few days I’ll work on conversion of the OpenSimulator prims and objects into “baked” mesh format and in various mesh file formats. Since an OAR file format contains all the asset descriptions as well as the region placement, and, since it is just a compressed TAR file, I will burst an OAR file and write routines to do the conversions and create new files. For display, I can just move that file structure under an HTTP server.

Comments

After sending more time than I wanted on a NodeJS version of Pesto, I came to the conclusion that NodeJS was not the language of choice for that service. This was given away when I found the multi-threaded Thrift server classes for Python.

Sound-of-forehead-slap.

Of course! JavaScript is not a multi-treaded language. Pesto, though, is supposed to be the responsive messaging center of the whole viewer framework. This kinda requires multi-threading.

I originally chose JavaScript/NodeJS because I wanted to build a fancy, interactive, and responsive web interface to Pesto. Guess I will have to do that with some Python libraries. The Python 2 vs Python 3 fork is concerning when thinking of the long term but I’ll have to see how that plays out.

Comments

In my continuing effort to learn all the new technologies, I wanted to use one of the NoSQL databases. Since I want to store geographical data as well as meta data for virtual world objects, I steered away from document oriented ones like ElasticSearch. That leaves ones like MondoDB or CouchDB or pure Hadoop.

The fickle finger of databases then lead to Cassandra. It was used in the Sirikata virtual world project and is scalable and clusterable and continues to be used in a lot of places. It is also available on AWS and other infrastructures so it’s a good candidate.

The two target usages for Basil development are standalone, single computer installations and in larger, production installations. So, how to easily run Cassandra on a single desktop.

Another concern of mine is how to stay up-to-date with the latest versions and sources of any package. I want an answer for ‘what will building and maintaining look like a year from now’.

I pulled the Cassandra sources and looked into building it. It requires a specific version of Java and several other packages. This looked like it was going to be a nightmare to build on general use Linux system and the dependencies would make upgrading difficult. And, like I said above, I wanted the latest and greatest and not the older versions that will be in the Ubuntu package repositories.

But wait. Looking around I found a Docker version of Cassandra. Docker containers would provide some of the isolation of the different versions four all the libraries are going to Cassandra and he would get me again tangled up in other interesting internet technologies.

Docker also provides a solution for creating simple standalone systems as well as complex, production, scalable systems. A simple Docker setup would allow someone to just run a local Cassandra or they could deploy all of the services into the cloud and have a production environment.

So, off down the rabbit hole of learning Docker and learning all of its setup and configuration options.

Comments

Previous posts have talked about choosing an interconnection system for the multiple parts of a Basil viewer session. I’ve started with Thrift and I am learning of the bumps in the road.

First of all, Thrift is not well documented. Since it is a low level transport that was built for larger projects, it makes some sense that the writers of Thrift really care more about the larger project, but, for us people trying to use Thrift in a new project, a bunch more documentation would be a help. Others have started to fill the gap (like Diwaker’s Thrift: The Missing Guide) but there is still a big barrier to getting started.

I started of trying to use Thrift in JavaScript (the Pesto server) and Python (the test routine for same). Both of these languages are not, shall I say, first class languages in the implementation. It seems most of the work on Thrift is for Java and the examples for the other languages show one simple example and leave the reader wondering about installation, configuration, and the exact syntax for including the generated code.

Thrift is perfectly happy with hyphens in names, Python is not. Make sure all your service names use camel case or similar rather than hyphens or underscores.

As of today (August 3, 2016) the latest Thrift sources require TwistedTrial to be installed to build the Python libraries. The documentation does not mention this and it is frustrating in that no error is generated – Python is just not built even with the --with-python configuration parameter. Since TwistedTrial is only needed to run the test suite, I edited configure.ac to remove that requirement. And viola, Python libraries built and installed.

The change is:

builder@sys:~/thrift-git$ git diff configure.ac
diff --git a/configure.ac b/configure.ac
index c72dd43..bcdaec3 100755
--- a/configure.ac
+++ b/configure.ac
@@ -281,8 +281,9 @@ AM_PATH_PYTHON(2.6,, :)
 AX_THRIFT_LIB(python, [Python], yes)
 if test "$with_python" = "yes";  then
   AC_PATH_PROG([PIP], [pip])
-  AC_PATH_PROG([TRIAL], [trial])
-  if test -n "$TRIAL" && test "x$PYTHON" != "x" && test "x$PYTHON" != "x:" ; then
+  if test "x$PYTHON" != "x" && test "x$PYTHON" != "x:" ; then
     have_python="yes"
   fi
 fi
builder@sys:~/thrift-git$

More notes are most likely on the way as I write Python code to use Thrift.

Comments

I attended the Portland Virtual Reality Meetup nearly a month ago and, at the time, I made a list of VR technologies that I needed to understand. This is that short list:

WebVR

WebVR is a JavaScript/DOM API standard for interfacing and application to VR input/output devices. The WebVR API specifies data structures and conventions for controlling the headset. This includes focal length, resolution, and timing. The interface is important because the realtime nature of displaying on the quickly moving head display requires a lot of coordination between the display and the renderer.

Hopefully this API will allow browser games and such to adapt to multiple different headsets and controllers.

aframe.io

AFrame is a JavaScript system that specifies HTML extensions to define 3D scenes which are displayed with WebGL. So it is a simple authoring tool for creating 3d scenes. It is fairly new and doesn’t have a lot of controls for very complex scenes (multi-threading, …).

OSVR

OSVR is an effort to build an open source VR system. OSVR on Github. While there are several proprietary systems, this tries to be the open, hackable version. They have an SDK and a 2160 X 1200 dual display headset which is part of their Hacker Development Kit (HDK). The software seems to be built for Windows and Mac. OSVR is maintained by Sensics and Razer.

JanusVR

JanusVR is an attempt to build a 3D browser for the web. Their description is: “JanusVR is a collaborative web browser for Virtual Reality, where links are portals and pages are rooms. The platform is written in C++ with the OpenGL, OpenAL and Qt libraries.”.

The principle innovation seems to be their integration with existing web pages. You can display web pages, navigate the 2D and 3D works through links, and enhance existing web pages so that they display 3D when viewed by the JanusVR viewer. This latter feature includes a definition of an embeddable XML scene description language. Haven’t looked to see if it has any relation to the XML3D definition.

Comments