After a year of ignoring the social portions of the Interwebs, it’s time to get back and work on virtual worlds.

This is mainly a test of commenting with Disqus. We’ll see if I like it.

Comments

I have a project that will be multiple, independent programs that network and message to each other. The connections between the modules will be tuned to the particular data types and requirements (size, latency, etc). But, control of the whole allocation needs some sort of ‘control plane’ for the setup, configuration, and control of all the programs. A message bus seems the right thing to send notices and updates around the whole distributed application.

I’m thinking some pub/sub system. Some criteria are:

• provides asynchronous notifications and status between network distributed programs
• low volume: after setup, messages will be rare
• light weight: both the clients and servers needs to be small and simple
• easy protocol: I’m not building a mission critical infrastructure here
• don’t need message ordering
• reliability and non-duplication would be good
• clients available in many different languages (C++, Javascript, Java)
• doesn’t require a ‘cloud service’: there are notification services that could be used but that should be an option and not a requirement

First candidate is MQTT. MQTT is simple-ish. Has lots of clients (like Eclipse/Paho and Mosquitto) and has several hardened brokers (Mosquitto, Moquette, Mosca, HiveMQ, and even multi-protocol servers like Apollo or [AMQP]). Conceptually a good simple and good model but there are gottchas in most of the clients and the brokers tend to be heavier than I need.

Could use node.js and socket.io but that needs a server running somewhere.

WAMP (Web Application Messaging Protocol) is another possibility. It is built around websockets but it basicily just needs a reliable connection (TCP). WAMP provides both pub/sub and RPC communication patterns which might be useful. It only moves JSON messages (text only) although it can use MsgPack for compressed/compacted messages. For a control plane message system, I don’t think I need binary so that is not a required feature.

Now, thinking about it, firewalls will always be a problem when running a centralized message service. If the distributed application is started on a tablet, it will start other components both locally (inside the firewall) and remotely (outside the firewall). If the message server is started with the application inside the firewall, any component outside the firewall would have problems connecting back to the ‘control plane’ server.

More discussion to come.

A base design decision is the inter-module protocol. The design of the 3D server is for multiple content sources to talk to the one renderer so the wire protocol definition is foundational. To complicate any selecion, wire protocols have been exercised and libraried many times and it would be foolish to start from scratch. But there are so many to choose from.

So, what are my requirements?

• High performance (going to move a lot of very small datas)
• Both RPC and streaming messages (non-blocking messages)
• Binary data (meshes and textures will be involved)
• Have a definition language (can define and possibly verify API and protocols)
• Adaptable to many languages (at least JavaScript and C# at the beginning)
• Have stub/interface API generation (for defined mapping from application to protocol)
• Allow versioning (the protocol definition on the first day will need changes)
• Multiple and extensible transports (will need networked, inter-process, and inner-process)

There seem to be two classes of interface definitions on the Internets at the moment: the pure REST interfaces (defined with something like RAML or Swagger) and the more API oriented ones (like Thrift or Protocol Buffers).

There is always roll-your-own which has some advantages in that one can customize the protocol specifically for the application and use case:

• balance the use of TCP and UDP depending on latency and interaction;
• specialize the formatting and compression of the line protocol for the application’s data; and
• specialize the message type queuing and throttling depending on the user experience. This is the approach taken by High Fidelity and many gaming systems. This is a trade-off of development work vs on-going support of an external package.

I don’t want applications building JSON strings and interpreting the HTTP error responses. Wrappers and libraries are necessary.

As described in Igor Anishchenko’s presentation PB vs Thrift vs Avro, ProtoBuffers and Thrift are pretty neck-and-neck as far as features and speed but Thrift has a slightly wider distribution of supported languages. They both support binary data, have protocol definition languages, and support protocol versioning.

Avro is a newer protocol but, rather than pre-compiling the code stubs, it evaluates the schemas and messages at runtime. The feature of schema evolution is considered the winning feature in a post by Jon Zuanich entitled Three Reasons Why Apache Avro Data Serialization is a Good Choice for OpenRTB. The three reasons are schema evolution, untagged (smaller) data, and dynamic typing.

A case against using Protobuf for transport in REST Services makes the argument that the lack of inheritance and polymorphism in Protobuf is a show stopper for game protocols. The article concludes that JSON with the Jackson library is the proper choice.

So, what to decide.

It feels to me like the REST protocols are best for “arms length” relationships between the server and the client while the more API/streaming systems (like Thrift or Protocol Buffers) are for more intimately related clients and servers. This leads me to think the decision also depends on the publicness of the interface.

Therefore, I am going to start by using both. Between the modules that hook together and communicate to share state and information (between the 3D renderer itself and the object modules, for instance) I will start with Thrift. For the asset service, I will use RAML.

I will learn and report how well that works.

My (previous post)1 referred to something called a ‘viewer’. Let me describe what I mean by that.

For a person to look into a virtual world, they need some device and some software to render a view into that virtual world onto that device. These days, the hot devices are head mounted, stereographic displays (see Oculus, GearVR, and many more) but 2D monitor screens and tablets and phones do the job.

There is software that creates the pixels for the device. This software must convert whatever representation the virtual world keeps its contents in into a coherent three dimensional view for the device. There are several very hard problems here. For instance, if the device is a head mounted display that is supposed to give an immersive 3D experience (the user is supposed to fell like they are “in” the 3D world), the view has to move with low latency with the head so as to not cause nausea.

These are problems of the view (latency, rendering, frame-rate, …) which don’t have a direct relationship with the actual content of the world (indirect in that types of content will cause different frame rates and such). But this leads me to think of the ‘viewer’ as separate from the content and even separate from the user interactions.

The X Window System was that for 2D windows – it provided a display server to which multiple window applications connected to. One of the applications was a windows manager which created the window decorations (title bar, scroll bars, …) and handled the user interactions (window movement, mouse control, …). In the X Window System, both the content and management were outside the viewer. This makes them distributable (could run on different computers, …) and modular (multiple window managers, …). The X Window System viewer focuses on efficiently displaying the 2D windows and lets everything else happen in the external applications.

What if a 3D version of this was built? Not just a version of the 2D windows viewer that draws 3D but a viewer built to fetch and display 3D objects and display them in a 3D space for some display. The viewer would handle efficiently displaying the right thing for the particular display and there is no reason why there couldn’t be completely different viewers for different displays.

Behind the viewer would be all the applications that present the 3D objects to the viewer as objects in a space to be displayed. Another application would manage that space for the user – whatever that management should be. And there could easily be different space and interaction managers.

In future posts I’ll talk about this viewer model and how I believe it is applicable for augmented reality and virtual worlds.

Over the next few months I will be working on a project I’m calling “Basil”.

Basil has grown out of my experiences with virtual worlds and games and thinking about how one could build a viewer to look into virtual worlds and, in particular, in a way that the virtual worlds could mix and combine.

OpenSimulator grew around the Second Life(r) architecture with a central world simulator and multiple viewers connected to same. OpenSimulator extended that model by allowing people to host their own virtual world spaces (“regions”) on their computers while connected to common services (account, asset, location, …).

That model has been further extended by the ‘hypergrid’ model which allows a session to teleport to loctions in other grids.

This has spawned a multiplicity of OpenSimulator based grids and virtual worldness has flourished.

But that is for just one type of grid. Others have sprung up and they each present their closed world with no way of hypergridding between. Continents have formed in the virtual world and there is no way of travelling between.

So, my question is: how could one build a virtual world viewer where an OpenSimulator avatar could stand next to a HighFidelity avatar?

I will describe my approach in future posts. Eventually, the GitHub project will go public.