Feb 4, 2022 - Notes on v-web Metaverse

There has been some discussion on adding metaverse selection to Vircadia-Web (here after “v-web”) and this note talks about how the existing code could be modified.

Currently (as of Feb 4, 2022), v-web has a default metaverse URL that is used to fetch the metaverse information and is used as the metaverse to log the user into. This default metaverse is specified in @Base/config.ts and is connected when the main view is mounted (in App.vue).

The real work connecting to a metaverse is in @Modules/Utility.metaverseConnectionSetup() which calls MetaverseMgr.metaverseFactory() to create a new metaverse connection class instance. This operation also has the side effect of making the network connection to the metaverse, setting the “active” metaverse, and updating the metaverse information in the Vue variable $store. So the overall effect of calling metaverseConnectionSetup() is to change the active metaverse and update all the display variables. The only work needed here is that the previous metaverse class instance is not saved or freed.

Looking at the user interface, there is a “Click to log in to metaverse” text at the top of the page. Clicking this opens the “Metaverse Login” dialog (@Components/dialogs/Login.vue). This dialog has fields for username and password. If this dialog has a “metaverse” field added that had the default metaverse URL preloaded then the user could enter a new metaverse URL if logging into a different metaverse. The action would be, when the user pressed “Login”, the code would check if the metaverse URL had changed and, if so, call metaverseConnectionSetup() with the new URL before logging in the user. This would change the active metaverse and set up the application to use the new metaverse (Explore and other operations use the active metaverse).

As a side note, @Base/config.ts has code to save some variables in the browser’s local storage. If the user is specifying a different default metaverse, there should be some place that does a Config.setItem(DEFAULT_METAVERSE_URL, newURL).


May 29, 2021 - ROS2 Sending Image

I don’t know why this is not documented better but…

While I was porting my old ROS2 Python test application to the newest ROS2 Foxy on my Raspberry Pi’s, I discovered that I needed to use OpenCV to capture images because picamera does not work in 64 bit](https://github.com/waveform80/picamera/issues/540) .

What’s not documented very well (and I spent several days figuring out) is that, after one has ported to OpenCV, converting an OpenCV image into a ROS2 message is not obvious. After a bunch of Google’ing, one discovers that the ROS2 folks have packaged all the magic into a module called CvBridge.

OpenCV keeps images in cv:im structures and converting that into a byte array that fits into the ROS2 message is tricky. Actually, what’s expected in a ROS2 image message is not documented very well so what works is hard to figure out in the first place.

The ROS2 developers have created some tools for the OpenCV users and one is a tool that converts OpenCV images into ROS2 image messages. So, the code ends up looking like:

from sensor_msgs.msg import Image
import cv2
from cv_bridge import CvBridge
self.bridge = CvBridge()
self.camera = cv2.VideoCapture(0)
self.frame_num = 0
self.publisher = self.create_publisher(Image, "topic", 10)
ret, frame = self.camera.read()
if ret == True:
    msg = self.bridge.cv2_to_imgmsg(frame, 'bgr8')
    msg.header.frame_id = str(self.frame_num)   # 'msg' is a ROS2 sensor_msgs/Image.msg
    self.frame_num += 1

That is, don’t try to create the image message yourself. Instead, rely on the CvBridge package to create the whole message and then fill it in with other information.


May 10, 2021 - Iamus Walkthrough

Since July of last year, I have been coding a metaverse-server for the Vircadia virtual worlds. The Iamus metaverse-server acts as the coordination point for the independent domain-servers which host the virtual world space.

When High Fidelity open-sourced the code for their virtual world, they kept the central coordination services private so a new service for accounts and discovery was needed.

I created Iamus with NodeJS, ExpressJS, and MongoDB.

For anyone picking up the project and making modifications, there is written documentation on the server’s design, building, running, running as a Docker image, and various notes.

In addition to the written documentation, I created a video walking through the internals of the code:

This should give anyone modifying Iamus an good start.


Apr 11, 2021 - Organizing 2021

I’ve been generally offline for the last year. Doesn’t have anything to do with the pandemic or anything going on in the world. Just me not focusing on computers. An odd thing after all these years.

Some of the time has been spent thinking about what I want to be when I grow up. I have been thinking about formulating a 5 year plan if only so the years don’t go by with a “where did all the time go” sort of outcome.

So, where do I want to be in a few years?


In the short term, I have three software/development projects: Vircadia, Robotics, and Basil.

Vircadia Project

The Vircadia Project is an open-source virtual-world grid, server, and viewer system derived from the High Fidelity business. I wrote the Iamus metaverse-server that handles centralized domain-server registration, user accounts, and other grid level functions.

A year ago I wrote in Vircadia Directions how I wanted the project to succeed and thus jumped in to help. A year later, there is lots of work still to be done and I have lots of other projects I want to work on.

Robotics Project

I am interested in “artificial intelligence” and I want to manifest it in the real world. To that end, I have a TurtleBot3 and a Petoi Bittle for ranging around the house.

My real focus is on their intelligence. There is a lot of AI work around neural networks, training, auditing, and otherwise doing vision and context classification. Once I get these robots working, I’ll be developing a reason for them to be around the house.

Basil Project

For many years, I have been hacking on my [Herbal3d] and Basil projects. I still think it is the correct architecture for an eventual VR and AR world but I just haven’t completed it. I keep rewriting and treaking the code.

In one vlog I watched some time ago, the vlogger promoted “Always finish your projects”. There is some sense to that as directories and workbenches full of half finished projects are depressing.

I haven’t given up on Basil.

Real Life

And then there is the garden (it’s Spring!) and the house and having relationships with the other people in the house. Reality takes up a lot of time.

Social Presence

Another question is what I should do for a social presence. When I was working, I loved being part of a group of like minded people working on a goal. I would like to build up a collection of people and be part of that collection. That means, at least to start, a social media presence.

I think that means picking a platform and then commiting to generating content. The content should also be every day or close to that to create a changing narrative that people can follow and, hopefully, join in to.

Which platform? There is this blog but there are the main stream people aggregators (Twitter, Facebook, Instagram, …) as well as the less data-collecting, non-ad-supported platforms (Mastodon, WeMe, …). There are the real-time collaboration systems (Discord, Slack, …) and a hundred different categories and platforms I don’t even know exist.

Besides finding other like-minded people, the other purpose for a social presence is forcing me to do and then report something every day. It would be a forcing function to keep me making progress and doing something reportable often.

So, I need to choose a place where I’d be visible and where I’d feel like I’d need to keep up-to-date because people would see me.


Aug 11, 2020 - ActivityPub and Matrix for Vircadia

As I discussed in my ideas on Vircadia Directions, it would be cool to implement virtual world federation using existing social networks. I discussed ActivityPub but the other one mentioned is Matrix.

Both ActivityPub and Matrix are open-source, non-centralized federation systems with active and busy communities. But they each seem to be built for different appliction designs and have different architectures and uses.

Let me discuss each.


Matrix is a decentrialized infrastructure where anyone can run a “home-server” that links into the larger synchronization space. The collection of “home-server”s implement “rooms” which contain JSON objects that are signed and securely synchronized across the home servers. Matrix design leans toward security and authorization so there is a strong emphasis on identity and encryption.

The base communication system is JSON sent between clients and servers using HTTP REST operations, the model presented is of “rooms” that are magically distributed across all of the connected home-servers. Of course, there are optimizations so rooms are only synchronized across home-servers with clients that are listening to those rooms. Encryption and identity management prevent snooping and corruption.

Matrix application model leans toward the concept of chat rooms with decentralized infrastructure thus applications that mimic Discord or IRC are easy to implement. But the infrastructure has been generalized into a de-centralized, JSON object synchronization system which enables applications like IoT sensor devices reporting status and many more applications.

One downside is that, even though room contents are synchronized between homeservers that have room clients, the room contents themselves “live” on that room’s homeserver. If that one homeserver is down, then the room is unavailable. This means Matrix is a distributed communication system rather than a distributed data system.


  • Secure encrypted communication.
  • “Rooms” could fit with chatting in domains.


  • Not completely distributed
  • Adding a metaverse-server to the federation would involve connecting it to a Matrix home-server;
  • One needs a “matrix account”. When a user creates a metaverse-server account, the metaverse-server would have to create an account on a home-server


ActivityPub is a W3C standard (the group that does the web application level standards like HTML, CSS, SVG, URL, RDF, …) that defines client-to-server and server-to-server interactions which implement a decentralized “social network”. Through multiple JSON-formatted HTTP requests, content in “streams” is added to, read, edited and deleted while activity notifications and content is distributed.

The definitions of ActivityPub is very much around “micro-blogging” (think Twitter). ActivityPub defines “actors” who each have an “inbox” and “outbox”. An actor POSTs a message to their outbox and the message is sent to the inboxes of addressed recepients. Actors can also have “collections”. A defined collection is “followers” thus a message can easily be addressed to one’s “followers” (like Twitter). This would be like a Tweet that shows up in the “streams” of the receivers (if a user views their inbox like user’s view their Twitter stream).

This is how Mastodon implements a distributed micro-blogging service where many servers make up a system where one could “follow” people and bots from other servers and view one’s own stream of subscribed posts.

Using other W3C standards, identities are distributed, referenced “names” can be dereferenced WebFinger into URLs to inboxes and profiles, messages can be authorized HTTP Authentication and verified HTTP Signatures.

Several services have adapted the microblogging model to other publishing services like PeerTube and PixelFed.

ActivityPub has the same architectural feature as Matrix in that an actor/collection/inbox/outbox all “live” on one server and when that server is down, that actor cannot be accessed.

ActivityPub keeps content encrypted in transport using TLS connections but end-to-end message encryption would have to be added. This is a feature that is built into Matrix.


  • Identites can easily be created and shared across federated servers
  • Actors and collections could fit with chatting in domains and groups.
  • REST/JSON APIs allowing scripts easy access to users, actors, …


  • Not completely distributed
  • Would have a lot of URL traffic between metaverse-servers