I’ve been generally offline for the last year. Doesn’t have anything to do with the pandemic or anything going on in the world. Just me not focusing on computers. An odd thing after all these years.

Some of the time has been spent thinking about what I want to be when I grow up. I have been thinking about formulating a 5 year plan if only so the years don’t go by with a “where did all the time go” sort of outcome.

So, where do I want to be in a few years?

WIP

In the short term, I have three software/development projects: Vircadia, Robotics, and Basil.

Vircadia Project

The Vircadia Project is an open-source virtual-world grid, server, and viewer system derived from the High Fidelity business. I wrote the Iamus metaverse-server that handles centralized domain-server registration, user accounts, and other grid level functions.

A year ago I wrote in Vircadia Directions how I wanted the project to succeed and thus jumped in to help. A year later, there is lots of work still to be done and I have lots of other projects I want to work on.

Robotics Project

I am interested in “artificial intelligence” and I want to manifest it in the real world. To that end, I have a TurtleBot3 and a Petoi Bittle for ranging around the house.

My real focus is on their intelligence. There is a lot of AI work around neural networks, training, auditing, and otherwise doing vision and context classification. Once I get these robots working, I’ll be developing a reason for them to be around the house.

Basil Project

For many years, I have been hacking on my [Herbal3d] and Basil projects. I still think it is the correct architecture for an eventual VR and AR world but I just haven’t completed it. I keep rewriting and treaking the code.

In one vlog I watched some time ago, the vlogger promoted “Always finish your projects”. There is some sense to that as directories and workbenches full of half finished projects are depressing.

I haven’t given up on Basil.

Real Life

And then there is the garden (it’s Spring!) and the house and having relationships with the other people in the house. Reality takes up a lot of time.

Social Presence

Another question is what I should do for a social presence. When I was working, I loved being part of a group of like minded people working on a goal. I would like to build up a collection of people and be part of that collection. That means, at least to start, a social media presence.

I think that means picking a platform and then commiting to generating content. The content should also be every day or close to that to create a changing narrative that people can follow and, hopefully, join in to.

Which platform? There is this blog but there are the main stream people aggregators (Twitter, Facebook, Instagram, …) as well as the less data-collecting, non-ad-supported platforms (Mastodon, WeMe, …). There are the real-time collaboration systems (Discord, Slack, …) and a hundred different categories and platforms I don’t even know exist.

Besides finding other like-minded people, the other purpose for a social presence is forcing me to do and then report something every day. It would be a forcing function to keep me making progress and doing something reportable often.

So, I need to choose a place where I’d be visible and where I’d feel like I’d need to keep up-to-date because people would see me.

Comments

As I discussed in my ideas on Vircadia Directions, it would be cool to implement virtual world federation using existing social networks. I discussed ActivityPub but the other one mentioned is Matrix.

Both ActivityPub and Matrix are open-source, non-centralized federation systems with active and busy communities. But they each seem to be built for different appliction designs and have different architectures and uses.

Let me discuss each.

Matrix

Matrix is a decentrialized infrastructure where anyone can run a “home-server” that links into the larger synchronization space. The collection of “home-server”s implement “rooms” which contain JSON objects that are signed and securely synchronized across the home servers. Matrix design leans toward security and authorization so there is a strong emphasis on identity and encryption.

The base communication system is JSON sent between clients and servers using HTTP REST operations, the model presented is of “rooms” that are magically distributed across all of the connected home-servers. Of course, there are optimizations so rooms are only synchronized across home-servers with clients that are listening to those rooms. Encryption and identity management prevent snooping and corruption.

Matrix application model leans toward the concept of chat rooms with decentralized infrastructure thus applications that mimic Discord or IRC are easy to implement. But the infrastructure has been generalized into a de-centralized, JSON object synchronization system which enables applications like IoT sensor devices reporting status and many more applications.

One downside is that, even though room contents are synchronized between homeservers that have room clients, the room contents themselves “live” on that room’s homeserver. If that one homeserver is down, then the room is unavailable. This means Matrix is a distributed communication system rather than a distributed data system.

Pros:

• Secure encrypted communication.
• “Rooms” could fit with chatting in domains.

Cons:

• Not completely distributed
• Adding a metaverse-server to the federation would involve connecting it to a Matrix home-server;
• One needs a “matrix account”. When a user creates a metaverse-server account, the metaverse-server would have to create an account on a home-server

ActivityPub

ActivityPub is a W3C standard (the group that does the web application level standards like HTML, CSS, SVG, URL, RDF, …) that defines client-to-server and server-to-server interactions which implement a decentralized “social network”. Through multiple JSON-formatted HTTP requests, content in “streams” is added to, read, edited and deleted while activity notifications and content is distributed.

The definitions of ActivityPub is very much around “micro-blogging” (think Twitter). ActivityPub defines “actors” who each have an “inbox” and “outbox”. An actor POSTs a message to their outbox and the message is sent to the inboxes of addressed recepients. Actors can also have “collections”. A defined collection is “followers” thus a message can easily be addressed to one’s “followers” (like Twitter). This would be like a Tweet that shows up in the “streams” of the receivers (if a user views their inbox like user’s view their Twitter stream).

This is how Mastodon implements a distributed micro-blogging service where many servers make up a system where one could “follow” people and bots from other servers and view one’s own stream of subscribed posts.

Using other W3C standards, identities are distributed, referenced “names” can be dereferenced WebFinger into URLs to inboxes and profiles, messages can be authorized HTTP Authentication and verified HTTP Signatures.

Several services have adapted the microblogging model to other publishing services like PeerTube and PixelFed.

ActivityPub has the same architectural feature as Matrix in that an actor/collection/inbox/outbox all “live” on one server and when that server is down, that actor cannot be accessed.

ActivityPub keeps content encrypted in transport using TLS connections but end-to-end message encryption would have to be added. This is a feature that is built into Matrix.

Pros:

• Identites can easily be created and shared across federated servers
• Actors and collections could fit with chatting in domains and groups.
• REST/JSON APIs allowing scripts easy access to users, actors, …

Cons:

• Not completely distributed
• Would have a lot of URL traffic between metaverse-servers

Comments

In my previous post ( Vircadia Directions ) on the directions I’d like to take the Vircadia project, I talked about integrating ActivityPub as the socialization backend to the metaverse-server. That has lead me to several days of researching ActivityPub, ActivityStreams, the various implmentations of same.

The idea is to use the ActivityPub system to create a “Fediverse” of virtual worlds and thus leverage an existing infrastructure and community.

The original Project Apollo metaverse-server design has a server that presents the MetaverseAPI (REST/JSON/…) and has a custom designed backend database to store information about accounts, domains, account relationships (friends, …), and domain connection information (ICE servers, chat servers, …).

The diagram to the left shows this structure (with the “domain-servers” simplified into one box that includes the “assignment clients”). In this structure, the only interface that is “standard” is the “MetaverseAPI”.

In thinking about adding ActivityPub, I first thought I’d add a MongoDB database on the backend (since everything is JSON) and an ActivityPub interface on the frontend. This would allow the existing domain-servers to talk to the metaverse-server and any other metaverse-servers and ActivityPub application to communicate with the users in that metaverse.

But this would mean writing a lot of code (the ActivityPub interface) just to interface with the metaverse database. Looking around, there are several system implmenting ActivityPub and some services that could just be used.

But then, I realized that the metaverse-server just creates and manages accounts and manages the domain-server interconnections. What if I just considered the metaverse-server a MetaverseAPI to ActivityPub protocol converter?

Since the ActivityPub protocol already has all the functions for creating and managing account as well as functions to create and and manage collections of objects (like domains), an ActivityPub server could act as the database for the metaverse-server.

ActivityPub servers already have authentication and access control conventions.

I first thought that I could just use a Mastodon server as the backend metaverse database but I believe that we will want to be extending the functionality of the server. This work is extending the ActivityPub social network model into virtual worlds so there will be special features like chat groups that only work with the people close together. Thus, location and distance need to be added to status and filtering.

At the moment, I am considering the ActivityPub-Express project which is a pretty well written NodeJS/ExpressJS/MongoDB based ActivityPub server.

Comments

Over the last several months I have been working on the Project Apollo metaverse-server for the Vircadia project. The Internet is awash in virtual world projects – both small and large. Because of this, I have been thinking lot about this project and why I would spend any time on Yet Another Virtual World. This captures some of where I think Vircadia could go to be a growing and interesting addition to the sea of virtual world technologies.

First, what is Vircadia? Vircadia is a volunteer, open-source, virtual world server and viewer project based on the High Fidelity code base. A small collection of interested people have pulled this code dump into a project and a community that chats, plays trivia games, and watches movies together while working to improve the code and content of the virtual world. The goal is to make a free, open-source, distributed, and non-corporate owned virtual world that could be useful for education, training, meeting, and socialization. The community and their passion for this particular virtual world technology keeps the project alive.

I have spent years working on the OpenSimulator virtual world technology. and more recently on my Herbal3d and Basil viewer projects. These are all for virtual worlds and the communities they create. I have watched OpenSimulator putter along with hobbyists running their own virtual world grids and small communities enjoying the ability to socialize and hyper-grid/teleport among the many worlds. But, in general, OpenSimulator never grew beyond its SecondLife dispora and the project has dwindled over time. It’s still alive and interesting, but it is not making a new and interesting addition to the virtual world ecosystem.

I got involved with Vircadia because of the great people and they needed some server-side software and that is one of my interest areas.

But, now that I’ve invested in this project, I don’t want it to slowly die as people move on and/or just get lost in the virtual world sea of projects.

There is the “marketing” direction which consists of building communities and finding creators and applyers (like teachers or trainers) to increase the number of people using the project. But I’m more of a tech person so I see some technologies to focus on in Vircadia development that would attach it to other technology focused communities and grow and enhance its application. The goal of these directions is to attach Vircadia to other like-minded technical, volunteer, and passionate communities.

So, the two directions I’d like to help Vircadia go in are:

• develop the technology for a decentralized virtual world community, and
• generalize the virtual world viewer for new and different rendering engines.

Decentralized Virtual World Community

Let’s talk decentralization.

One of the main motivators of the Vircadia project and many other hobbyist and open-source development projects is the idea of creating a non-centralized service/application/resource. And by “centralization” most think of the large, ad-based concentrators like Google or Facebook.

So, by “decentralization”, most projects want there to be no single owner of all the data, no single one place everything is stored, and no one place that is necessary for the project to exist.

I feel like there are two approaches to this “decentralization”: 1) decentralized storage and linking and 2) decentralized infrastructure.

“Decentralized infrastructure” is where multiple people host a common infrastructure and the service/application is spread across these hosts. So, like Tor or BitCoin, the service is not in a single location but it spread out across multiple hosts so, one host going down or being compromised, does not stop the service/application and no one entity holds or owns the service/application.

“Decentralized storage and linking” is where multiple people host their own stuff (no single owner or storer) and there are conventions for naming, addressing, and fetching content and services. In this case, data is spread out across the Internet and availability and access is arranged and controlled by the source of the data or service.

“Decentralized storage and linking” is more about linking together smaller island while “decentralized infrastructure” is more about creating one large island. Both of these have their excellent uses but, for creating a set of virtual world communities, I’m leaning more toward the original inter-net, web focus of tying communities together.

Hope you’re still with me here.

“Decentralized storage and linking” is the original design of the World Wide Web where everyone hosted their own site and there were hyper-links to get from one hosted resource to another. W3C is the World Wide Web standards body that carries on this architectural direction and has recently published several “social community” standards which enable applications for decentralized social communities.

These are the standards ActivityStreams and ActivityPub which support social sharing services like Mastodon, PeerTube, and many others.

The model presented by ActivityPub is of actors that have “streams” of “activities”. Thus, a person can have a newsfeed, a community could have a stream of newsletters, an event could have a stream of events, and a myriad other combinations. The standard allows for publishing and subscribing to streams.

This creates a collection of hosts that are created for an interest area, a community, or just for fun and the hosts exist for as long as the hoster wishes. There is no central control or ownership. Just interested and passionate individuals.

This architecture is useful for Vircadia for two reasons:

1. It matches how I see developer based virtual worlds growing – as small hosted “grids”, and
2. Linking the “girds” together with an existing socialization standard pulls the community into a larger socialization world of like-minded developers and users.

So, how do I see this playing out?

I see the continued development of a metaverse-server to act as the center of a “grid” of virtual world spaces (“domain-servers”). This metaverse-server would provide the services to tie this collection of virtual world spaces together – account creation, storage, and permissions, domain-server registry and discoverability, and any cross-domain communication). This provides the basis for anyone to create their own, independent virtual world.

Then, adding to this, extending the metaverse-server with an ActivityPub interface. For multiple grids, this allows the grids to share some account identity/profile information, have interest groups that span grids, have conversations that span grids, and other sharing.

For wider usage, one could use existing ActivityPub based tools to share and communicate with users in the virtual worlds. For instance, use Mastodon to chat with or check the presence of people in the virtual world. Or use one of the many gateway applications (like ActivityPub to Discord) to connect between social groups and virtual world groups.

One way of looking at it, this makes [Vircaida] the virtual world extension of ActivityPub or maybe the virtual world version of Mastodon.

Check out The-Federation and Fediverse.Party for some of the other federation-based, decentralized communities.

Generalize the Viewer

This was all about the socialization side of Vircadia, but what about the viewer. There are many people very passionate about the technologies and devices for viewing virtual worlds.

The existing viewer is a custom C++ engine. It combines world logic (avatars, location, scripts), with simulator interface (TCP and UDP communication), with the rendering engine (and the interfaces to all the different VR devices).

A presentation by Sam Gateau (the High Fidelity core engine team lead) suggests the rendering part of the viewer is separated from all the world logic by a scene representation that is acted on by “transactions” that come from the world logic. This gives a good beginning to separate the virtual world logic from the renderer.

So, what I’m suggesting is that the interface between the “scene” and the world logic (the “transactions”) be formalized and refactored to make code pieces separable. Then, the project could go in different directions. The rendering engine could be replaced (some people have suggested that the Unreal Engine is a candidate).

Or, building on my Herbal3d architecture ideas, build a system where multiple world logic interfaces can talk to the same renderer and thus allowing viewing multiple virtual world systems with the same viewer rather than requiring a different viewer for each virtual world.

Conclusion

I believe that by adding an interface to existing and growing distributed social network systems and by separating the renderer pieces, the Vircadia project can attract many new developers, users, and advancers and live on as a vibrant community.

Well, that’s way too much typing. Back to coding.

Comments

I’m on a get-work-done kick. This means more updates here. I considered moving to Medium to get a wider audience but that is not my goal. My purpose of these pages is for me and I’m not working on my career and my visability.

So, just more updates.

This push toward organization and making visible progress is coming from my recent interest in all the project that Elon Musk has started. Between SpaceX, Boring Company, NeuraLink, and others, this man is changing things.

I could go on-and-on about what I think about these companies but, what applies to this post is that this one person is making world changing things happen. Reading about what he has made happen makes me think: “I have some big ideas that could change things. Why aren’t I doing big things?”.

So, more organization and more visible work.

Comments