What is Sobamail?¶

Sobamail is, first and foremost, an email solution built using a stack of well-known open-source technologies that support SMTP, IMAP, and POP3 as well as webmail.

On top of this foundation, Sobamail provides a distributed application platform that uses highly optimized yet email-compatible protocols for storage and transport of mutation events as well as app code, data and assets.

Sobamail's own email client, Sobamail Desktop, is the main interface to the Sobamail application platform. On top of conventional email capabilities, it also contains an application runtime based on the V8 Javascript engine and CEF, the Chromium Embedded Framework.

Mailboxes¶

Each Sobamail user is assigned one primary email address, which also identifies their mailbox, which is their one and only data store.

Mailboxes contain:

• A folder hierarchy, where each folder contains zero or more message threads, comprised of one or more related messages, along with their metadata.
• Application instances, which contain a graph of mutations along with the latest app state that is the result of applying said mutations,
• Zero or more blobs used to organize and transfer immutable data like email attachments, application assets, etc.

A mailbox has one or more replicas. These are typically on different devices that the user controls and in the Sobamail Cloud.

Mailbox replicas are identified by their randomly generated Univesally Unique Identifiers (UUID v4).

Application Model¶

Applications are identified by their application ids, which are domain-name-like strings just like the ones mobile applications have. An application can have multiple versions, which are identified by cryptographic hashes of their components as well as a human-readable version string².

In Sobamail, applications need to be instantiated in order to be useful. Application instances are identifed by an instance id¹. Each instance has its own independent data store tied to a given mailbox replica.

Sobamail's application model is a bit different from conventional platforms:

• In desktop platforms, apps typically have have one instance per user, and one version per device.
• In mobile platforms, apps typically have one instance and one version per device.

In both, apps are local-only and it is the apps' responsibility to implement synchronization logic (or not).

In Sobamail, apps can have an infinite number of instances. Instances are either local to a single mailbox replica (developer mode) or sychronized across all replicas of one (private mode) or more (shared mode³) mailboxes.

In other words, a given list of users can share a single app instance. This allows multiple users to collaborate on the same instance of, for example, a to-do list app.

So in Sobamail platform;

• Apps have zero or more instances replicated across zero or more users where any app version can be executed to replay any mutation as needed.

Version Progression¶

Sobamail Apps typically start their life as a single instance in developer mode. When in this mode, no replication is performed and all hash value comparisons return true. This is to ensure that the app developer has a hassle-free environment to work in.

As part of deployment process, all components of an application are hashed and frozen, and the app is now ready to be instantiated with all replication features enabled.

Once an app version is deployed, it is set in stone. A mutation generated by a particular version of an application is processed by the same version in all replicas. This is prevents app states from diverging because of behavioral differences between different app versions, assuming the schema changes were backwards-compatible with all the older versions.⁴

An instance has a primary version that is used to produce mutations. Older versions are kept around to replay mutations that are produced by older modules in other mailbox replicas.

Just like apps, the Sobamail runtime is also versioned. Runtime versions are called computers. Apps need to declare the computer version they want run in in the first import statement. As of now, the Sobamail platform only has 1 computer called Replicated-1 whose import identifier is: "soba://computer/R1"

The R1 is still in experimental mode and can exhibit subtle behavioral differences between versions. If you don't want this, please get in touch with us.

Events¶

Sobamail apps are reactive. This means that they are only invoked to handle incoming events. Events are identified by their object keys.

Object keys are strings of format "{Namespace}Name". The namespace string conforms to the XML Namespace spec. Name is a valid Javascript class name. In case it is not obvious, braces are part of the object key string. This is sometimes called the clark notation.

Outside events arrive exclusively in the form of email messages. Sobamail can handle two types of email messages:

• Message, identified by object key: {https://sobamail.com/module/base/v1}Message. This is just regular email -- it has body, attachments, subject, etc.
• Structured Message, identified by object key: {https://sobamail.com/module/base/v1}SMessage

Structured messages are also regular email messages, but they additionally contain a X-Message-Structure or Message-Structure header.

Message-Structure header in structured messages contain the content id of the attachment that contains the Structured Message Content (SMC).

The SMC is the main event format of Sobamail -- It's a JSON or a msgpack document that obeys the following format:

{
  "namespace": "https://example.com/some/ns/v1",
  "name": "Example",
  "content": ["anything", "can", {"go": "here"}]
}

The namespace and name are concatenated in the form of {namespace}name to form the object key. ({https://example.com/some/ns/v1}Example in our example) This key is in turn used to look up which applications are interested to process the event at hand.

Mutations¶

Mutations are events that modify application state. In simple terms, processing a mutation results ultimately in one or more INSERT or DELETE statements to application tables (tables defined by soba.schema.table calls), followed by additional metadata manipulation to system tables (tables whose name starts with an underscore) ensure that other replicas replay the same event in an efficient way.

Sobamail runtime does not support the UPDATE statement -- updates are emulated using consecutive DELETE and INSERT operations.

Again, all Sobamail events, including mutations, obey the SMC format. The SMC is serialized as msgpack in order to be hashed and form the mutation graph that is replayed by other replicas.

Application Components¶

Sobamail apps have two distinct components:

• A Mutator module that defines how the Application reacts to incoming events. The Mutator can import other modules which are identified using a cryptographic hash of the imported source code text.
• A User Interface package that provides a local web interface to the application. This part is just a zip file that contains a regular frontend app built using HTML/CSS/Javascript. GUI packages are identified by the cryptographic hash value of this said zip file and are distributed as part of app deployments. When a user launches the GUI for a given app, they are shown the GUI version tied to the primary mutator version.

The Mutator Module¶

The Mutator module (aka the backend or the root module) is the heart of a Sobamail application. Written as a modern JavaScript module, each mutator module has to export a default class that contains a process() method along with metadata like objects of interest, application id, name, version, etc.

The process() method serves as the main entry point to the event handler (just like the main() function in C). It is only invoked to handle incoming events which arrive exclusively in form of email messages.

When an email arrives at a mailbox replica, it is "broadcasted" in a way to all applications. Applications need to express interest in a given object type to be able to react to it. This is done by adding object keys to the objects map in the root module.

There are two types of events: Replicated and non-replicated. The runtime treats these two event types differently:

• Replicated events can only perform mutations in a way that avoids data races with other application instances. Since no coordination is possible with other Application instances, which may or may not be online, only strictly monotonic operations are allowed.
• Non-replicated events can do two things:
  • Run arbitrary queries on underlying SQLite databases via a read-only database connnection
  • Emit mutations in form of replicated events.

Non-replicated events are typically used to respond to requests that arrive via the email or the web interface.

Imports¶

Mutators can contain one or more import statements -- top-level statements that take a string in the URI format.

The first import statement of a mutator module must be the import statement that designates the computer where the module will run on.

In Sobamail, import statements are comprised of two parts: The namespace and the cryptographic hash corresponding to the desired module (found in the query string). Since any app can import any module from another app, the apps running in the Sobamail platform actually form one giant codebase.

This reduces barriers to app interoperability -- as long as apps export relevant classes, any app can use data types of any other app.

As of now, the Sobamail computer does not allow sharing of actual data between app instances, but that's on the roadmap.

The User Interface¶

The UI layer in Sobamail is a regular frontend application which can be built using regular JS tools and frameworks. Sobamail uses Chromium Embedded Framework to display messages and run app UI. As of now, your GUI code needs to be compatible with Chrome 117.

The UI communicates with the Mutator module exclusively through an email-compatible RPC interface. This not a RESTful interface -- one can define objects that go back and forth but it's not possible send additional methods like GET or PUT. If you rather like the REST mindset, remember that nothing stops you from adding a method property to your objects.

Sobamail prefers msgpack as object encoding since it supports full-width integers and binary data, but JSON is also supported.

Replication¶

Each mailbox in Sobamail has exactly one instance of the following two apps:

1. The Application Manager: Manages application lifetime (modules, assets, etc.)
2. The Mailbox Manager: Manages folders, messages and their metadata.

The Mailbox Manager and Application Manager are special system applications that use the facilities of the Sobamail runtime to synchronize mailbox replicas. They are automatically replicated across all replicas of a user's mailbox, private to the mailbox owner (no replication accross mailbox boundaries), and restricted to one instance per mailbox (ie. they are singletons).

Applications in Sobamail operate in a truly distributed manner:

• Each mailbox replica has its own independent copy of the application instance.
• Applications share mutations automatically as replicas come online

As mentioned above, to prevent state divergence due to version progression, all events are propagated with the mutator that produced them. This means that once installed, mutator modules never get deleted as long as the instance that has mutations produced by these modules is alive.

Email Compatibility¶

The POP3 or IMAP protocols can be used to access the mailbox replica data kept in Sobamail servers. App instances can be exposed as folders, and mutations as structured messages⁵.

Next Steps¶

The best way to internalize the Sobamail concepts is to get your hands dirty! Head over to the Hello World chapter to see how to implement a Sobamail app!