-
Sequential module logic
-
Async callbacks
-
Component based SE
-
Distributed operating system
-
Using modern technologies for these
-
Focus on interface and interchangeable platform/runtime implementation
Execution model
Runtime design must choose how it deals with parallelism, and concurrency
Single thread model: No parallelism, no concurrency Actors, all the logic is encoded in the actor, it has a single control flow and can only handle one thread of execution at the same time. If an actor needs to be able to react to different kinds of messages and possibly interact with request and response patterns, all the logic to “multiplex” the messagebox and determine which action should be taken for each message at a specific time must be coded in user code. Consequently, often actor code will need to read a message then dispatch it to different kinds of logic depending on its state. This makes the management of multi-step interactions complex, as if an actor must be able to manage them from multiple clients, it must keep track of the state of the interaction for each of the clients and each time it receives a message it must choose how to process it depending on its state.
Multi thread model: Yes parallelism, yes concurrency If we allow for multiple threads of execution we can encode multiple logic paths that can handle different kinds of interactions. For multi-step interactions we can dedicate a thread to each client performing the interaction and each one will manage the state of the interaction without needing to manually encode all the dispatching logic. However if we allow multiple threads to coexist and interact with the same state at the same time, we incur in synchronization problems: while we gain advantages for concurrent interactions, we now move the responsability of ensuring that the code correctly handles synchronization and does not incur in data races to the developer. Consequently the advantages that multi threading over shared state bring, come at the cost of added development concerns that the developer must follow.
Cooperative model with interrupts: No parallelism, yes concurrency To address the problems raised by the two previous approaches, we choose to adopt a cooperative execution model, in which
-
Sequential module logic
-
Async callbacks
-
Component based SE
-
Distributed operating system
-
Using modern technologies for these
-
Focus on interface and interchangeable platform/runtime implementation
Intro
Need for large processing power and fast response Distributed application break the application in many services or microservices The division gives flexibility and scalability But it adds complexity such as managing communication, deployment, naming and addressing
Many models tackle these problems from different points of view Serverless functions abstract the deployment and naming, but leave the communication details to the developers and cannot exploit co-locality
Message passing and RPC implementations such as MPI or gRPC abstract the messaging giving high level primitives for interacting with other networked components, but leave the topology and architecture of the system to be handled by the developer
Actor frameworks can partially address these problems, but do so enforcing strong limitations to what each component can and cannot do, such as the lack of shared state and implicitly enforcing a “fire and forget” coding strategy due to the added complexity of having to dispatch messages depending on their type during complex interactions. An actor that should communicate interactively with another actor with a FSA-like interaction must either stop processing all messages from other actors stashing them aside or it must dispatch them managing the state of all “conversations” manually.
As a first objective, we propose a system that can
- Simplify development with abstractions over communication
- Simplify deployment and resource management
- Can exploit locality for efficient communication and state management
- Goes beyond the limitations of the actor framework that we described above
Furthermore, we
Present this model as similar to a specification for a distributed operating system. In this model applications invoke “syscalls” in the runtime, which provides the distributed primitives that can be combined to write a distributed application.
Define the set of runtime operations, syscalls, as an interface layer/specification that can be used to develop different runtimes that can then execute any application conforming to this specification.
Implement an efficient execution engine using Rust and WebAssembly and demonstrate the effectiveness of our proposed model by implementing [Distributed KVS, dataflow engine, …]
Distributed operating system Using modern technologies for these Focus on interface and interchangeable platform/runtime implementation
API
The definition of the interfaces of this system is key
- Deployment strategy
- Communication protocol
- Runtime-Atom interface
One of the core objectives of the system is to cut these interfaces and allow for mixing and matching implementations depending on the requirements of the system.
Making a single platform that can handle all kinds of applications and deployment scenarios is complicated, different applications have different objectives and constraints, as such, the optimal choice for an application may instead be detrimental to another.
For this reason, one key aspect of our work is to define an interface through which applications and runtimes can interact. The runtime should implement a set of functions that are available to the components of the application and act as the system calls of a distributed operating system.
On the other side, application components must implement functions that are exported to the runtime to allow it to interact with them.
For the latter, each module must provide:
- The invoke function
- The interrupt function
- Interrupt table
- The future function
- Future table
In exchange, the runtime provides ergonomic functions for storage, communication, service discovery and single-node utilities that would be provided by an operating system.
[…]
The interface is defined as a set of exported functions through the webassembly interface. Such an interface can be implemented by any binary target that compiles to webassembly and by webassembly runtimes that can be embedded in the compiled runtime. This ensures the compatibility between applications and runtimes and allows for portable execution independent of the hardware architecture and programming language.
Integration hell 🔥
Goal of giving tools to the developers that allow them to easily implement complex applications.
On this path we tried to implement higher abstraction components that could be reused by applications
Describe complexity of integrating general purpose components with specific distributed applications. Challenges:
- Replication
- Partitioning
- Membership (join/leave)
The limits of reusability
Either high level components that can be reused as whole or low level primitives. In the middle there is troubles:
Generic components of medium complexity must be aware of the challenges presented above, otherwise they cannot integrate with the application. To integrate with those challenges we must define an interface that either delegates from the application to the library or that gives the library knowledge and ways to interact with the application-specific logic. If we delegate, our components must be able to fit all scenario and requirements that may be required and, if they must be able to cooperate, they must implement the cartesian product of combinations between components and interact properly with all the configurations that could be needed. This results in an explosion of code and complexity and still limits the developers to the walled garden between the features that the developer of the runtime has thought of. If we expose, we force the developers to implement complex interfaces that are still complex to decouple in case we want more components to interact with one another. While doing so we necessarily enforce a structure on the application code, that must fit the hole in the puzzle of the runtime with the correct shapes in order for the runtime to be allowed to apply its algorithms and logic using the application components. In doing so, the interface that gets built risks becoming more restricting than the features it provides: the developers must learn the interfaces and must fit the way they implement their applications in the shape enforced by the interface. Yet again, by wanting to give freedom to developers to implement their custom platform for the task, we end up constraining their design with the risk of moving the user code to a second-class citizien that must integrate with the existing runtime.
For these reasons, we find a good compromise in providing lower level primitives directly in the runtime and more complex features provided as components that applications can interact with. We stray away from the middle ground as (?) it encourages bad design and limits developer freedom. Note: there is still benefit in providing complex complete components that applications can interact with directly as (atoms) rather than interacting with external systems as they will implicitly benefit from all the optimizations that are available to the application components, such as fast local communication using in-memory channels or automatic deployment and management of execution environment. They can also benefit from the interchangeable runtime interface to allow as an example to seamlessly write and deploy a single application to a datacenter, edge machine or even a browser.
Eval plan
- Simple distributed KV with no guarantees
- Distributed ACID KV
- Dataflow system
Embedded threads and execution
In order to model different kinds of execution patterns, we define a model in which each atom is executed by a single physical thread. However, we adopt an implicit cooperative model for multiple logical threads in a way similar to interrupts used in operating system development. We define two tables, an interrupt table and a future table. These two components serve a similar function, but differ in utilization patterns. In the interrupt table, we register functions associated to a number, defined as interrupt code. In the futures table, we register future handlers that have a numeric id associated.
Interrupts are meant to be invoked multiple times, an interrupt handler function can handle a specific kind of event or message, while futures represent a future computation that will only be executed once. After that, the future handler will be disposed.
The two tables fit in the implicit cooperative execution model we presented by altering the flow of execution when an interrupt or a future needs to be handled: runtime functions that may block (as is the case for receiving a message) yield control to the runtime, which performs the actual blocking operations. What happens is that the runtime “event loop” can inject interrupts and futures by calling the respective handler function through the interrupt table.
In this way, the main “thread” of execution can appear sequential, while in specific point the control flow can be diverted to the handler for asynchronous tasks on a different semantic thread. This is achieved without exposing the task of managing these multiple interaction logics to the developer, like it is the case with the actor pattern, where different kinds of message for the same actor must be manually dispatched by the library user.
Furthermore, this hybrid approach can cover also the cases of fully synchronous or fully asynchronous code depending on how it is used, without additional effort from the developer.
While this model may seem limiting in the sense that it constrains these “threads” to be executed always within a single physical thread, we complement this by having primitives that allow co-located atoms to interact with shared structures efficiently and safely.
- Messaging primitives
- KV-Store
- Log (?)
- Scratch-unsafe memory (?) GPU-like
- Sharded RWLock structs with atomic ops (?)
Differentiation between Library and Service
Vertical layers
- Low-Level is integrated in runtime (kernel)
- Low but more complex is a library that uses the kernel function (kmod)
- Higher level is a service you can use with messages (application)
Flying/soaring through the cloud while/by looking down below
OS-Distributed OS analogy play on words
Garbage ^