I’m currently mostly looking at the styx implementation and reverse engineering some of the components to understand how to better approach the task
- Styx Rev
- I am also experimenting a bit with maturin to see whether it would be possible/smart to start from replacing the front and back of styx with Rust components
- The idea would be to develop rust components with a python API and basically gut out the parts of styx that can be extended/improved and replace them with Rust components
- Later on we could pivot and have python styx logic running inside of rust as we would have actual multithreading and a proper event loop, but this would come later
- I have confirmed with Kyriakos that basically the only part with mobile code is the operation to send the graph of operators
- This essentially means that if we change the format of the data sent to be a collection of wasm modules instead of python classes + we develop a stateful function interface and harness for wasm, we should then be able to keep the rest of styx as-is
- Concerning the reconfigurability “from below” I am having some doubts
- When I reason coming from the Radon point of view it is very much relevant because we map abstractions to platforms
- But from the Styx point of view we have a single abstraction, and at that point I wonder if there is novelty in saying that we simply defined an interface that allows us to use different backends (maybe there is, like if you take most systems you don’t have this, but still I wonder)
- Completed PoC to run stuff seamlessly from python to rust and back to python again
- Python script use python PyO3 rust lib
- Rust lib loads and runs code in WASM module
- WASM module can call back the rust library
- The module state can carry over state from rust
- The state can keep stuff from python
- So the WASM code can call the python code through rust (including state, tried with a closure)
- Improved the linking process to capture python objects associated to the wasm object
- Drafted a subclass of Operator + StatefulFunction that uses WASM for the run function