Dema's Quartz đź”®

CloudLego Sectioning

3 min read

Stuff

  • Cloud programming paradigms
    • Cloud Vendors
    • VMs
    • FaaS
    • DBaaS
  • Consolidation of applications into a “standard” specification
    • Analytics: Apache Flink
    • Messaging: Apache Kafka (RedPanda, …)
    • Databases: PostgreSQL (Cockroach, …), MySQL (TiDB, …)
  • Components are not enough for final applications
    • Compute is required as glue
    • Real world applications need to bridge the gaps between the components
  • Compute requires special attention
    • Components give guarantees, compute does not
    • Preserving correctness and guarantees is a developer concern
    • Imperfect compute breaks application guarantees
    • Application correctness is not verifiable (PL/FM)1
  • There is no real standard for the compute component
    • Containers are close to a standard, but they give absolutely no guarantees
    • K8s/Terraform/… standardize structure, but no logic
    • Compute is often vendor specific and brinks lock-in
    • Programming language market segmentation, no lingua franca for cooperation2
  • However common application fall between distinct categories of requirements (exactly-once, tx, …)3
  • The technological background
    • Recent developments propose WASM as an alternative to container technology, mostly for performance reasons (Faasm)
    • Other works study its use to enforce a polyglot restricted common interface between entities (Radon)
  • The argument
    • There is need for a specification for compute with guarantees in the cloud
    • We now have the required tools to implement such a standard
  • What is the structure of the specification
    • Client facing interface as set of functions
    • System implements the interface and runs user logic
    • Guarantees enforced transparently from user code
  • Specification layers
    • Client specification
    • Full System specification with Fixed guarantees
    • Partial component specification, combine them with shims to get final guarantees
    • Specification language for guarantees over chains of operations and for component capabilities
    • Compiler combines and shims components to get the target guarantees
    • Optimizer can select the best combination to optimize features while preserving guarantees
  • Specification details
    • Client interface is a set of functions
    • Client guarantees specification is attributes over operations and attributes over chains of operations (get then set → atomic/exactly-once ?)
    • System specification through a set of traits describing the operations they can do, annotated with the guarantees
    • Also for the system it should be a graph that links operations that preserve guarantees when chained
  • Proof of feasibility (Experiments + Related Works)
    • Styx + WASM spec → Decouple polyglot application from the system
    • Radon maybe for the reconfigurability without guarantees4

Footnotes

  1. Probably not the right place ↩

  2. Let’s be careful with this, because there might be an argument for something at the level of gRPC/protobuf or similar ↩

  3. This is a strong statement, may need to be argued or may not be correct at all ↩

  4. Not really sure if it is a good idea ↩

Graph View