A Scala DSL for Rust code generation

Sammanfattning: Continuous Deep Analytics (CDA) is a new form of analytics with performance requirements exceeding what the current generation of distributed systems can offer. This thesis is part of a five year project in collaboration between RISE SICS and KTH to develop a next generation distributed system capable of CDA. The two issues which the system aims to solve are computation sharing and hardware acceleration. The former refers to how BigData and machine learning libraries such as TensorFlow, Pandas and Numpy must collaborate in the most efficient way possible. Hardware acceleration relates to how the back-end of current generation general purpose data processing systems such as Spark and Flink are bottlenecked by the Java Virtual Machine (JVM). As the JVM abstracts over the underlying hardware, its applications become portable but also forfeit the opportunity to fully exploit the available hardware resources. This thesis aims to explore the area of Domain Specific Languages (DSLs) and code generation as a solution to hardware acceleration. The idea is to translate incoming queries to the system into low-level code, tailor suited to each worker machine’s specific hardware. To this end, two Scala DSLs for generating Rust code have been developed for the translation step. Rust is a new, low-level programming language with a unique take on memory management which makes it as safe as Java and fast as C. Scala is a language which is well suited towards development of DSLs due to its flexible syntax and semantics. The first DSL is implemented as a string interpolator. The interpolator splices strings of Rust code together, at compile time or runtime, and passes the result to an external process for static checking. The second DSL instead provides an API for constructing an abstract syntax tree, which after construction can be traversed and printed into Rust source code. The API combines three concepts: heterogeneous lists, fluent interfaces, and algebraic data types. These allow the userto express advanced Rust syntax such as polymorphic structs, functions, and traits, without sacrificing type safety.