Synopsis

The ClojureScript compiler currently directly prints JavaScript source code strings. It is desirable to instead produce a JavaScript Abstract Syntax Tree to simplify code generation, emit source maps, and to enable higher level optimizations.

Problems

• Compiler should be more functionally pure
  • Code generation currently emits strings as a side effect
  • Enhancing the compiler is difficult because printing forces ordering and limits higher order composition
  • It's not safe to interleave multiple passes of analysis, transformations, and code generation
• emit complects Code Generation and Code Printing
  
  • requires simultaneous consideration of JavaScript's abstract structure and particulars of syntax
  • code generation is trivially functionally pure; code printing could be, but wouldn't benefit much
• SourceMaps! Strings lack structure to associate mappings
  • If you currently are printing "foo(bar)", you might need to assign different source lines to both foo and bar
  • Adding source mappings to printing would give very low mapping resolution for the current ad-hoc strings
  • Increasing the resolution would yield something that looks very much like an AST

Goals

• Decouple code generation from code printing
• Simplify compiler.clj
• Include source mappings on outputted AST
• Preserve compiler and generated code performance

Approach

• Target the Google Closure AST
  • Advantages
    • Keeps us out of the JS AST design business
    • Existing code printer
    • Includes facilities for Closure Compiler type system
    • Perf? Can skip roundtrip to JS source and to disk
  • Disadvantages
    • Closure AST nodes are stateful
      • Each Node has an explicit parent reference
      • Cloning policy is necessary to reuse subtrees (wasn't an issue in my port so far, see below)
    • Increases our dependence on Closure
      • Closure AST evolves more quickly than the JS spec
      • Recent changes: http://code.google.com/p/closure-compiler/source/list?path=/trunk/src/com/google/javascript/jscomp/parsing/IRFactory.java&start=2295
  • Alternative: Intermediate JavaScript AST as Clojure data-structures
    • Requires an additional translation step
    • GWT and Dart use a different AST; See: http://code.google.com/p/google-web-toolkit/source/browse/trunk/dev/core/src/com/google/gwt/dev/js/ClosureJsAstTranslator.java?r=10778
  • Closure Compiler Discussion: https://groups.google.com/forum/?fromgroups=#!topic/closure-compiler-discuss/OEwLWkw4Kug
• Thin JS-AST DSL
  • Recursively transpiles arguments
  • Can be extended to automatically attach source mappings from dynamic bindings
  • Does a Hiccup like seq expansion
  • Eliminate js* form, replaces it with DSL invokes: (js* "foo({0})" x) becomes (js*/call 'foo x)

Preliminary Progress

• Fork: https://github.com/brandonbloom/clojurescript/tree/js-ast
  • Interesting files: compiler.clj and js.clj
• As of Nov 9
  • Compiles all the CLJS in the ClojureScript tree
  • Bugged in advanced compilation mode
  • Does not generate source mappings
  • Does not bypass the extra round-trip to JS source and to disk
  • Takes 3X as long as naive string printing; 10 vs 30 seconds for (compile-root "src/cljs/cljs") on my machine
  • core.clj and core.cljs are fully ported from (js* "..." ...) to (js*/... ...)