Skip to end of metadata
Go to start of metadata

Problem

For the most part, async dom manipulation functions in JavaScript are both ugly and imperative. The same is currently true in ClojureScript. As an example let's imagine a requirement for a auto-complete box that must make an async call back to the server. However, to avoid creating a network storm we want to limit queries to once a second. In ClojureScript, we may write it thusly:

 

(def current-text (atom nil))

(defn on-changed [txt]
    (reset! current-text txt))

(defn make-server-request [txt]
    (ajax/get (make-url txt) 
              (fn [response error]
                  (when-not error
                            (dom/replace "#suggestions"
                                         response)))))
(defn timeout []
    (when-let [txt @current-text]
         (reset! current-text nil)
         (make-server-request txt)))

(js/setInterval timeout 1000)

(set! (.onchange dom-element) on-changed)
	

 

This code is ugly and every function impure. For a language that espouses immutability and pure functions, there has to be a better way.

Current Solutions

Related work in the JavaScript world:

Flapjax - http://www.flapjax-lang.org/

A fairly normal interpretation of FRP in JavaScript. Requires the user to manually convert event handlers to event streams. From there, streams can be composed and bound to the DOM.

 

Rx - http://msdn.microsoft.com/en-us/data/gg577609.aspx

Reactive extensions take the Observable pattern and extend it to support functional event stream transforms. Many examples can be seen here (http://rxwiki.wikidot.com/101samples)

Potential Solution

I suggest we stick pretty close to the Rx solution. Due to the fact that ClojureScript is single threaded and will remain so for the foreseeable future (web workers communicate purely via message passing), it is possible to construct a much simplified version of Rx in ClojureScript.

 

I suggest the following interfaces:

(defprotocol IObserver
    (onnext [this evt] "Called when a new event arrives")
    (onerror [this err] "Called when an error occurs")
    (onend [this end] "Called when the stream ends"))

(defprotocol IObservable
    (subscribe [this ^IObserver observer] 
       "Registers the observer with the observable. Returns a function
        that when called, unregisters the Observer))

(defprotocol IObservableLookup
    (to-observable [this key] "Returns a IObservable based on the key"))

 

Based on these interfaces, it is fairly easy to start composing blocks of code that call the methods on IObserver differently, and that actually emulate several of the functions found in clojure.core. I suggest the following stream functions:

map - applies a function to every event in a stream. If more than one stream is specified, the events of each stream are queued until a new value for each stream is available. Then all streams are processed one element at a time

reduce - reduces a stream by applying a function to each item and the accum value until onend is called. At this point the accum is pushed to the observers.

merge - simply interleaves the contents of each stream in the order that they are received

interleave - same as merge, but events are interleaved in a round-robin order.

throttle - same as map, but extra events are thrown way, not queued. Only the most recent of each event stream is processed

foreach - executes a function against every element in the stream. Assumes that side-effects take place.

 

Using the above, it's fairly simple to see how one could write our auto-complete in the following way:

 

(-> (to-observable dom-element :changed)
    (throttle (to-observable timer [1 :sec]))
    (rmap make-url)
    (ajax/serverCall)
    (foreach #(dom/replace "#suggestions" %)))

 

Not only is this code concise, but it also presents the logic in a fairly functional manner.

 

After the above interfaces have been proven to work. The implementation of a DSL will be investigated (as a separate proposal). In this DSL we would expose to-observable as symbols where a observable/key pair are written as observable:key.

 

For example:

(bind dom:#suggestions
    (-> (throttle dom-element:changed timer:1sec)
        make-url
        ajax/serverCall))
    

 

 

Topics for Discussion

Should a naming convention be adopted for event stream functions? Map is a good name for the map function, but I'd rather not confuse users.

Suggestions:

  • $map
  • rmap
  • mapr
  • |map

 

Can we simplify this any more? Three interfaces aren't much, but those interfaces hide a lot of mutability. This mutability may be less of an issue in the single-threaded world of JS, but can we see any issues?

References

Some work by Stuart Sierra on Rx in Clojure: https://github.com/stuartsierra/cljque

Event programming in Clojure: https://github.com/ztellman/lamina/

TPL Dataflow, like Rx, but uses queues and processor blocks: http://msdn.microsoft.com/en-us/devlabs/gg585582.aspx

Labels:
  1. Jul 27, 2012

    I'm concerned about the next/error/complete trio due to my experience with jQuery/Backbone style success/error callbacks in an options object vs Node.js style (error, ret0, ..., retN) callbacks. The former were far less composable, since you needed to merge/combine option dictionaries. In general, I'd prefer a single argument and a single return value, so composition is simplified.

    I'm wondering if you had (subscribe [this callback]) where callback was of the form (callback promise). Then you could define (on-success x), (on-error e), and (on-complete) in terms of subscribe, assuming the promise protocol was expanded to query failed state.

  2. Jul 27, 2012

    Right now this page is just a summary. I think people could get a better understanding of the issues at play if it followed the format of this page: http://dev.clojure.org/display/design/Feature+Expressions. That is: what's the problem, a solution, alternatives.

    1. Jul 27, 2012

      Noted. I'll fix it up.

  3. Jul 28, 2012

    > Due to the fact that ClojureScript is single threaded and will remain so for the foreseeable future

    Whatever we come up with should be workable in both a single-threaded and multi-threaded environment. Presumably, somebody would want to use this some sort of stuff in JVM Clojure for Swing.

    1. Jul 28, 2012

      I've run into problems with this approach in the past. The complexity of the design suddenly increases quite dramatically. For example, Observers now have to handle subscribe events in the middle of of executing onnext/onerror calls. In an email (that I can't find), Stuart mentioned that Rich suggested having each Observer be implemented as an Agent (this was for Stuart's Cljque library). While this is an option, now we're talking about implementing agents, in CLJS. I'm not sure what David thinks of that. It wouldn't be that hard to implement Agents using setTimeout though.

      1. Jul 28, 2012

        Observers now have to handle subscribe events in the middle of of executing onnext/onerror calls

        That's a problem in an imperative language, where you need to worry about locking the observer collection, but we've got immutable collections. Couldn't we just use an atom and swap! in the new observer collection? The resulting behavior would be: observers added after an event has begun firing will not receive that event.

        The complexity of the design suddenly increases quite dramatically

        The complexity of the design, or of the implementation? If the former, in what way? If the later, are the single-threaded wins substantial enough to justify letting the implementation dictate the design?