Copy Page

You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account. You can also Sign Up for a new account.

<p> </p><p><strong>UPDATE 06052013 - Herwig</strong></p><p>Updated the page with input from David Nolen, the description of the prototype is left intact at the end of the page.</p><p>I concur with giving specify an extend-like interface, which can be passed functions.</p><p><strong>UPDATE 06052013-2 - Herwig</strong></p><p>I pushed a commit to github that makes most of clojurescript runnable again with the refactoring: <a href="https://github.com/bendlas/clojurescript/commit/5d827ee8f11b2a575dff2647f7321bc2c47db1de">https://github.com/bendlas/clojurescript/commit/5d827ee8f11b2a575dff2647f7321bc2c47db1de</a></p><p>There is still an issue with accessing locals from reify, when implementing IFn. Due to the call method hack.</p><p>Added replys to comments.</p><h1>Problem statement</h1><p>Even with namespacing extend-type is a global modification of a Clojure program. While powerful, this feature comes at some cost to modularity. On dynamic hosts like JavaScript, Clojure can provide an efficient modification of individual instances so that they may participate in a protocol. Such a feature would have the additional benefit of actually being more general than extend-type - that is the global modification (via JavaScript prototypes in the case of ClojureScript) is a special case of instance modification.</p><h1>Proposal</h1><p>Add two new macros: specify and specify!. Both have a syntax similar to clojure.core/extend, but take a target object in place of the type and are macros.</p><p>specify! will mutate the target object by assigning protocol methods as keys. This is mostly useful to create new prototypes, but can occasionally be useful for external objects from libraries or eval.</p><p>specify will clone the target object before specify! ing the clone.</p><table class="wysiwyg-macro" data-macro-name="code" style="background-image: url(/plugins/servlet/confluence/placeholder/macro-heading?definition=e2NvZGV9&amp;locale=en_GB&amp;version=2); background-repeat: no-repeat;" data-macro-body-type="PLAIN_TEXT"><tr><td class="wysiwyg-macro-body"><pre>(specify! x
 IFoo 
 {:-bar existing-fn
  :-baz (fn ([x] ...) ([x y] ..)})</pre></td></tr></table><p>The function implementing the method (existing-fn) will be called method-fn in this.</p><h2>Costs</h2><p>Giving specify an interface like extend has following difficulties</p><ol><li>Since there is one slot per method, you need to pass a multi-arity fn when implementing a multi-arity method.<br />In a naive implementation, that would cost another arity dispatch, which already has been done at the method call site.<br />This can be mitigated by having specify reuse the existing infrastructure for arity optimization: properties on the fn object called <span style="color: rgb(221,17,68);">.cljs$lang$arity$*</span><br />specify can peel off those properties and use them as methods, which leads to the next cost</li><li>A redundant fn object when implementing inline (the one that would do the arity dispatch otherwise)<br />this could be mitigated by having specify recognize fn forms and emitting the arity-methods directly </li><li>specify will have to dispatch on the type(s) of method-fn<br />the type being whether it has <span style="color: rgb(221,17,68);">.cljs$lang$arity$*<br /><span style="color: rgb(0,0,0);">see Issues &gt; Method arities </span></span></li></ol><h1>Issues</h1><h3>Method arities</h3><p>Since the arity dispatch is already done at the method call site, we want to directly call the appropriate arity of the method-fn. Since method-fns are lexically bound, we cannot statically know in all cases, how to get the target arity thunk from method-fn.</p><p>As the fallback, specify will have to look at the method-fn in every call (of specify!), but this could be mitigated, when the type of a method-fn is known from analysis.</p><h3>Compatibility with external objects / Name clashes</h3><p>Since specify may come into contact with external data, there is a risk of name clashes, since under advanced compilation method names are shortened to few characters.</p><p>specify should check if a property exists on the target object before assigning and should throw if so. This will also guard against double specification with possibly different method-fns.</p><p>For external data, such conflicts can and should be resolved with externs files.</p><p><span style="color: rgb(153,51,0);">We need to think about this some more - we really don't want any checks for modifying prototypes. We need specify! to work elegantly for both scenarios - David</span></p><p><span style="color: rgb(0,0,0);">Ok, that's true. Maybe a warning would be more appropriate but the warning strings would already equal half a reflection layer. </span><span style="font-size: 10.0pt;line-height: 13.0pt;color: rgb(0,0,0);">In production, </span><span style="font-size: 10.0pt;line-height: 13.0pt;color: rgb(0,0,0);">people really shouldn't specify the same object more than once. For development and macros that will use specify on the prototype (extend-type), there could be another metadata flag ^:force-specify - Herwig</span></p><p><span style="font-size: 10.0pt;line-height: 13.0pt;color: rgb(153,51,0);">I'm inclined to say that specify! need not warn at all. If you're going to specify! foreign data you don't control that's a user error. For foreign data I think it's probably far wiser to use reify.</span></p><p><span style="font-size: 10.0pt;line-height: 13.0pt;color: rgb(0,0,0);">It's a user error, alright, but it's one that might manifest as a heisenbug from compilation to compilation, because the set of clobbered property names will change randomly. It won't show up under normal compilation and not even under advanced compilation with minification disabled (there is an option in gclosure for that, we should expose it). A truly horrifying perspective in my eyes. We should at least warn about it. - Herwig</span></p><p><span style="color: rgb(153,51,0);"><span>The docstring of specify! should make it clear that it's not intended for data objects outside the ClojureScript compilation process and suggest reify for those cases. - David</span></span></p><p>I'm not exactly happy with that, but I think its manageable, given that clojurescript has still some way to go towards 1.0 - Herwig</p><h3>Object methods and IFn</h3><p>As per the refactoring from CLJS-414, generating (unmangled) Object methods is possible with (the private version of) specify.</p><p>When implementing IFn, extend-type generates a multi-arity fn and uses that as a call method. This hack was carried over into CLJS-414.</p><p>specify will generate arity methods, call and apply directly, when implementing IFn.</p><p>David, can you comment on how .apply vs .<span style="color: rgb(0,0,0);">cljs$lang$applyTo are used?</span></p><p><span style="color: rgb(153,51,0);">.apply is for regular JavaScript fns and simple ClojureScript fns, .cljs$lang$applyTo is for everything else, .cljs$lang$applyTo gets checked first - David</span></p><p>I will need to unify the code for emitting a function with the code for specifying IFn. <span style="font-size: 10.0pt;line-height: 13.0pt;">That move might even make fn* redundant in the analysis format, since it can then be implemented as (specify! (Function.) IFn ...) - Herwig</span></p><p><span style="color: rgb(153,51,0);">I don't see how since you still want the general version that dispatches via argument.length - David</span></p><p><span style="color: rgb(0,0,0);">Well, my example presupposed that specifying an Object as IFn would generate .call and .apply, as requested, which would supposedly need to dispatch on argument.lenght and do all the other things fn is doing, won't it? - Herwig</span></p><p><span style="color: rgb(153,51,0);">I don't see why you need to generate .apply instead of cljs$lang$applyTo. We do probably need to generate .call, this is mostly needed for compilation mode other than advanced - David</span></p><p><span style="color: rgb(0,0,0);">1. pass a function-like object into jquery 2. jquery calls .apply on it. ... We don't need to generate .apply for every specify IFn; a single .apply method that calls .call or .<span style="color: rgb(0,0,0);">cljs$lang$applyTo can be assigned to all implementors of IFn. Maybe just Function.prototype.apply - Herwig</span></span></p><p><span style="color: rgb(153,51,0);">It's an entirely unrealistic expection to think you can pass instances of IFn to external JavaScript library and think they will work. The point of adding a .call property is about a uniform ClojureScript calling convention. JS libs will not generally invoke functions via .call or .apply. Adding .apply serves no purpose for ClojureScript that clj$lang$applyTo doesn't solve. - David</span></p><p><span style="color: rgb(0,0,0);">Right, adding .call doesn't automatically make an object callable. Thanks for catching that! </span></p><p><span style="color: rgb(0,0,0);">But we do want cljs$core$IFn$_invoke$arity$variadic and cljs$lang$maxFixedArity, right? - Herwig</span></p><h3><span style="color: rgb(0,0,0);">Method names</span></h3><p><span style="color: rgb(0,0,0);">Should the method names be written as symbols instead of keywords? extend has keywords, but it's a function. </span></p><p><span style="color: rgb(0,0,0);">specify might become a function on clojurescript if and when cljs gets reflection, which it probably won't.</span></p><p><span style="color: rgb(0,0,0);">Still, other dynamic hosts, that have reflection built in might profit, if we stick to keywords now.</span></p><p><span style="color: rgb(153,51,0);">there's also the added benefit of specifying maps of fns at the macro level and merging them there. This one is probably worth asking the community about once the proposal is more fully baked - David</span></p><h1><span style="color: rgb(0,0,0);">Implementation</span></h1><h6><span style="color: rgb(0,0,0);">Herwig</span></h6><p><span style="color: rgb(0,0,0);">I'm willing to implement the optimization for cost 1: to pull arity methods from the method-fn, if available. That would leave the overhead of the type dispatch(es) when invoking specify + a couple of redundant fn objects per specify call site.</span></p><p><span style="color: rgb(153,51,0);">I don't see why we need type dispatches, redundant fns for inline specifications do not add any more cost than what we're currently paying - which is nonexistent as far as I can tell from benchmarking - David</span></p><p><span style="color: rgb(0,0,0);">By type dispatch I mean checking for arity-methods on method-fn / other type specific optimisations. Currently extend-type just generates the arity-methods. A full fn would also generate an arity dispatch and apply that will never be used by specify. - Herwig</span></p><p><span style="color: rgb(153,51,0);">Right we should probably avoid generating the full fn - David</span></p><p><span style="color: rgb(0,0,0);">My point is that we don't generate the full fn, the user does:</span></p><table class="wysiwyg-macro" data-macro-name="code" style="background-image: url(/plugins/servlet/confluence/placeholder/macro-heading?definition=e2NvZGV9&amp;locale=en_GB&amp;version=2); background-repeat: no-repeat;" data-macro-body-type="PLAIN_TEXT"><tr><td class="wysiwyg-macro-body"><pre>(extend-type
 IFoo
 (-m1 ; this only generates IFoo$._m1$arity methods
  ([o] :m1) 
  ([o x] [:m1 x])))

(specify (js-obj)
  IFoo
  {:-m1 (fn ; this generates an unused arity-dispatch, ... unless we do something about it 
         ([o] :m1)
         ([o x] [:m1 x]))})</pre></td></tr></table><p><span style="color: rgb(0,0,0);">- Herwig</span></p><p><span style="color: rgb(153,51,0);">Yes, I think we should special case this since in this case that the fn arities map back to the protocol, there's no need for the wrapper. - David</span></p><p><span style="color: rgb(0,0,0);">==</span></p><p><span style="color: rgb(0,0,0);">Both could be thought of as confined costs, but I would like to make sure that they won't be show-stoppers, if nobody gets around to figure out how the specify macro could interact with the analyzer in such a way that they are removed. </span></p><p><span style="color: rgb(153,51,0);">analyzer/resolve-existing-var is all you need, this will contain all the known information about the fn, arities, max-arity, whether it's variadic etc - David</span></p><p><span style="color: rgb(0,0,0);">Does this also work for lexically bound names? What about expressions? Can I analyze them from a macro? - Herwig</span></p><p><span style="color: rgb(153,51,0);">It does work for lexically bound names though I'm not sure why you need that for specify. Expression themselves don't generally have any information. I don't see why you couldn't analyze from a macro, you're passed the environment so I think you should be able to call analyzer from the macro yes. - David</span></p><p> </p><table class="wysiwyg-macro" data-macro-name="code" style="background-image: url(/plugins/servlet/confluence/placeholder/macro-heading?definition=e2NvZGV9&amp;locale=en_GB&amp;version=2); background-repeat: no-repeat;" data-macro-body-type="PLAIN_TEXT"><tr><td class="wysiwyg-macro-body"><pre>(specify x IFoo {:m1 existing-fn}) ; existing-fn is lexically bound and could be an arbitrary expression. Remember, I'm referring to existing-fn as method-fn - Herwig</pre></td></tr></table><h6><strong>UPDATE 06052013-3 - Herwig</strong></h6><p>Most major decisions seem to be ironed out. I'll start with the ground work for:</p><ol><li>Implementing protocol methods with the arity-methods of a regular fn</li><li>Emitting arity-dispatch for .call outside of cljs.compiler/emit for use in specify!</li></ol><p>There is still time to chime in on the design; it might take a week or two before I can allocate another time block like this to start pushing patches</p><h1><s>Proposal</s> Description of prototype implementation as part of refactoring: <a href="http://dev.clojure.org/jira/browse/CLJS-414">CLJS-414</a></h1><p>The operation to extend an instance will be called specify in the next proposal and have an interface similar to deftype. See discussion at <a href="http://dev.clojure.org/jira/browse/CLJS-398">CLJS-398</a></p><p> </p><h6><strong>UPDATE 06052013</strong></h6><p>The refactoring done as part of this implementation will be rebased to current master with names specify and specify* set to private, until the design for specify is fully fleshed out.</p><h3>Outline of the implementation</h3><p>extend-type now expands to specify on the .prototype property.</p><p>specify expands to specify* of protocols and method-arity-maps with fn expressions.</p><p>Since the whole protocol/type stack in clojurescript is based on modifying the prototype anyway, the patches from CLJS-414 basically just pull out the redundancies and label them.</p><h3>Kernel operation: specify*</h3><p>specify* is loosely based on clojure.core/extend, in that it takes maps of implementing closures.</p><h4>Justification</h4><p>specify* is needed, because currently CLJS doesn't lift lambdas, so new function objects are allocated every time specify is evaluated. This is fine for specifying prototypes or configuration-object-like protocol instances. It is not so fine for specifying 1000 objects to take part in IEquiv and IHash.</p><h4>Differences from extend</h4><ul><li>It is a macro<ul><li>bootstrapping: takes a PersistentMap parameter</li><li>gclosure minification + no reflection layer: clojurescript doesn't know how to access name mangled protocol fields</li></ul></li><li>It takes method names as symbols<ul><li>by the time we have an optional reflection layer there might be a cljs.reflect/specify with same interface as extend</li></ul></li><li>Takes arities in addition to protocol and method name: (specify* o P {-m {1 m-impl-1, 3 m-impl-3}})<ul><li>OK for a low-level user visible operation??</li><li>we allow users to implement only parts of a protocol anyway</li><li>use single-arity fast path</li><li>no need to unpack single-arity impls from multi-arity fns, which breaks down if user passes javascript fn</li><li>abstracts protocol name mangling: essentially a setter generator</li></ul></li></ul><h3>extend-type refactored: specify</h3><ul><li>Takes fn bodies like extend-type and instantiates them</li><li>extend-type now just expands to a specify on the .prototype property</li><li>'Object' pseudo protocol lives here</li></ul><h3>Example</h3><table class="wysiwyg-macro" data-macro-name="code" style="background-image: url(/plugins/servlet/confluence/placeholder/macro-heading?definition=e2NvZGV9&amp;locale=en_GB&amp;version=2); background-repeat: no-repeat;" data-macro-body-type="PLAIN_TEXT"><tr><td class="wysiwyg-macro-body"><pre>(deftype A
  ILookup
  (-lookup ([o k] (aget s k))
           ([o k nf] (if (.hasOwnProperty o k) (aget o k) nf))) 
;; expands to

(specify (.-prototype A)
  ILookup
  ...)

;; expands to

(specify* (.-prototype A)
  ILookup {-lookup {2 (fn [o k] ..)
                    3 (fn [o k nf] ..)}})

;; and if we want to specify a whole batch
;; we can have the same method instances attached to every object
  
(let [m2 (fn [o k] ..)
      m3 (fn [o k nf] ..)] 
  (defn specify-alookup! [o]
    (specify* o
      ILookup {-lookup {2 m2 3 m3}})))

(defn jsonp-callback [json-list]
  (process (map specify-alookup! json-list))) </pre></td></tr></table><p> </p><h3>Issues</h3><ul><li><s>specify* can't-be used with IFn in the current patchset to <a href="http://dev.clojure.org/jira/browse/CLJS-414">CLJS-414</a> as of writing </s></li><li>specify expects to see each method only once, should error when user tries to do</li></ul><table class="wysiwyg-macro" data-macro-name="code" style="background-image: url(/plugins/servlet/confluence/placeholder/macro-heading?definition=e2NvZGV9&amp;locale=en_GB&amp;version=2); background-repeat: no-repeat;" data-macro-body-type="PLAIN_TEXT"><tr><td class="wysiwyg-macro-body"><pre>(extend-type T
 P
 (meth [a] "one")
 (meth [a b] "two") )</pre></td></tr></table><ul><li>Is specify* too low level?<ul><li>Are users that just want the methods reused better served by defspecifyer with deftype syntax?</li><li>Might just be a temporary solution until we get lambda lifting anyway</li><li>Are there other use cases?</li></ul></li></ul>