Error formatting macro: pagetree: java.lang.NullPointerException
Skip to end of metadata
Go to start of metadata
You are viewing an old version of this page. View the current version. Compare with Current  |   View Page History

 

UPDATE 06052013 - Herwig

Updated the page with input from David Nolen, the description of the prototype is left intact at the end of the page.

I concur with giving specify an extend-like interface, which can be passed functions.

UPDATE 06052013-2 - Herwig

I pushed a commit to github that makes most of clojurescript runnable again with the refactoring: https://github.com/bendlas/clojurescript/commit/5d827ee8f11b2a575dff2647f7321bc2c47db1de

There is still an issue with accessing locals from reify, when implementing IFn. Due to the call method hack.

Added replys to comments.

Problem statement

Even with namespacing extend-type is a global modification of a Clojure program. While powerful, this feature comes at some cost to modularity. On dynamic hosts like JavaScript, Clojure can provide an efficient modification of individual instances so that they may participate in a protocol. Such a feature would have the additional benefit of actually being more general than extend-type - that is the global modification (via JavaScript prototypes in the case of ClojureScript) is a special case of instance modification.

Proposal

Add two new macros: specify and specify!. Both have a syntax similar to clojure.core/extend, but take a target object in place of the type and are macros.

specify! will mutate the target object by assigning protocol methods as keys. This is mostly useful to create new prototypes, but can occasionally be useful for external objects from libraries or eval.

specify will clone the target object before specify! ing the clone.

The function implementing the method (existing-fn) will be called method-fn in this.

Costs

Giving specify an interface like extend has following difficulties

  1. Since there is one slot per method, you need to pass a multi-arity fn when implementing a multi-arity method.
    In a naive implementation, that would cost another arity dispatch, which already has been done at the method call site.
    This can be mitigated by having specify reuse the existing infrastructure for arity optimization: properties on the fn object called .cljs$lang$arity$*
    specify can peel off those properties and use them as methods, which leads to the next cost
  2. A redundant fn object when implementing inline (the one that would do the arity dispatch otherwise)
    this could be mitigated by having specify recognize fn forms and emitting the arity-methods directly 
  3. specify will have to dispatch on the type(s) of method-fn
    the type being whether it has .cljs$lang$arity$*
    see Issues > Method arities 

Issues

Method arities

Since the arity dispatch is already done at the method call site, we want to directly call the appropriate arity of the method-fn. Since method-fns are lexically bound, we cannot statically know in all cases, how to get the target arity thunk from method-fn.

As the fallback, specify will have to look at the method-fn in every call (of specify!), but this could be mitigated, when the type of a method-fn is known from analysis.

Compatibility with external objects / Name clashes

Since specify may come into contact with external data, there is a risk of name clashes, since under advanced compilation method names are shortened to few characters.

specify should check if a property exists on the target object before assigning and should throw if so. This will also guard against double specification with possibly different method-fns.

For external data, such conflicts can and should be resolved with externs files.

We need to think about this some more - we really don't want any checks for modifying prototypes. We need specify! to work elegantly for both scenarios - David

Ok, that's true. Maybe a warning would be more appropriate but the warning strings would already equal half a reflection layer. In production, people really shouldn't specify the same object more than once. For development and macros that will use specify on the prototype (extend-type), there could be another metadata flag ^:force-specify - Herwig

I'm inclined to say that specify! need not warn at all. If you're going to specify! foreign data you don't control that's a user error. For foreign data I think it's probably far wiser to use reify.

It's a user error, alright, but it's one that might manifest as a heisenbug from compilation to compilation, because the set of clobbered property names will change randomly. It won't show up under normal compilation and not even under advanced compilation with minification disabled (there is an option in gclosure for that, we should expose it). A truly horrifying perspective in my eyes. We should at least warn about it. - Herwig

The docstring of specify! should make it clear that it's not intended for data objects outside the ClojureScript compilation process and suggest reify for those cases. - David

I'm not exactly happy with that, but I think its manageable, given that clojurescript has still some way to go towards 1.0 - Herwig

Object methods and IFn

As per the refactoring from CLJS-414, generating (unmangled) Object methods is possible with (the private version of) specify.

When implementing IFn, extend-type generates a multi-arity fn and uses that as a call method. This hack was carried over into CLJS-414.

specify will generate arity methods, call and apply directly, when implementing IFn.

David, can you comment on how .apply vs .cljs$lang$applyTo are used?

.apply is for regular JavaScript fns and simple ClojureScript fns, .cljs$lang$applyTo is for everything else, .cljs$lang$applyTo gets checked first - David

I will need to unify the code for emitting a function with the code for specifying IFn. That move might even make fn* redundant in the analysis format, since it can then be implemented as (specify! (Function.) IFn ...) - Herwig

I don't see how since you still want the general version that dispatches via argument.length - David

Well, my example presupposed that specifying an Object as IFn would generate .call and .apply, as requested, which would supposedly need to dispatch on argument.lenght and do all the other things fn is doing, won't it? - Herwig

I don't see why you need to generate .apply instead of cljs$lang$applyTo. We do probably need to generate .call, this is mostly needed for compilation mode other than advanced - David

1. pass a function-like object into jquery 2. jquery calls .apply on it. ... We don't need to generate .apply for every specify IFn; a single .apply method that calls .call or .cljs$lang$applyTo can be assigned to all implementors of IFn. Maybe just Function.prototype.apply - Herwig

It's an entirely unrealistic expection to think you can pass instances of IFn to external JavaScript library and think they will work. The point of adding a .call property is about a uniform ClojureScript calling convention. JS libs will not generally invoke functions via .call or .apply. Adding .apply serves no purpose for ClojureScript that clj$lang$applyTo doesn't solve. - David

Method names

Should the method names be written as symbols instead of keywords? extend has keywords, but it's a function.

specify might become a function on clojurescript if and when cljs gets reflection, which it probably won't.

Still, other dynamic hosts, that have reflection built in might profit, if we stick to keywords now.

there's also the added benefit of specifying maps of fns at the macro level and merging them there. This one is probably worth asking the community about once the proposal is more fully baked - David

Implementation

Herwig

I'm willing to implement the optimization for cost 1: to pull arity methods from the method-fn, if available. That would leave the overhead of the type dispatch(es) when invoking specify + a couple of redundant fn objects per specify call site.

I don't see why we need type dispatches, redundant fns for inline specifications do not add any more cost than what we're currently paying - which is nonexistent as far as I can tell from benchmarking - David

By type dispatch I mean checking for arity-methods on method-fn / other type specific optimisations. Currently extend-type just generates the arity-methods. A full fn would also generate an arity dispatch and apply that will never be used by specify. - Herwig

Right we should probably avoid generating the full fn - David

My point is that we don't generate the full fn, the user does:

- Herwig

Yes, I think we should special case this since in this case that the fn arities map back to the protocol, there's no need for the wrapper. - David

==

Both could be thought of as confined costs, but I would like to make sure that they won't be show-stoppers, if nobody gets around to figure out how the specify macro could interact with the analyzer in such a way that they are removed. 

analyzer/resolve-existing-var is all you need, this will contain all the known information about the fn, arities, max-arity, whether it's variadic etc - David

Does this also work for lexically bound names? What about expressions? Can I analyze them from a macro? - Herwig

It does work for lexically bound names though I'm not sure why you need that for specify. Expression themselves don't generally have any information. I don't see why you couldn't analyze from a macro, you're passed the environment so I think you should be able to call analyzer from the macro yes. - David

 

UPDATE 06052013-3 - Herwig

Most major decisions seem to be ironed out. I'll start with the ground work for:

  1. Implementing protocol methods with the arity-methods of a regular fn
  2. Emitting arity-dispatch for .call outside of cljs.compiler/emit for use in specify!

There is still time to chime in on the design; it might take a week or two before I can allocate another time block like this to start pushing patches

Proposal Description of prototype implementation as part of refactoring: CLJS-414

The operation to extend an instance will be called specify in the next proposal and have an interface similar to deftype. See discussion at CLJS-398

 

UPDATE 06052013

The refactoring done as part of this implementation will be rebased to current master with names specify and specify* set to private, until the design for specify is fully fleshed out.

Outline of the implementation

extend-type now expands to specify on the .prototype property.

specify expands to specify* of protocols and method-arity-maps with fn expressions.

Since the whole protocol/type stack in clojurescript is based on modifying the prototype anyway, the patches from CLJS-414 basically just pull out the redundancies and label them.

Kernel operation: specify*

specify* is loosely based on clojure.core/extend, in that it takes maps of implementing closures.

Justification

specify* is needed, because currently CLJS doesn't lift lambdas, so new function objects are allocated every time specify is evaluated. This is fine for specifying prototypes or configuration-object-like protocol instances. It is not so fine for specifying 1000 objects to take part in IEquiv and IHash.

Differences from extend

  • It is a macro
    • bootstrapping: takes a PersistentMap parameter
    • gclosure minification + no reflection layer: clojurescript doesn't know how to access name mangled protocol fields
  • It takes method names as symbols
    • by the time we have an optional reflection layer there might be a cljs.reflect/specify with same interface as extend
  • Takes arities in addition to protocol and method name: (specify* o P {-m {1 m-impl-1, 3 m-impl-3}})
    • OK for a low-level user visible operation??
    • we allow users to implement only parts of a protocol anyway
    • use single-arity fast path
    • no need to unpack single-arity impls from multi-arity fns, which breaks down if user passes javascript fn
    • abstracts protocol name mangling: essentially a setter generator

extend-type refactored: specify

  • Takes fn bodies like extend-type and instantiates them
  • extend-type now just expands to a specify on the .prototype property
  • 'Object' pseudo protocol lives here

Example

 

Issues

  • specify* can't-be used with IFn in the current patchset to CLJS-414 as of writing 
  • specify expects to see each method only once, should error when user tries to do
  • Is specify* too low level?
    • Are users that just want the methods reused better served by defspecifyer with deftype syntax?
    • Might just be a temporary solution until we get lambda lifting anyway
    • Are there other use cases?
Labels: