<< Back to previous view

[CLJS-1764] Double warning for undeclared Var Created: 26/Aug/16  Updated: 26/Aug/16

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Mike Fikes Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

A regression occurred where an undeclared Var in a {{require}}d file causes two diagnostics:

$ more src/foo/core.cljs
(ns foo.core)

(def x 2)

abc
$ rm -rf .cljs_node_repl
$ java -cp cljs-1.9.227.jar:src clojure.main -m cljs.repl.node
ClojureScript Node.js REPL server listening on 52749
To quit, type: :cljs/quit
cljs.user=> *clojurescript-version*
"1.9.227"
cljs.user=> (require 'foo.core)
WARNING: Use of undeclared Var foo.core/abc at line 5 /Users/mfikes/Desktop/src/foo/core.cljs
WARNING: Use of undeclared Var foo.core/abc at line 5 /Users/mfikes/Desktop/src/foo/core.cljs
nil
cljs.user=> :cljs/quit
$ rm -rf .cljs_node_repl
$ java -cp cljs-1.9.211.jar:src clojure.main -m cljs.repl.node
ClojureScript Node.js REPL server listening on 56704
To quit, type: :cljs/quit
cljs.user=>  *clojurescript-version*
"1.9.211"
cljs.user=> (require 'foo.core)
WARNING: Use of undeclared Var foo.core/abc at line 5 /Users/mfikes/Desktop/src/foo/core.cljs
nil
cljs.user=> :cljs/quit





[CLJS-1763] Defining a var that clashes with `cljs.core` throws a compiler error instead of warning Created: 24/Aug/16  Updated: 24/Aug/16  Resolved: 24/Aug/16

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: Next

Type: Defect Priority: Major
Reporter: António Nuno Monteiro Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File CLJS-1763.patch    
Patch: Code and Test

 Comments   
Comment by António Nuno Monteiro [ 24/Aug/16 10:43 AM ]

Attached patch with fix and test.

Comment by David Nolen [ 24/Aug/16 11:40 AM ]

fixed https://github.com/clojure/clojurescript/commit/7a06d008fadf56b11dba0f9e2ab97e61059f44fc





[CLJS-1762] Bump Closure Compiler Created: 22/Aug/16  Updated: 22/Aug/16

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: Next

Type: Task Priority: Minor
Reporter: António Nuno Monteiro Assignee: António Nuno Monteiro
Resolution: Unresolved Votes: 0
Labels: closure


 Description   

A new version of the Closure Compiler features an optimization that results in faster compilation, as well as some some changes to ES6-related features.






[CLJS-1761] Allow parallel Transit analysis cache writes Created: 19/Aug/16  Updated: 22/Aug/16

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Fikes Assignee: Mike Fikes
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File CLJS-1761.patch    

 Description   

With CLJS-1759, Transit analysis cache writes are being serialized. This ticket asks that analysis be done to find the root cause, so we can fix it and removed the workaround in CLJS-1759.



 Comments   
Comment by Francis Avila [ 19/Aug/16 1:53 PM ]

I suspect these lines in transit are a race condition:

https://github.com/cognitect/transit-java/blob/3f359c434264f675460bc2662dde2a1b2d8e9559/src/main/java/com/cognitect/transit/impl/WriterFactory.java#L74-L76

newHandlerCache is a cognitect.transit.impl.Cache instance, which is just a normal LinkedHashMap with size 10.

In between the containsKey() call and the get() call a cached item may have been evicted, causing the JsonEmitter to have a null map handler, causing the "Not supported: class java.lang.Integer" exception to bubble out of the emit() call inside write().

I think the fix is to call newHandlerCache.get(customHandlers) by itself and check for null instead of looking up twice. (I am not a seasoned Java dev, so I am not sure if concurrent reads are safe here without synchronization.) The synchronized block should probably also monitor the cache itself instead of the factory class.

Comment by Francis Avila [ 19/Aug/16 1:55 PM ]

Also, all of this is just from reading the code and your stacktrace, i.e., it is purely hypothesis.

Comment by Francis Avila [ 19/Aug/16 2:19 PM ]

On the other hand, the transit writer is created repeatedly with the same handler map instance, so I don't know why the write handler cache would ever have more than one entry (its limit is 10).

Comment by Mike Fikes [ 19/Aug/16 10:38 PM ]

Francis has logged a ticket against transit-java https://github.com/cognitect/transit-java/issues/20 and I've done some further experimentation and recorded what I found in that ticket.

Comment by Francis Avila [ 22/Aug/16 7:04 AM ]

This patch uses write-handler-map and read-handler-map to completely pre-assemble the transit handler maps and sidestep all handler cache construction caching. (I think this is a good thing to do even without these transit problems.)

If there are indeed races related handler construction and caching, this patch should sidestep them all and allow fully parallel use of transit.





[CLJS-1634] Track bound dynamic variables to support binding in async mechanisms. Created: 26/Apr/16  Updated: 20/Aug/16

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: 1.7.228
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Christian Weilbach Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: cljs, enhancement
Environment:

Any cljs version.



 Description   

The issue has been raised before:

While the reasoning behind the proposal is still valid, the original approach has made no progress due to the performance penalty. I have implemented a simplified approach with mutable JavaScript datastructures to minimize the performance impact. Because we are single-threaded we can use js assignment and don't need to port Clojure's binding frame. A small penalty is paid by the user of binding (see benchmark8) and a higher one by async mechanisms capturing and restoring the bindings (benchmark1-7):

https://gist.github.com/whilo/a8ef2cd3f0e033d3973880a2001be32a

I would provide patches to ClojureScript, if this looks like a worthwhile approach.



 Comments   
Comment by Antonin Hildebrand [ 30/Apr/16 6:05 AM ]

Just for record I commented on it here: https://gist.github.com/whilo/a8ef2cd3f0e033d3973880a2001be32a#gistcomment-1764489. Not sure if GitHub sends out notifications about new gist comments.

Comment by Christian Weilbach [ 30/Apr/16 6:18 AM ]

Thanks for pointing it out. David Nolen has also pointed out prototype chains to address this issue and now I see what he meant. I am not familiar enough with the internals of "this" in JavaScript, but one problem I see is that you need to distinguish dynamic vars at the call site. The advantage of using an object directly and capturing and restoring the frame explicitly when you enter and leave the code is that call sites are totally unaffected. The cost is only paid a little at the binding site and mostly in async libraries (bound-fn). But I might still need to look further into "this" . I have not got the gist comment from github.

Comment by Antonin Hildebrand [ 30/Apr/16 7:23 AM ]

Correct.

> you need to distinguish dynamic vars at the call site

I agree. My initial motivation was to solve a bit different problem without cooperation from library authors. I didn't want to modify ClojureScript behaviour and wanted to be just touching own code or doing trivial changes in library forks. Just wanted to share my thoughts about the implementation.

I have a feeling that solving this "async context" problem will be difficult. You will need async library authors to adapt their libraries. And users unaware of this will be running into issues anytime they step outside of bound-fn aware async libraries (for example using raw js interop). I believe Angular people solved this robustly in https://github.com/angular/zone.js. The implementation is quite scary monkey-patching, but if they were able to wrap all existing async calls at lowest level, maybe we could just build on top of their foundation and use zone.js as parallel mechanism for `binding`.

Comment by Christian Weilbach [ 02/May/16 4:58 PM ]

The angle I am coming from is roughly described here: https://github.com/fullcontact/full.monty/pull/9#issuecomment-131152058

I only found out at the very end when I had supervision of go-channels completely implemented, that the cljs binding was not behaving like the Clojure one. Arguments pro/contra binding in Clojure are also referenced. The zone monkey patching looks very heavy and prone to cause hairy bugs. It is noteworthy that Clojure does not embrace bindings, but keeps them always thread-local. So there you also have to use bound-fn or something similar whenever code is executed concurrently. core.async for instance uses the Clojure mechanism to push (capture) and pop (restore) bindings. I would like to have this in ClojureScript as well. I think one should not retain all bindings automatically, but rather allow the library author to handle dynamic bindings. I only track the supervisor binding for instance. For ClojureScript as for Clojure libraries and wrappers this should be fine. Pure JavaScript libraries usually have their own binding concepts like zone.js, right?

Tracking bindings is neither for free in Clojure nor in ClojureScript and it is an important design goal to embrace the host. In fact originally I tried to capture and restore all bindings. My benchmarks for tracking more dynamic vars (instead of just the currently active binding), were linearly more expansive than rebinding fewer selected vars and become prohibitive when you reach a few hundred.

Comment by Antonin Hildebrand [ 02/May/16 5:16 PM ]

I would be happy if your proposal went through. It would help my use-cases as well.

I'm going to explore zone.js when I get some spare time. I will try do write a wrapper library and implement an alternative mechanism to bindings using zone.js. I would like to provide this functionality as a library without a need to modify ClojureScript compiler or involvement from library authors.

Comment by Christian Weilbach [ 03/May/16 1:39 AM ]

Ok, I am curious how well this will work. Would this work with the state-machine transformation of core.async?

Comment by Antonin Hildebrand [ 03/May/16 3:25 AM ]

I believe so. Core.async state machine uses only setTimeout and goog.async.nextTick. We can teach zone.js to deal with nextTick by setting goog.async.nextTick.wrapCallback_ with zone wrapping. Also if user decided to use any async API in their go blocks it should work, because zone.js will carry proper zone binding over async boundaries.

Comment by Antonin Hildebrand [ 03/May/16 3:50 AM ]

I have opened this issue in zone.js: https://github.com/angular/zone.js/issues/342

Comment by Christian Weilbach [ 26/May/16 8:56 AM ]

Hey. Have you made any progress with implementing a small cljs demo with zone.js yet?

Comment by Antonin Hildebrand [ 27/May/16 5:57 AM ]

Hi Christian. No, unfortunately I didn't get to it.

Comment by Christian Weilbach [ 07/Aug/16 4:26 PM ]

Interestingly to implement the Common Lisp like condition system chouser presented here https://www.youtube.com/watch?v=zp0OEDcAro0 the dynamic binding working over async boundaries is also important.

Comment by Antonin Hildebrand [ 08/Aug/16 5:53 PM ]

Christian, please have a look at my implementation:
https://github.com/binaryage/cljs-zones

I have implemented the prototype trick as a library. It is just a gist of the idea, didn't spend time to make it robust yet and ES3-compatible. Re-binding frames should be as cheap as changing pointers (inside JS runtime).

Comment by Christian Weilbach [ 10/Aug/16 6:10 AM ]

Very nice work! I am checking it out atm. Nice that it is self-contained. (The Klipse version throws a goog.object not found error for me btw.)

Comment by Christian Weilbach [ 16/Aug/16 4:05 PM ]

I have updated the gist to incorporate cljs-zones in benchmark 8, 9 and 10. It is a bit faster in restoring dynamic bindings, but not much:

https://gist.github.com/whilo/a8ef2cd3f0e033d3973880a2001be32a

But there is a significant performance penalty on var access:

full.binding_test.benchmark10()
core.cljs:150 [], ((fn [] v1)), 10000 runs, 1 msecs
core.cljs:150 [], ((fn [] (zones/get v1))), 10000 runs, 4 msecs
null
full.binding_test.benchmark10()
core.cljs:150 [], ((fn [] v1)), 10000 runs, 2 msecs
core.cljs:150 [], ((fn [] (zones/get v1))), 10000 runs, 6 msecs
null
full.binding_test.benchmark10()
core.cljs:150 [], ((fn [] v1)), 10000 runs, 3 msecs
core.cljs:150 [], ((fn [] (zones/get v1))), 10000 runs, 4 msecs

Is this penalty addressable?

One other issue is that JavaScript prototype manipulations through "this" would interfere with the cljs binding mechanism, but this might be acceptable.

Comment by Antonin Hildebrand [ 16/Aug/16 4:17 PM ]

Thanks for measuring it. I didn't really get to benchmarking yet.

Did you run the benchmarks under :advanced optimizations?

zones/get currently emits a call to goog.object/get, I'm not sure if this gets inlined by closure compiler, but if not, we can probably improve it by generating raw js object access:
https://github.com/binaryage/cljs-zones/blob/fdfa1421c39d64c9c6b9efbff474b7677f908197/src/lib/zones/core.clj#L94

> One other issue is that JavaScript prototype manipulations through "this" would interfere with the cljs binding mechanism
Can you elaborate? I don't understand how it would interfere.

Comment by Christian Weilbach [ 17/Aug/16 4:10 AM ]

With advanced compilation I get:

full.binding_test.benchmark10()
client.js:278 [], ((fn [] v1)), 10000 runs, 1 msecs
client.js:278 [], ((fn [] (zones/get v1))), 10000 runs, 2 msecs
null
full.binding_test.benchmark10()
client.js:278 [], ((fn [] v1)), 10000 runs, 2 msecs
client.js:278 [], ((fn [] (zones/get v1))), 10000 runs, 3 msecs
null
full.binding_test.benchmark10()
client.js:278 [], ((fn [] v1)), 10000 runs, 3 msecs
client.js:278 [], ((fn [] (zones/get v1))), 10000 runs, 2 msecs
null
full.binding_test.benchmark10()
client.js:278 [], ((fn [] v1)), 10000 runs, 1 msecs
client.js:278 [], ((fn [] (zones/get v1))), 10000 runs, 2 msecs
null
full.binding_test.benchmark10()
client.js:278 [], ((fn [] v1)), 10000 runs, 1 msecs
client.js:278 [], ((fn [] (zones/get v1))), 10000 runs, 3 msecs
null
full.binding_test.benchmark10()
client.js:278 [], ((fn [] v1)), 10000 runs, 1 msecs
client.js:278 [], ((fn [] (zones/get v1))), 10000 runs, 1 msecs

But this is a very simple test. If you see a better way to benchmark it, go ahead . When the chains get prototype chains get longer, it might be more painful to walk the chain for each global lookup.

> Can you elaborate? I don't understand how it would interfere.
There has been a proposal somewhere to pass "this" in all cljs functions instead of null as first argument. Your explicit model of a zone is pretty safe I think, but if people would interefere with prototypes in JS, then this might break the "this" approach in subtle ways. Without passing "this" as a direct argument and doing it in a more JS way, you have to setup the zones all the time instead of using the scope chain, which might also contribute to the managing cost. So the "this" approach might help with performance, but I am not sure. You still have the chain overhead. But I am no JS expert, you know much better than me what you are doing.

Comment by Christian Weilbach [ 20/Aug/16 5:37 AM ]

My last comment was a little bit confusing. I only see the problem with an impact on dereferencing vars. In the benchmark above the prototype chain is very short (v1 can be directly resolved) and already has a significant penalty, but I think the penalty gets even worse for not rebound root vars if you have a higher nesting level for the binding and the chain gets longer. Can you address this somehow?





Generated at Fri Aug 26 22:01:21 CDT 2016 using JIRA 4.4#649-r158309.