<< Back to previous view

[ASYNC-10] cljs nested go blocks do not work Created: 18/Jul/13  Updated: 19/Jul/13  Resolved: 19/Jul/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Rich Hickey
Resolution: Completed Votes: 0
Labels: None


 Description   
(go (.log js/console (<! (go 5))))

Results in a channel undefined error



 Comments   
Comment by David Nolen [ 18/Jul/13 5:20 PM ]

This bug is likely due to CLJS-544

Comment by David Nolen [ 19/Jul/13 8:08 AM ]

Not a core.async bug, CLJS-544 was the source of the problem which has been resolved in CljoureScript master.





[ASYNC-11] core.async CLJS support incorrect handling of try/catch Created: 19/Jul/13  Updated: 26/Jul/13  Resolved: 26/Jul/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None

Approval: Accepted

 Description   

core.async is defaulting to lein cljsbuild's ClojureScript instead of more recent versions, this has hidden an error with respect to try/catch. core.async try/catch handling emits code like:

(try ex_0 (do ...)))

This is not correct. ClojureScript requires that you specify the prototype of the error that you want to match. Code like this will result in an error about invalid binding form.

You must emit code that looks like this:

(try js/Error ex_0 (do ...)))


 Comments   
Comment by Timothy Baldridge [ 19/Jul/13 11:48 AM ]

lol, I had this code in place, but removed it because it wouldn't compile correctly. This must have changed in a recent version of CLJS.

Comment by David Nolen [ 19/Jul/13 11:51 AM ]

I commented out the try/catch tests in master. I'll hand this one over to you

Comment by Timothy Baldridge [ 26/Jul/13 8:17 AM ]

Fixed in master





[ASYNC-15] go macro interferes with macros that use &env Created: 30/Jul/13  Updated: 02/Aug/13  Resolved: 02/Aug/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None


 Description   
(defmacro locals-test []
  (if (get &env 'x)
    :works
    :sad-panda))

(let [x 1]
  (locals-test)) ;; :works

(<!! (go
       (let [x 1]
         (locals-test)))) ;; :sad-panda


 Comments   
Comment by Timothy Baldridge [ 02/Aug/13 10:56 AM ]

fixed in : https://github.com/clojure/core.async/commit/9371afbe6b90270f005eb003a08fdcbb9aaa6117





[ASYNC-29] 50 parallel blocking takes in go-blocks as loop-binding or recur argument break go Created: 28/Oct/13  Updated: 29/Oct/13  Resolved: 29/Oct/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Rich Hickey
Resolution: Completed Votes: 0
Labels: bug, go, go-macro
Environment:

JVM



 Description   

This problem raises the question whether what the implications are of using <!! and >!! inside of go and whether it is legal.

What if I provide API functions that are using <!! >!! and may be invoked from within go?
Should I then also provide !! and ! versions?

Please evalute this code at the REPL to reproduce the problem: https://www.refheap.com/20225



 Comments   
Comment by Leon Grapenthin [ 29/Oct/13 9:46 AM ]

I'd like to rephrase my question: Do I have to provide {!} and {!!} macros that do the same thing and use {<!} {>!} and {<!!} {>!!} respectively? Would it make sense for core.async to provide a defasync macro that creates those two versions from the same body where you could for example use {<!!!} and {>!!!}, {alts!!!} and so on so that they would be replaced by a postwalk before the macro is defined defined twice with {!} and {!!} appended to the name? Or are there other plans of abstraction? [Quoting symbols because of jira markup]

Comment by Rich Hickey [ 29/Oct/13 9:55 AM ]

You shouldn't be using <!! and >!! in library code. We may at some point be able to detect at runtime their use in go blocks and throw errors, but currently do not.





[ASYNC-34] method calls not correctly handled in go blocks Created: 08/Nov/13  Updated: 09/Nov/13  Resolved: 09/Nov/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Ghadi Shayban
Resolution: Completed Votes: 0
Labels: None


 Description   
(go (. js/console (log "foo")))

Doesn't work.



 Comments   
Comment by Ghadi Shayban [ 09/Nov/13 11:52 PM ]

Fixed on master.





[ASYNC-37] failing set! case Created: 15/Nov/13  Updated: 20/Nov/13  Resolved: 20/Nov/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Ghadi Shayban
Resolution: Completed Votes: 0
Labels: None

Patch: Code and Test

 Description   
(ns example.core
  (:require-macros [cljs.core.async.macros :refer [go]]))

(def foo nil)

(go
  (set! foo "nope"))

Results in the following exception:

clojure.lang.ExceptionInfo: failed compiling file:src/example/core.cljs
                core.clj:4327 clojure.core/ex-info
             compiler.clj:991 cljs.compiler/compile-file
            compiler.clj:1052 cljs.compiler/compile-root
              closure.clj:398 cljs.closure/compile-dir
              closure.clj:437 cljs.closure/eval2663[fn]
              closure.clj:301 cljs.closure/eval2591[fn]
              closure.clj:451 cljs.closure/eval2650[fn]
              closure.clj:301 cljs.closure/eval2591[fn]
              compiler.clj:44 cljsbuild.compiler.SourcePaths/fn
                core.clj:2485 clojure.core/map[fn]
              LazySeq.java:42 clojure.lang.LazySeq.sval
              LazySeq.java:60 clojure.lang.LazySeq.seq
                  RT.java:484 clojure.lang.RT.seq
                 core.clj:133 clojure.core/seq
                 core.clj:617 clojure.core/apply
                core.clj:2514 clojure.core/mapcat
              RestFn.java:423 clojure.lang.RestFn.invoke
              compiler.clj:44 cljsbuild.compiler/cljsbuild.compiler.SourcePaths
             closure.clj:1007 cljs.closure/build
              closure.clj:981 cljs.closure/build
              compiler.clj:58 cljsbuild.compiler/compile-cljs[fn]
              compiler.clj:57 cljsbuild.compiler/compile-cljs
             compiler.clj:158 cljsbuild.compiler/run-compiler
form-init2323732218988862436.clj:1 user/eval3005[fn]
form-init2323732218988862436.clj:1 user/eval3005[fn]
              LazySeq.java:42 clojure.lang.LazySeq.sval
              LazySeq.java:60 clojure.lang.LazySeq.seq
                  RT.java:484 clojure.lang.RT.seq
                 core.clj:133 clojure.core/seq
                core.clj:2780 clojure.core/dorun
                core.clj:2796 clojure.core/doall
form-init2323732218988862436.clj:1 user/eval3005
           Compiler.java:6619 clojure.lang.Compiler.eval
           Compiler.java:6609 clojure.lang.Compiler.eval
           Compiler.java:7064 clojure.lang.Compiler.load
           Compiler.java:7020 clojure.lang.Compiler.loadFile
                 main.clj:294 clojure.main/load-script
                 main.clj:299 clojure.main/init-opt
                 main.clj:327 clojure.main/initialize
                 main.clj:362 clojure.main/null-opt
                 main.clj:440 clojure.main/main
              RestFn.java:421 clojure.lang.RestFn.invoke
                 Var.java:419 clojure.lang.Var.invoke
                 AFn.java:163 clojure.lang.AFn.applyToHelper
                 Var.java:532 clojure.lang.Var.applyTo
                 main.java:37 clojure.main.main
Caused by: clojure.lang.ExceptionInfo: nth not supported on this type: Symbol
                core.clj:4327 clojure.core/ex-info
             analyzer.clj:243 cljs.analyzer/error
            analyzer.clj:1222 cljs.analyzer/analyze
            analyzer.clj:1219 cljs.analyzer/analyze
             compiler.clj:873 cljs.compiler/compile-file*
             compiler.clj:984 cljs.compiler/compile-file
Caused by: java.lang.UnsupportedOperationException: nth not supported on this type: Symbol
                  RT.java:857 clojure.lang.RT.nthFrom
                  RT.java:807 clojure.lang.RT.nth
           ioc_macros.clj:418 cljs.core.async.impl.ioc-macros/eval5251[fn]
             MultiFn.java:227 clojure.lang.MultiFn.invoke
           ioc_macros.clj:627 cljs.core.async.impl.ioc-macros/eval5429[fn]
           ioc_macros.clj:110 cljs.core.async.impl.ioc-macros/all[fn]
            protocols.clj:143 clojure.core.protocols/fn
             protocols.clj:19 clojure.core.protocols/fn[fn]
             protocols.clj:31 clojure.core.protocols/seq-reduce
             protocols.clj:54 clojure.core.protocols/fn
             protocols.clj:13 clojure.core.protocols/fn[fn]
                core.clj:6177 clojure.core/reduce
           ioc_macros.clj:108 cljs.core.async.impl.ioc-macros/all[fn]
           ioc_macros.clj:674 cljs.core.async.impl.ioc-macros/parse-to-state-machine[fn]
            ioc_macros.clj:68 cljs.core.async.impl.ioc-macros/get-plan
           ioc_macros.clj:668 cljs.core.async.impl.ioc-macros/parse-to-state-machine
           ioc_macros.clj:809 cljs.core.async.impl.ioc-macros/state-machine
                macros.clj:18 cljs.core.async.macros/go
              RestFn.java:142 clojure.lang.RestFn.applyTo
                 core.clj:621 clojure.core/apply
            analyzer.clj:1129 cljs.analyzer/macroexpand-1
            analyzer.clj:1164 cljs.analyzer/analyze-seq
            analyzer.clj:1230 cljs.analyzer/analyze


 Comments   
Comment by Ghadi Shayban [ 18/Nov/13 11:07 PM ]

Tentative fix on branch ASYNC-37. Can someone review?





[ASYNC-41] Mult gets stuck when you put a value on the source chan and there are no taps on the first tick Created: 30/Nov/13  Updated: 12/Dec/13  Resolved: 12/Dec/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Radford Smith Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None
Environment:

ClojureScript


Attachments: Text File 0001-ASYNC-41-Make-mult-drop-values-when-there-are-no-tap.patch    
Patch: Code and Test

 Description   

If you have a mult and you put a value on its source channel, but there are no taps at the moment, the mult will get stuck and not put any values on future tap channels. Here's an example: http://cljsfiddle.net/fiddle/rads.mult-bug

In the example, no value will ever get put on the `tap1` chan. There are two possible solutions to this: a) drop all values going to the mult until a tap is created, or b) queue the values until a tap is created. I believe the first option is more straightforward, since if you have 1 tapped channel already and you go to 2 taps, the mult does not put old values on the new tap. This would make the behavior consistent going from 0 taps to 1 tap as well.



 Comments   
Comment by Radford Smith [ 30/Nov/13 10:45 PM ]

It looks like dropping values is already the default behavior on the JVM. In fact, this is the only difference in implementation for `mult` between the JVM and CLJS. I created a pull request with the fix and a regression test: https://github.com/clojure/core.async/pull/38

Comment by Leon Grapenthin [ 02/Dec/13 2:21 PM ]

This is similar to http://dev.clojure.org/jira/browse/ASYNC-26 of the CLJ version and has been fixed. Apparently the fix has not yet been ported to the CLJS version?

Comment by Radford Smith [ 02/Dec/13 4:24 PM ]

I attached the patch that was originally in the pull request.

Comment by David Nolen [ 09/Dec/13 12:16 PM ]

Thanks for the patch. Radford have you submitted your CA, it's required in order for us to merge patches in. Thanks!

Comment by Ghadi Shayban [ 09/Dec/13 12:32 PM ]

Sorry I missed this discussion. I copy ported 3f98 to master, Rich's original fix from the CLJ side of the project.

Comment by Timothy Baldridge [ 12/Dec/13 10:34 PM ]

Fixed by Ghadi





[ASYNC-45] Creating a timeout channel with a double causes the timeout-daemon to die Created: 09/Dec/13  Updated: 13/Dec/13  Resolved: 12/Dec/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ben Poweski Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None


 Description   

I inadvertently passed a double to a timeout channel and discovered the following behavior using 0.1.242.0-44b1e3-alpha.

<code>
user=> (require '[clojure.core.async :as a])
nil
user=> (a/timeout 100.0)
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@1009d11d>
user=> Exception in thread "clojure.core.async.timers/timeout-daemon" java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.Long
at java.lang.Long.compareTo(Long.java:50)
at java.util.concurrent.ConcurrentSkipListMap.doRemove(ConcurrentSkipListMap.java:1064)
at java.util.concurrent.ConcurrentSkipListMap.remove(ConcurrentSkipListMap.java:1896)
at clojure.core.async.impl.timers$timeout_worker.invoke(timers.clj:61)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:722)

user=> (.isAlive clojure.core.async.impl.timers/timeout-daemon)
false
<code>



 Comments   
Comment by Timothy Baldridge [ 12/Dec/13 10:32 PM ]

Fixed in master, added `:pre [(integer? msec)]` to timeout.

Comment by Rich Hickey [ 13/Dec/13 7:24 AM ]

I'd prefer we type hint ^long vs use a precondition





[ASYNC-36] Waiters should unblock on close! Created: 14/Nov/13  Updated: 29/Mar/14  Resolved: 22/Nov/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Herwig Hochleitner Assignee: Timothy Baldridge
Resolution: Declined Votes: 1
Labels: None


 Description   

Expected behavior

When a channel is closed, all readers and writers waiting for it, whether on a thread with the !! family of functions or in a go block, should unblock and return nil, as if the channel had been closed in the first place. This is expected since channels are frequently discarded after being closed.

Problem description

The operator >!! (used as example here) immediately returns, when used on a closed channel, however, if >!! is already waiting for a channel, which is then closed, >!! doesn't unblock. , stalling the thread indefinitely.
EDIT: The thread/go block can still be unblocked, by taking from the channel, however since close! signifies the end of the lifetime of a channel, subsequent takes on it cannot be considered idiomatic.

Example REPL Sessions

In this example, the future blocks indefinitely, even though the target channel is closed

This is a problem, since the >!! might be called in a try..catch holding resources and such.

The example also shows, that values can be read from a closed channel (and trigger processing), which might be ok for channel buffers, not so much for pending >!! that have side effects in their continuation.

> (def ch (async/chan))
> (future (>!! ch :cool)
          (println "done"))
> (<!! ch)
"done"
:cool
> (future (>!! ch :cool)
          (println "done"))
> (close! ch)
;; FIXME "done" should be printed here
> (<!! ch)
"done" ;; EDIT pending >!! can be taken from, same as with >!
nil
> (future (>!! ch :cool)
          (println "done"))
;; Same as here
"done"

EDIT Removed example documenting different behavior of threads and go blocks, since subsequent tests turned them out to behave the same.

References

This was first posted to github https://github.com/clojure/core.async/issues/36
Some discussion already happenend there.

http://dev.clojure.org/jira/browse/ASYNC-31 seems related with regard to the close! semantics of a write.



 Comments   
Comment by Leon Grapenthin [ 14/Nov/13 12:52 PM ]

If I try to reproduce your second future, after (<!! ch) "done" is printed correctly. I tried it twice today with "0.1.242.0-44b1e3-alpha".
Probably you didn't see "done" because it was printed in the nrepl-server buffer?

This would be the only proof of infinite blocking. Aside from that this report raises the question whether parked puts should be cancelled when a channel is closed. It would be great if the docstring of close! would state this more clearly (currently it says "Data in the channel remains available" which could also refer to the channels buffer).

Comment by Leon Grapenthin [ 15/Nov/13 11:43 AM ]

I could also not reproduce the second future hanging after the take with "0.1.256.0-1bf8cf-alpha"

Comment by Herwig Hochleitner [ 16/Nov/13 9:22 PM ]

You are right, I couldn't reproduce the infinite stall (and hope I never will).

I would indeed argue that parked gets and puts should be cancelled when a channel is closed. Otherwise the answer to the question "when can I read/write a channel and when should I do it?" gets very non-deterministic. Don't forget, channels are not only about data transmission but also about coordination, (i.e. thread scheduling, thus resource management).

e.g. the current behavior yields the rule: "only write if somebody's listening or if the channel is not going to be closed anytime soon otherwise you might stall forever". Is this intentional? Better than the alternatives?

Comment by Leon Grapenthin [ 17/Nov/13 10:55 AM ]

If you are taking from a channel, you can determine that it's closed by taking nil from it. If you are putting on a channel you can't determine whether it's closed, however the operation will immediately return if the channel is closed and not block.

However in all cases I used async in so far it seemed only natural to handle the closing of a channel on the same site where the puts are made and the channel was created.

Comment by Herwig Hochleitner [ 17/Nov/13 6:52 PM ]

Yes, but normally you wouldn't take/put/do anything to a channel after calling close! on it.
That's what I meant when I said it wouldn't be idiomatic to require taking from a channel after a close! just to make sure that no writers are blocked.

Comment by Leon Grapenthin [ 18/Nov/13 3:16 PM ]

I would just say that there is no reason to close a channel at all when puts are still pending. You close! the channel only when you are done putting with the reason to communicate to the taking site that nothing more is going to come through the channel.

I find it hard to see the benefit of unblocking parked puts! with close!. They could be in a loop and they don't know that they were unblocked by a close!. If the putting site could determine that the target channel has been closed (by an exception thrown from put! that is unimplemented), it could stop putting permanently - But then it would know what ended up on the channel (and can be taken) and what did not end up on the channel (and can not be taken). Both information would be lost on the putting site if parked puts can be cancelled by a close without notification. You are right that they would be unblocked, but they would probably run infinite loops or simply do more useless puts.

As I see it: Until the put! exception (from the put! docstring) has been implemented one has to avoid putting onto closed channels. If it ever will be implemented it adds another way to determine whether a channel was closed (without taking from it) which I would prefer over cancellation via close!.

EDIT: I forgot to add that you can of course invoke @(.closed ch) to determine whether a channel has been closed but it is questionable whether this is officially supported or an implementation detail that may change.

Comment by Herwig Hochleitner [ 19/Nov/13 2:55 PM ]

> I would just say that there is no reason to close a channel at all when puts are still pending

Channels are designed to be multi-writer and multi-reader, so often you will want to close! a channel from outside. e.g. when the user clicks cancel.
As far as I understand it, close! is the intended interface for such things.

> They could be in a loop and they don't know that they were unblocked by a close!.

Well, that's actually the same when the channel is closed beforehand. And yes, failed puts should be distinguishable from regular puts, but that's ASYNC-31
Should also throw if a pending put gets unblocked by close.

Comment by Leon Grapenthin [ 19/Nov/13 7:15 PM ]

I think I understand your point now. Basically your idea is that channels should be close-able from everywhere, anytime, independently of any puts or takes. However the more I think about it, I can not prefer it over the current implementation:

Being able to close channels from everywhere seems like something that would make it easier to deal with channels and associated processes on the first look. But it would always require a special implementation on the putting sites: Whenever you put on a channel, you'd have to check whether it has been closed and you'd have to implement further decision making based on that information.

The current implementation leads one to do closing in sync with puts: If the user clicks cancel, this is the information the code should process as "Stop putting, Close channel" in that order. If the channel was closed first, that would first produce a new (less valuable) information which would need to be processed (with less information about what happened) which in my view would make things only more complicated.

The user having clicked cancel is specific and valuable information about something that has happened in the past and thus can not be realized in zero time or instantly anyway. Users have to wait as well. The user request can be taken from one channel of user-events. Putting onto the an output channel can properly come to an end first and then the output channel can be closed. Display or other further processing of the last produced output value will happen and the put site can rely on it. With the current constraints one writes code where this information of the user having clicked cancel is directed to the site affected by it, the site were the output values are produced, what is what the user desires to come to an end soon.

This enforcement of handling closing on the calling site is in my view the win for the current implementation: Channels are really like cables were the possibility of them being cut suddenly is not a desired feature but something one would rather avoid.

Comment by Herwig Hochleitner [ 19/Nov/13 10:55 PM ]

> Basically your idea is that channels should be close-able from everywhere, anytime, independently of any puts or takes.

In fact, they are! It's just that the semantics don't seem well-defined to me.

> This enforcement of handling closing on the calling site is in my view the win for the current implementation

There is no such enforcement. Just out of interest: How would you design such enforcement, from an API / Semantics POV? How would you disallow arbitrary closes?

> Channels are really like cables were the possibility of them being cut suddenly is not a desired feature but something one would rather avoid.

We should design not only for desired cases, but also for failure modes. "would rather avoid" isn't sufficient when thinking about the possible combinations of an API.

Also in the real world, cables do get unplugged, without either side getting stuck permanently and plugs/sockets very much are desired features in cable-related technology.

Please do me a favor, play advocatus diaboli and try to argue for a second that channels should be closeable from outside of the writer context.
A good starting point: The writer might already be blocking for an unforseen amount of time when the time to close it has come.

Comment by Leon Grapenthin [ 20/Nov/13 5:25 PM ]

I know that there is no real enforcement of handling closing on the putting site. But it's what you are lead to do when you want to avoid unnecessary puts.

If you call your suggestion (closing from anywhere) a failure - it's a failure that can be avoided by simply not doing it.

I tried to imagine cases where channels should be close-able from outside the writer context yesterday a lot before I posted. I simply can't find any. What you are suggesting as a starting point to find such a case is a scenario where you want to cancel a put operation that has already begun. For that scenario we have alts! where you have one put operation that can take a long time and another channel to communicate the cancellation through. If the cancellation comes first, the put operation will not happen even though it was blocking.

Here is a link to an extended example where I have implemented such a scenario: https://www.refheap.com/21103

Comment by Herwig Hochleitner [ 21/Nov/13 10:14 AM ]

I'm tempted to respond with a sole [closed from the reader side] to demonstrate a case where it's valid to close from outside of the writer context, but of course that would be rude. Instead let me try to get an exact grasp on our differing views:

I think, the full API of a library should be usable or produce visible errors immediately. In my view there is no such thing as "avoid by simply not doing it" in API design (except for bad Java APIs
You are certainly free to view a certain legal combination of API calls as failure but I wouldn't be so quick. I think the API of core.async offers close! as a standalone operation for a reason. I think the reason is, that channels are designed for broader applicability than just "dump into channel, then close".

I'd still like to hear a proposal on how to effectively prevent closing from outside of the writer (my guess is: there is no feasible way with the current API).
The reason I insist on this:
There are 2 things, programmers don't like: deadlocks and "don't do that". The combination of those two are what's broken with lock concurrency. Please let's not regress to that "state of the art".

OK here is an example, please don't discard it due to the fact that writers currently can't distinguish between an open channel and a closed one, that's for ASYNC-31:
Take a channel akin to range: producing an infinite number of values. You generate it somewhere in your program, pass it around and then consume it somewhere else.
How should this work?
1) Requiring me to design my program such I have the close condition available at the channel creation site, even though the producer might be completely generic, while only the consumer logically knows about the proper closing time, OR
2) Just let me close the channel from the consumer, when it doesn't want to consume anymore?

Your alts! example is also an excellent point in my case: Even though the put operation will never happen, it's still remaining in a pending state, tying up resources.

Comment by Leon Grapenthin [ 21/Nov/13 8:37 PM ]

Let me get right back to your last point: I have added a little benchmark of the blocking put operation to my example to prove that the put operation is not in pending state and unblocks as soon as :stop is dispatched.

Unfortunately I don't quite get what you mean by a "sole". "Closing from the reader side" is not necessary with the current API.

Now regarding your example:
1) In this point it is not quite clear to me whether you mean to distinguish between "creation site" and putting site. I will assume that you mean both and the same thing and call it production site. My answer would be: Yes. Design your api/program so that you have a close-condition available, even though the producer might be completely generic. One example for such a close-condition is in the example I have linked to: Taking :stop from a seperate channel stops production and closes the channel. The consumer can determine the proper closing time and send :stop, as demonstrated in my example.
2) I know that this is what you want, but you would still need the close condition at the production site to stop putting. Unless you have determined that the channel has been closed, you'd put the next value onto the chan. With the current implementation, such a scenario would heat up the CPU. If you could determine closing on the production site for example by an exception, you would do exactly the same thing as in point 1), but with less information, as I have pointed out in earlier comments.

I believe programmers like "don't do that" much more than "you can't do that". So I think it's fine that you can close! from everywhere. I had some cases myself where very few unnecessary put operations where tolerable as I knew that they would not be possible again after a very short predictable amount of time (In fact when the user clicked cancel: I knew that the interface would be removed from the screen immediately and that a few puts triggered by event listeners wouldn't mind the CPU). So I wouldn't even go so far as to prevent closing. But what I'd really like is that when I develop an API for a library built on top of core.async is this: I don't want to expect channels being closed every time I do a put. I don't want to write an exception-catcher every time. But if what you have proposed would be reality, that would be an unavoidable necessity. Otherwise, I'd never be sure whether values I have put were received, or the channel has been closed. What if I write a function that takes values from one channel and puts them onto another (like core.async/pipe), and suddenly the output-channel was closed? Would I have to close the input channel then as well? Ownership would be totally undefined in such a scenario and lead to all kinds of confusion and most likely lots of boiler-plate code. If I wrote anything that does an unpredictable amount of puts, I'd provide one or multiple facilities to stop the puts.

You may also want to have a look at the C# implementation of the cancellation of tasks: Cancellation tokens are used to determine cancellation on the production site at a specific point in execution order and are passed as an additional parameter: "If library code provides cancellable operations, it should also provide public methods that accept an external cancellation token so that user code can request cancellation." (http://msdn.microsoft.com/en-us/library/vstudio/dd997364.aspx).

Or erlang, where a "finished" message is passed from the production site: http://www.erlang.org/doc/getting_started/conc_prog.html#id67009

Or in go: "A sender can close a channel to indicate that no more values will be sent." and "Note: Only the sender should close a channel, never the receiver." (http://tour.golang.org/#66)

And then there is Haskell, where you can't close channels, but only kill threads...

Comment by Timothy Baldridge [ 22/Nov/13 2:47 PM ]

Copy of my comments to the email in clojure-dev:

Example repl session:

user=> (def c (chan))
#'user/c
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@29794f5b>
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@7848089e>
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@1b6b43ac>
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@464f0728>
user=> (close! c)
nil
user=> (<!! c)
done
42
user=> (<!! c)
42done

user=> (<!! c)
done
42
user=> (<!! c)
done
42
user=> (<!! c)
nil
user=>

The library works as expected. If you want the pending puts to be dropped, feel free to write something like this:

(defn close-and-flush [c]
(close! c)
(clojure.core.async/reduce (fn [_ _] nil) [] c))

Semantics are now:

user=> (def c (chan))
#'user/c
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@284b38f9>
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@43bbaad3>
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@16cc601f>
user=> (go (>! c 42) (println "done"))
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@630f83c9>
user=> (close-and-flush c)
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel@3f1d023f>done
done

done
done
user=>

--------

The semantics are not going to change. Pending puts are flushed. Pending takes are given nil after a closed channel flushes. This is the way the library was designed to operate. By default the library will not discard data unless you configure it to do so, to have pending puts automatically throw away values on a channel would cause many errors, and make libraries harder to reason about.

Leon is right, close! basically means "after all pending values have been written, close the channel".

Comment by Alexander Kauer [ 29/Mar/14 4:36 AM ]

Hi,

since I ran into the same problem (see report ASYNC-60 , somehow I overlooked this report first) I think the documentation should explicitly state that behaviour to avoid further confusion for others.





[ASYNC-35] Using onto-chan with nonending sequence causes non-gc-able, infinitely-looping go block Created: 12/Nov/13  Updated: 20/Apr/14  Resolved: 20/Apr/14

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Brian Lubeski Assignee: Rich Hickey
Resolution: Completed Votes: 0
Labels: None
Environment:

org.clojure/core.async 0.1.256.0-1bf8cf-alpha



 Description   
(close! (to-chan (range)))

The above code causes my CPU to run at around 95% until I kill the process. (NOTE: This eventually leads to an OutOfMemoryError – see ASYNC-32).

Here is what I think is happening: after closing the channel returned by to-chan, all subsequent puts to that channel by the to-chan go block succeed immediately without blocking. Because the to-chan go block never blocks on its drain channel, it runs continuously and can't be GC'd (if I understand things correctly).



 Comments   
Comment by Leon Grapenthin [ 15/Nov/13 10:16 AM ]

I'd expect the behavior. Neither can to-chan know that (range) returns an infinite sequence, nor can it know that it's output channel has been closed.

Comment by Brian Lubeski [ 11/Feb/14 11:27 AM ]

Resolved in 0.1.278.0-76b25b-alpha.





[ASYNC-85] Add transducer support for ClojureScript Created: 13/Aug/14  Updated: 13/Aug/14  Resolved: 13/Aug/14

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Luke VanderHart Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: File cljs-transducers.diff    
Approval: Ok

 Description   

The attached patch adds transducer support, to keep ClojureScript APIs in sync with Clojure's (as far as possible).

Tests pass and the code is correct as far as I can discern but review is appreciated.



 Comments   
Comment by Luke VanderHart [ 13/Aug/14 12:58 PM ]

I pushed this patch directly after getting push access from Rich & Stu.





[ASYNC-13] go blocks don't handle explicitly namespaced calls to <! and >! Created: 22/Jul/13  Updated: 26/Jul/13  Resolved: 26/Jul/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Edward Cho Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.5.1"]
[org.clojure/clojurescript "0.0-1844"]
[org.clojure/core.async "0.1.0-SNAPSHOT"]



 Description   

When <Unable to render embedded object: File (, >) not found., or the go macro is explicitly namespaced, the assertion that <! or >! was called outside of a go block is raised.

Example:
(ns foo
(:require [cljs.core.async :as async)
(:require-macros [cljs.core.async.macros :refer [go]))

;; raises assertion error
(let [c (async/chan)]
(async/go
(loop []
(async/<! c)
(recur))))



 Comments   
Comment by Timothy Baldridge [ 26/Jul/13 9:34 AM ]

Fixed in master https://github.com/clojure/core.async/commit/12af7d8dc207a42f388936318dd2eeb48d1b03c8

Was a problem only in CLJS





[ASYNC-22] IOC macro does not handle Dot special form Created: 09/Sep/13  Updated: 04/Oct/13  Resolved: 04/Oct/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Ben Mabey Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None
Environment:

I've tested with using the latest snapshot of core.async and latest on github.



 Description   

This code:

(go (. clojure.lang.Numbers (add 1 2)))

Results in this error:

CompilerException java.lang.RuntimeException: Unable to resolve symbol: add in this context, ...

Looking at the expansion of the state machine the problem seems to be that SSA transformation is treating the (add 1 2) as a separate expression that needs to be evaluated (and assigned) before the clojure.lang.Numbers:

...
(clojure.core/let [inst_4648 (add 1 2)
                   inst_4649 (. clojure.lang.Numbers inst_4648)
                   state_4651 state_4651]
...

The newer / form works fine, e.g. (Math/abs -2), and that is how I would write static method calls myself but I am using macros that expand to the older dot syntax above.

Do you see this as a case the IOC macro should be handling?



 Comments   
Comment by Timothy Baldridge [ 04/Oct/13 1:42 PM ]

Fixed in master





[ASYNC-12] Add chan? or channel? function to identify a channel Created: 19/Jul/13  Updated: 12/Dec/13  Resolved: 12/Dec/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Brenton Ashworth Assignee: Timothy Baldridge
Resolution: Declined Votes: 0
Labels: None


 Description   

There are cases where one may be handed a value which could possibly be a channel. It would be nice to have a function which returns true if a value is a channel. This can currently be done from user code but requires reaching into the clojure.core.async.impl.channels namespace and depending on an implementation detail.

(defn channel? [c]
(satisfies? channels/MMC c))



 Comments   
Comment by Hugo Duncan [ 26/Sep/13 9:20 AM ]

I ran into the need for this when writing a schema to validate input arguments.

Comment by Timothy Baldridge [ 12/Dec/13 10:37 PM ]

We're rejecting this feature for now. If you need to check if an object implements the correct interfaces, feel free to dive into clojure.core.async.impl.protocols and check for satisfies? against ReadPort and WritePort. Not only do these allow you to accept ReadPorts that aren't channels, but it also keeps the main interface from getting cluttered.

At least this was the argument Rich proposed last time we talked. If this won't work, feel free to submit a new ticket or start a clojure-dev mailing list thread.





Generated at Thu Oct 02 03:30:00 CDT 2014 using JIRA 4.4#649-r158309.