Allow user supplied executor/thread-pool (and potentially other options) to be used in go, thread macros and chan
Description
Environment
Attachments
Activity
Alex Miller March 31, 2025 at 7:30 PM
As of the latest 1.8 betas, you can now supply custom executors for all of these via the clojure.core.async.executor-factory.
Stuart Halloway June 7, 2023 at 2:25 PM
Related: How can a user control the thread pool used by core.async? If you are working in an environment where and app or container manages thread pools, how would you let it control async’s thread pool?
Needing an alter-var-root hack is not great.
Max Penet November 27, 2018 at 3:53 PM
After re-reading the details of this issue I created years ago I am actually now agreeing with both Stu and Ghadi comments, I think my understanding of core.async go internals is a lot better now, threading at this level (and callbacks) is merely for "supervision"/context switching, so fine grained control over these is not so important. I think I was probably influenced by the design of manifold, which differs quite a bit in that respect (and other ways), but these 2 libraries are quite different.
I think there is still a case for a better thread-call function and potentially another macro like thread that takes a user supplied executor.
Would a patch that adds an arity to `tread-call` to allow to pass the executor be considered?
What form could take a macro version of that new thread-call arity (if it's something wanted; I think a new macro would make more sense as thread takes a &body that might be a bit tricky to retrofit there).
Would a patch that makes the various threadpool initialization "lazy" (delayed) be considered?
Sameer Rahmani May 8, 2018 at 9:21 PM
Hey guys,
I added couple of functions to threadpool ns in order to allow user to provide an executor for the main thread pool and the one for the `thread` macro.
It's a draft at the moment, I wanted to know other people opinion on it before spend more time on it. Please checkout the
commit on my repo at:
https://github.com/Codamic/core.async/commit/5cfa3ac34f963dc67329722149d041629a6c6e6f
It should gives you the idea. Looking forward to your opinion.
Cheers
Max Penet July 17, 2015 at 6:42 AM
Sure, as long as this is improved/fixed.
If you would read the patch, or the source for channels you would see
that channels run their callbacks on a threadpool.
When this patch was submitted I spent time discussing it with Timothy
Baldridge on IRC and made some changes at his suggestion. Since you
probably share offices with him I suggest you discuss the current
approach with him as he is more familiar with the codebase than both
of us.
Details
Details
Assignee
Reporter
Max PenetApproval
Patch
Priority
Major
Right now thread pools used by core.async are defined as globals, they are shared by all go or thread macro on their respective pools:
see https://github.com/clojure/core.async/blob/ac0f1bfb40237a18dc0f03c0db5df41657cd23a6/src/main/clojure/clojure/core/async.clj#L391-L411
and https://github.com/clojure/core.async/blob/ac0f1bfb40237a18dc0f03c0db5df41657cd23a6/src/main/clojure/clojure/core/async/impl/dispatch.clj#L16
It would be very useful to have more control over this, hopefully not via a simple setter, like the one we have in core for agents (set-agent-send-executor!), since it only allows one pool to be used at a time.
One possible solution would be to do that via an option map parameter to thread-call, and/or some other mechanism for go blocks (go-call with the same kind of signature). Making this parameter a map would also allow to further extend internals in the future, if necessary, without requiring heavy changes to the API.