Design page for Promises / Futures with callbacks.
promise provide implementations of
deref that block the calling thread. There is currently no facility to invoke callback functions on values when they become available. Asynchronous styles of programming are not possible without this facility, especially in single-threaded runtimes such as ClojureScript.
future will return objects implementating a new protocol,
INotify, supporting a function
attend with two arities:
attend [promise fn] [promise fn executor]
This function attaches a callback to the promise/future which will be invoked on the executor when the promise is delivered with a value. (attend, transitive verb, "occur with or as a result of")
Promises also implement the new protocol
IDeliver, supporting two functions:
deliver [promise value] fail [promise exception]
deliver function provides a value to the promise, invokes any pending callbacks, and releases any pending derefs. The
fail function delivers an exception to the promise (see "Exceptions" below).
The behavior of
deref on futures/promises does not change, except that a failed promise may throw an exception (see below).
Higher-Level Functions & Macros
follow function has the same signatures as
attend, but it invokes the callback with the value of the promise as its argument. It does not invoke the callback if the promise failed. The follow function returns a new promise, which will be delivered with the result of executing the callback. If the callback throws an exception, or if the source promise was failed, the promise returned by follow will fail with the same exception.
The follow function can be adapted into a macro in two different ways:
on [[binding-form promise] & body]
then [promise binding-form & body]
on macro is more consistent with other Clojure binding forms, whereas the
then macro facilitates composition with the
I'm not sure about having both
then, we should probably pick one or the other.
If not supplied,
executor defaults to a dynamic Var,
*callback-executor*, resolved when the callback is created, that is, at the time
attend was called. This Var is initially set to the Agent send-off thread pool executor, and can be globally reset or thread-bound by applications.
Running Callbacks Locally
If callbacks are known to be fast, it may be more efficient to run them directly on the thread that invoked
deliver. This is enabled by calling
attend with a
nil executor (or binding
A callback function (really, any function) can do one of three things:
- Return a value
- Throw an exception
- Never return at all
Failing to return is almost certainly a bug anywhere execpt on the main application thread. I assume that returning a value indicates a successful execution, while throwing an exception indicates failure. However, it is also possible to return an exception object as a value, which does not necessarily indicate failure. (For example, a logging framework might handle exceptions as values.)
Given that throwing an exception is different from returning a value, promises need to handle it on a different path. This is the
fail function, which acts like
deliver but takes an exception as its argument and puts the promise in a "failed" state.
When it comes time to invoke callback functions, there are several possible ways to handle failure:
- Provide separate callback attachment points for success and failure
- Pass the exception to the callback as a normal value
- Wrap the value or exception in a union type
Method 1 leads to a proliferation of error-handling functions. Method 2 makes it impossible to tell if the exception object was thrown or delivered as a normal value. Method 3 is more natural in a statically-typed language. In both 2 and 3 the callback function must always check whether its argument is a normal value or an exception.
I have chosen a variant of 3 by using the promise itself as the argument to the callback function. The callback is responsible for calling
deref on the promise. If the promise is in a failed state, dereferencing it will throw the exception. Callback functions can use standard try/catch blocks to handle the exception or allow it to propagate. Macros such as
on handle propagation automatically and allow users to program in terms of values.
Delivering a promise A as a value to another promise B causes B to
attend on A. Whenever A is delivered/failed, B is delivered/failed with the same value. This makes it convenient to implement multiple try/retry operations with promises, as in this example:
There is no API to remove or cancel a callback created with
attend. Instead, the
on macro returns a promise which receives return value of the return value of the callback function. If you deliver a value to that returned promise before the underlying promise is delivered, the callback function will not be executed. There is an implicit race condition with cancellation, so this cannot guarantee that the callback function will never be executed at all.
Futures can still be cancelled with
future-cancel, which attempts to stop the Future's thread.
- Promise/Futures with callbacks (Clojure group)
- lamina and channels-driven concurrency with clojure (Clojure group)
- reactive programing (dev.clojure.org)
- can a better Future be exposed by API? (Datomic group)
- cljque promises (Stuart Sierra)
References / Related Work
- SIP-14: Futures and Promises (Scala)
- Listenable Futures in Google Guava (Java)
- ListenableFuture in Async HTTP Client (Java)
- Cancellation in Managed Threads (.NET)
- Defer (Python)
- Deferred cancellation (Python)