Error formatting macro: pagetree: java.lang.NullPointerException

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This document describes work in progress in the master (1.3) branch. This work is not promised to become part of Clojure, as is or otherwise, now or in the future. The purpose of this document is to explain the work and encourage discussion.

Objective - Support primitives as fn arguments and return values

  • Clojure supports primitive arithmetic locally, and primitives in deftype/defrecord
    • But not in fn arguments or returns
  • Means you can’t break up your logic, or write helper functions, without boxing

Issues

  • fn is specified by an interface (IFn), taking/returning objects
    • fns must still satisfy that interface
  • How will callers know about primitive params/return, and how to invoke?
  • Overloading
    • we leverage in Java root ops (e.g. Numbers) but don't expose
    • even? as an example of something that would have to be primitivized in order to be efficient with all arg types

Objective - normalize arithmetic semantics between boxed and primitive numbers

Currently (1.2 and prior), operations between e.g. longs and Longs have different semantics. The latter can return a BigInteger on overflow while the former can't (and thus throws an exception)

  • Use (type flexible) equiv for =
    • but narrow equiv, only same category numbers are =
    • so floats not = to integers
      • but == still available for that
  • new clojure.lang.BigInt class
  • BigInts do not auto-reduce, and are contagious (akin to doubles)
    • BigInts will enable optimizations when fits in long
      • optimzations not yet in place
    • unlike BigInteger, BigInt has consistent hashcodes with Long, through range of long
  • all long-or-smaller integers are boxed as Long
    • You can't use primitive coercions to force specific box types
      • Your code was broken if you were relying on that
  • hash maps and sets now use = for keys
    • this will make maps and sets unify 42 and 42N, since using =
    • will use stricter .equals when calling through java.util interfaces
      • not there yet
      • this will require renaming Associative.containsKey, since semantics will now differ
  • +' (prime ops, see below) are available when auto-promotion is required, but they are not needed for polymorphic arithmetic
  • The plan is to offer 2 sets of ops for the few ops that might overflow: +, -, *, inc, dec.
  • +, -, *, inc, dec have been enhanced. They will never auto-promote on integer overflow, and will instead throw an exception. Thus, the semantics of these operators are now uniform when used with boxed or primitive arguments. They can return integer primitives.
  • ' is now a constituent character. That means it may appear as a character in symbols in other than the first position, as can #.
  • +', -', *', inc', dec' For all arguments other than long-or-smaller integers, they do exactly what their non-prime counterparts do. For long-or-smaller integers, they will auto-promote on overflow in all cases, even when given integer primitives. As such, they will never return integer primitives. They can and do return double primitives when given doubles, as the semantics match. Unless you need to work with arbitrary-precision integers where BigInteger contagion is insufficient, there is no reason to use them.
  • literal numbers are always implicitly primitives, and never need (long 42) style coercion except for host interop overloading disambiguation.
    • Note: this means that locals initialized with literals will have primitive type, e.g. (let [x 42] ...), and especially: (loop [x 42] ...). If you intend to recur with a non-primitive, init like this, (loop [x (num 42)] ...)

Performance

Code Block
(defn fib [n]
  (if (<= n 1)
    1
    (+ (fib (dec n)) (fib (- n 2)))))

(time (fib 38))
"Elapsed time: 2973 msecs"

;; hint arg and return
(defn fib ^long [^long n]
  (if (<= n 1)
    1
    (+ (fib (dec n)) (fib (- n 2)))))

(time (fib 38))
"Elapsed time: 390 msecs"

;Use promoting op' ops for arbitrary precision
(defn fib [n]
  (if (<= n 1)
    1
    (+' (fib (dec' n)) (fib (-' n 2)))))

(time (fib 38))
"Elapsed time: 3092 msecs"

;; double arg/return
(defn fib ^double [^double n]
  (if (<= n 1)
    1
    (+ (fib (dec n)) (fib (- n 2)))))

(time (fib 38))
"Elapsed time: 944 msecs"

;; double arg/return, double operands
(defn fib ^double [^double n]
  (if (<= n 1.0)
    1.0
    (+ (fib (dec n)) (fib (- n 2.0)))))

(time (fib 38))
"Elapsed time: 448 msecs"

Promotion Options

Code Block
(set! *warn-on-reflection* false)

(defn fact [n]
  (loop [cnt n acc 1]
    (if (zero? cnt)
      acc
      (recur (dec cnt) (* acc cnt)))))

#'user/fact

(set! *warn-on-reflection* true)

(defn fact [n]
  (loop [cnt n acc 1]
    (if (zero? cnt)
      acc
      (recur (dec cnt) (* acc cnt)))))

NO_SOURCE_FILE:1738 recur arg for primitive local: acc is not matching primitive, had: Object, needed: long
Auto-boxing loop arg: acc
#'user/fact

;; hinted version, non-promoting
(defn fact [^long n]
  (loop [cnt n acc 1]
    (if (zero? cnt)
      acc
      (recur (dec cnt) (* acc cnt)))))

 #'user/fact
;;no warning

(fact 12)
479001600

;; but not auto-promoting
(fact 42)
ArithmeticException integer overflow  clojure.lang.Numbers.throwIntOverflow (Numbers.java:1575)

;; non-hinted version *is* general
(set! *warn-on-reflection* false)

(defn fact [n]
  (loop [cnt n acc 1]
    (if (zero? cnt)
      acc
      (recur (dec cnt) (* acc cnt)))))

;; caller can just leverage BigInt contagion
(fact 42N)
1405006117752879898543142606244511569936384000000000N

;; or, force promoting operations with prime (') ops
(defn fact [n]
  (loop [cnt n acc 1]
    (if (zero? cnt)
      acc
      (recur (dec' cnt) (*' acc cnt)))))

(fact 42)
1405006117752879898543142606244511569936384000000000N