Clojure

Significant performance regression of code loaded in user.clj in Java 8u202 / 11.0.2

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: Release 1.10.1
  • Component/s: None
  • Environment:
    Java 8u201 and Java 11.0.2
  • Patch:
    Code
  • Approval:
    Ok

Description

The recent releases of Java 8u201/8u202 and 11.0.2 include a fix for a 0-day vulnerability that could occur while running a class static initializer. The fix included in these releases prevents JVM optimization during the static initializer (before the class is initialized). Doing significant work in the static initializer can see dramatic reduction in performance from the previous versions 8u192 / 11.0.1.

Clojure supports a well-known file user.clj that is loaded during runtime initialization (in the clojure.lang.RT class). This is sometimes used to load tools during development. Currently user.clj is loaded in the scope of RT's static inititializer, so any Clojure code run in user.clj is significantly slower.

One common pattern popularized by Stuart Sierra is the "reloaded" pattern which will load and initialize the entire server from the user.clj. Users of this pattern are likely to see dramatic startup time performance issues with these Java releases.

Links:

Repro: see https://github.com/puredanger/slow-user-load

Proposed:

Move the clojure.core init from doInit() into the RT static initializer, and then force explicit invocation of RT.doInit() in all expected RT entry points:

  • clojure.main/main - main launcher stub
  • clojure.java.api.Clojure<clinit> - Java API (this path will still incur the cost via Clojure<clinit>)
  • clojure.lang.Compile/main - used only in Clojure's own build afaik
  • clojure.lang.Util/loadWithClass - gen-class/AOT class static initializer

Patch: clj-2484-5.patch

Other alternatives considered:

1) Another option is to take the hints from the scope of the issue at https://cl4es.github.io/2019/02/21/Cljinit-Woes.html and avoid calling static methods on an uninitialized class. The basic idea here is to move all of the RT static methods into an inner class RT.Impl. That class can fully load so all methods are initialized. The RT static init can then just invoke doInit() on it. All references to RT static methods in Clojure must be rerouted from RT to RT.Impl. The existing RT methods are left and forward to RT.Impl - this allowed previously compiled Clojure classes to continue to work (or any stray RT calls made by advanced Clojure code).

Patch: clj-2484-nested-class.patch - this works, but is an enormous patch as it touches all of RT and every call to RT in Clojure.

2) Do nothing and wait for Java to mitigate the perf impact. At this point, it does not look like a simple fix.

  1. clj-2484-1.patch
    21/Feb/19 1:02 AM
    3 kB
    Alex Miller
  2. clj-2484-3.patch
    27/Feb/19 3:41 PM
    6 kB
    Alex Miller
  3. clj-2484-4.patch
    15/Mar/19 11:44 AM
    9 kB
    Alex Miller
  4. clj-2484-5.patch
    19/Mar/19 10:02 AM
    10 kB
    Alex Miller
  5. clj-2484-nested-class.patch
    21/Feb/19 3:31 PM
    398 kB
    Alex Miller

Activity

Hide
Alex Miller added a comment -

The Java bug http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8219233 is accumulating a lot of additional information if you are looking to follow updates on the Java side. Seems to be narrowing specifically to static initializers that make calls to static methods in same class (this is particularly why user.clj loading in RT.init is impacted - loading all that Clojure code makes tons of self calls into the runtime).

Show
Alex Miller added a comment - The Java bug http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8219233 is accumulating a lot of additional information if you are looking to follow updates on the Java side. Seems to be narrowing specifically to static initializers that make calls to static methods in same class (this is particularly why user.clj loading in RT.init is impacted - loading all that Clojure code makes tons of self calls into the runtime).
Hide
Alex Miller added a comment -

New -4 patch uses sneakyThrow to reduce exception handling stuff.

Show
Alex Miller added a comment - New -4 patch uses sneakyThrow to reduce exception handling stuff.
Hide
Alex Miller added a comment -

Added -5 that keeps RT.doInit() private and re-activates existing RT.init().

Show
Alex Miller added a comment - Added -5 that keeps RT.doInit() private and re-activates existing RT.init().

People

Vote (5)
Watch (7)

Dates

  • Created:
    Updated:
    Resolved: