ClojureScript

Perhaps a compiler issue?

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Not Reproducible
  • Affects Version/s: 1.10.238
  • Fix Version/s: None
  • Component/s: None
  • Labels:
  • Environment:
    Sorry, but we're not sure. We use CircleCI which is likely a linux flavor.

    shadow-cljs is our build tool at 2.4.10.

Description

It appears that we very occasionally get generated javascript that is invalid. Invalid here means it will compile and run, but during runtime it will throw an error because of missing variable names / null or undefined.

It's really hard / impossible to reliably recreate, and as mentioned only seems to happen once out of every few hundred builds.

The code in question:

(defn priorities-last-modified [priorities]
  (last (sort (concat
                (map :last-modified-date priorities)
                (mapcat
                  (comp (partial map :last-modified-date) :goals) priorities)))))

(Yes, this is terrible code but it works 99% of the time.)

The erring output looks like this:

jT(
    v([
      fG,
      function(a) {
        a = uh(a, new E(null, 4, 5, F, [Iu, Zk, fy, LF], null));
        a = Lf(BH, a);
        var b = yg.j(
          G.j(gs, priorities__$1),
          mh(Xg.j(Yg.j(G, gs), GA), v([priorities__$1]))
        );
        b = Jf(Gf, b);
        return new h(null, 2, [LF, a, MN, We(b)], null);
      }
    ])
  );

A recompile fixes the output and spits out this:

hT(
    v([
      eG,
      function(a) {
        var b = uh(a, new E(null, 4, 5, F, [Hu, $k, dy, KF], null));
        a = Lf(zH, b);
        b = We(Jf(Gf, yg.j(G.j(fs, b), mh(Xg.j(Yg.j(G, fs), FA), v([b])))));
        return new h(null, 2, [KF, a, NN, b], null);
      }
    ])
  );

The issue from the first compilation results in a runtime error of "priorities__$1 is not defined"

We are using

Activity

Hide
Luke Horton added a comment -

Will provide a minimum reproducible project shortly.

Show
Luke Horton added a comment - Will provide a minimum reproducible project shortly.
Hide
Mike Fikes added a comment -

Does your project have some other namespace that starts off with priorities.? (Such as priorities.core.)
Are you using the :parallel-build compiler option set to true?

Show
Mike Fikes added a comment - Does your project have some other namespace that starts off with priorities.? (Such as priorities.core.) Are you using the :parallel-build compiler option set to true?
Hide
Luke Horton added a comment -

Yes to both counts:
```
src/priorities/views.cljs
1:(ns priorities.views

src/priorities/goals.cljs
1:(ns priorities.goals

src/priorities/specs.cljs
1:(ns priorities.specs

src/priorities/events.cljs
1:(ns priorities.events

src/priorities/core.cljs
1:(ns priorities.core
```

https://shadow-cljs.github.io/docs/UsersGuide.html#compiler-options (parallel builds is always enabled).

Show
Luke Horton added a comment - Yes to both counts: ``` src/priorities/views.cljs 1:(ns priorities.views src/priorities/goals.cljs 1:(ns priorities.goals src/priorities/specs.cljs 1:(ns priorities.specs src/priorities/events.cljs 1:(ns priorities.events src/priorities/core.cljs 1:(ns priorities.core ``` https://shadow-cljs.github.io/docs/UsersGuide.html#compiler-options (parallel builds is always enabled).
Hide
Luke Horton added a comment -

After closer inspection I'm fairly confident the above code working/not-working are at least pointing to the same compiled fn. We use keywords "goals" and "order", which are allocated in the working file as:
```
FA = new C(null, "goals", "goals", -1712076283),
zH = new C(null, "order", "order", -1254677256),
```
and we can see that the output code makes use of both `zH` and `FA`.

Show
Luke Horton added a comment - After closer inspection I'm fairly confident the above code working/not-working are at least pointing to the same compiled fn. We use keywords "goals" and "order", which are allocated in the working file as: ``` FA = new C(null, "goals", "goals", -1712076283), zH = new C(null, "order", "order", -1254677256), ``` and we can see that the output code makes use of both `zH` and `FA`.
Hide
David Nolen added a comment -

This is just not minimal enough to consider

Show
David Nolen added a comment - This is just not minimal enough to consider
Hide
Thomas Heller added a comment -

FWIW this is an actual CLJS compiler race condition.

cljs.compiler/munge decides to munge a variable if a namespace "root" starts with that variable name but it decides that based on the currently known namespaces. Due to parallel compilation a namespace might be added while still emitting one form so at the beginning it would not munge but then later on it would. This is extremely difficult to reproduce reliable due to the time window where this may happen.

(ns demo.bug)

(let [bar 1]
  (do-other-stuff)
  (do-with-bar bar))

;; elsewhere parallel compilation begins
(ns bar.app)
...

Resulting code somewhat looks like

var bar = 1;
demo.bug.do_other_stuff();
demo.bug.do_with_bar(bar__$1);

This was sort of fixed accidentally recently [1]. Since the memoize ensures that at least the same munge/unmunged name is used throughout the file. Not sure anything needs to be done but the strategy for choosing which namespaces to munge seems rather brittle considering this.

[1] https://github.com/clojure/clojurescript/commit/284872fb50cac0bd3a5d13c8e3aeaecd3481e03f#diff-55b85385d2d0bfb6dc20d59ed982d5c8R1477

Show
Thomas Heller added a comment - FWIW this is an actual CLJS compiler race condition. cljs.compiler/munge decides to munge a variable if a namespace "root" starts with that variable name but it decides that based on the currently known namespaces. Due to parallel compilation a namespace might be added while still emitting one form so at the beginning it would not munge but then later on it would. This is extremely difficult to reproduce reliable due to the time window where this may happen.
(ns demo.bug)

(let [bar 1]
  (do-other-stuff)
  (do-with-bar bar))

;; elsewhere parallel compilation begins
(ns bar.app)
...
Resulting code somewhat looks like
var bar = 1;
demo.bug.do_other_stuff();
demo.bug.do_with_bar(bar__$1);
This was sort of fixed accidentally recently [1]. Since the memoize ensures that at least the same munge/unmunged name is used throughout the file. Not sure anything needs to be done but the strategy for choosing which namespaces to munge seems rather brittle considering this. [1] https://github.com/clojure/clojurescript/commit/284872fb50cac0bd3a5d13c8e3aeaecd3481e03f#diff-55b85385d2d0bfb6dc20d59ed982d5c8R1477

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: