ClojureScript

Possible race condition in compiler w/ parallel-build true

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

I'm seeing errors trying to compile DataScript on 1.9.473. Judging from output, it looks like a race error in ClojureScript compiler: output JS files are damaged, removing parallel-build true fixes the problem.

Due to the nature of error, it might not be 100% reproducible. I get it 100% times on my machine though, but reported syntax errors in JS are always different.

The problem was probably introduced in 1.9.229. I'm not getting any errors compiling DataScript in earlier versions (1.9.227 and before)

HOW TO REPRODUCE:

unzip datascript_compiler_race.zip
cd datascript_compiler_race
wget https://github.com/clojure/clojurescript/releases/download/r1.9.473/cljs.jar
rm -rf target && java -cp cljs.jar:src clojure.main build.clj

As this is a race condition, it might not reproduce perfectly every time.

Examples of what I get on my MacBook Pro Retina Mid 2012 (Core i7-3615QM, 4 cores):

[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj
WARNING: parse-binding at line 215 is being replaced at line 215 src/datascript/parser.cljc
WARNING: collect-vars-acc at line 470 is being replaced at line 470 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
Feb 21, 2017 1:07:13 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/target/datascript/parser.js:140: ERROR - Parse error. Semi-colon expected
datascript.parser.collect.cljsdatascript.parser.collect.cljs$lang$mavar G__16546 = cljs.datascript.parser.collvar seq16545__$1 = cljs.core.next(seq16545);
                                                                        ^

Feb 21, 2017 1:07:13 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/target/datascript/parser.js:1676: ERROR - Parse error. IE8 (and below) will parse trailing commas in array and object literals incorrectly. If you are targeting newer versions of JS, set the appropriate language_in option.
return faif(cljs.coredatascript.parser.RuleVars.prototype.cljs$core$IMap$_dissoc$arity$2 = (function (this__7667__auto__,k__7668return ((this__7654__auto____$1.constructor === other__7655__autoif(cljs.core.contains_QMARK_(new cljs.core.PersistentHashSet(null, new cljs.core.PersistentArrayMap(null, 2, [cljs.core.cst$kw$free,return true;
                                                                                                                                                                                                                                                                                                                                    ^

Feb 21, 2017 1:07:13 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 2 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. Semi-colon expected at /Users/prokopov/datascript_compiler_race/target/datascript/parser.js line 140 : 72
ERROR: JSC_TRAILING_COMMA. Parse error. IE8 (and below) will parse trailing commas in array and object literals incorrectly. If you are targeting newer versions of JS, set the appropriate language_in option. at /Users/prokopov/datascript_compiler_race/target/datascript/parser.js line 1676 : 324
[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj
WARNING: parse-binding at line 215 is being replaced at line 215 src/datascript/parser.cljc
WARNING: parse-clause at line 598 is being replaced at line 598 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
WARNING: parse-clauses at line 611 is being replaced at line 611 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
Feb 21, 2017 1:05:50 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/target/datascript/parser.js:458: ERROR - Parse error. Semi-colon expected
datasreturn cljs.core.pr_sequential_writer(writer__7673__auto__,pr_pair__7675__auto__,"#datascript.parser.Variable{",", ","}",opts__7674__auto__,cljs.core.concat.cljs$core$IFn$_invoke$arity$2(new cljs.core.PersistentVector(null, 1, 5return (new cljs.core.RecordIter((0),G__16721__$1,1,new cljs.core.PersistentVector(null, 1, 5, cljs.core.PersistentVector.EMPTY_NODE, [cljs.core.cst$kw$symbol], null),(cljs.core.truth_(self__.__extmap)?cljs.core._iterator(self__.__extmap):cljs.core.nil_iter())));
            ^

Feb 21, 2017 1:05:50 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/target/datascript/parser.js:4080: ERROR - Parse error. IE8 (and below) will parse trailing commas in array and object literals incorrectly. If you are targeting newer versions of JS, set the appropriate language_in option.
throw cljs.core.ex_info.cljs$core$IFn$_invoke$arity$2([cljs.core.str.cljs$cothrow cljs.core.ex_info.cljs$core$IFn$_invoke$arity$2([cljs.core.str.cljs$core$IFn$_invoke$arity$1("Cannot parse pull expression, expect ['pull' src-var? variable (constant | variable | plain-symbol)]")].join(''),new cljs.core.PersistentArrayMap(null, 2, [cljs.core.cst$kw$error,}
                                                                                                                                                                                                                                                                                                                                                                  ^

Feb 21, 2017 1:05:50 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 2 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. Semi-colon expected at /Users/prokopov/datascript_compiler_race/target/datascript/parser.js line 458 : 12
ERROR: JSC_TRAILING_COMMA. Parse error. IE8 (and below) will parse trailing commas in array and object literals incorrectly. If you are targeting newer versions of JS, set the appropriate language_in option. at /Users/prokopov/datascript_compiler_race/target/datascript/parser.js line 4080 : 354
[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj
WARNING: parse-binding at line 215 is being replaced at line 215 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
Feb 21, 2017 1:02:55 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/target/datascript/parser.js:4507: ERROR - Parse error. ',' expected
throw cljs.core.ex_info.cljs$core$IFn$_invoke$arity$2([cljs.core.str.cljs$core$IFn$_invoke$arity$1("Cannot parse :find, expected: (find-rel | find-coll | find-tuple | find-scalar)")].join(''),new cljs.core.PersistentArrayMap(null, 2, [cljs.core.cst$kw$error,cljs.core.cst$kw$parser_SLASH_find,cljs.core.cst$kw$fragment,forvar or__6983__}
                                                                                                                                                                                                                                                                                                                                      ^

Feb 21, 2017 1:02:55 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. ',' expected at /Users/prokopov/datascript_compiler_race/target/datascript/parser.js line 4507 : 326
[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj
WARNING: collect-vars-acc at line 470 is being replaced at line 470 src/datascript/parser.cljc
WARNING: parse-clause at line 598 is being replaced at line 598 src/datascript/parser.cljc
WARNING: parse-clauses at line 611 is being replaced at line 611 src/datascript/parser.cljc
Feb 21, 2017 1:02:00 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/target/datascript/parser.js:227: ERROR - Parse error. ',' expected
this.datascript.parser.Placeholder.prototype.cljs$core$ILookup$_lookup$arity$2 = (function (this__7658_this.cljs$lang$protocol_mask$partition1$ = 81var thidatascript.parser.Placeholdereturn this__7658__auto____$1.cljs$core$ILookup$_lookup$arity$3(null,k__7659__auto__,null);
                                                                                                           ^

Feb 21, 2017 1:02:00 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. ',' expected at /Users/prokopov/datascript_compiler_race/target/datascript/parser.js line 227 : 107
[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj
WARNING: parse-binding at line 215 is being replaced at line 215 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
Feb 21, 2017 1:00:46 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/out/datascript/parser.js:215: ERROR - Parse error. Character '@' (U+0040) is not a valid identifier start char
datascript.parser.Placeholder = (functio * @implements {cljs.core.IAthis.__meta  * @implemthis.__extmap = __extm * @this.__hash = __hash;
                                           ^

Feb 21, 2017 1:00:46 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. Character '@' (U+0040) is not a valid identifier start char at /Users/prokopov/datascript_compiler_race/out/datascript/parser.js line 215 : 43
[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj
WARNING: collect-vars-acc at line 470 is being replaced at line 470 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
WARNING: parse-clause at line 598 is being replaced at line 598 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
WARNING: parse-clauses at line 611 is being replaced at line 611 /Users/prokopov/datascript_compiler_race/src/datascript/parser.cljc
Feb 21, 2017 12:59:33 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/prokopov/datascript_compiler_race/out/datascript/parser.js:7639: ERROR - Parse error. '}' expected

Feb 21, 2017 12:59:33 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. '}' expected at /Users/prokopov/datascript_compiler_race/out/datascript/parser.js line 7639 : 0

Note how output JS is not even syntactically correct:

functio * @implements
__meta  * @implemthis.__extmap
,forvar or__6983__}

This is a clear sign that two or more threads are writing to the same file without coordination.

Also note that with parallel build enabled, compiler confuses declare with function defenition and warns about collect-vars-acc at line 470 is being replaced at line 470. In fact, the function is first decalre-d, then defn-ed just once in the same file, which, under normal circumstances, should not trigger any warnings.

Also note that setting :parallel-build false leads to 100% stable, error-less builds:

[~/datascript_compiler_race] rm -rf target && java -cp cljs.jar:src clojure.main build.clj --no-parallel
[~/datascript_compiler_race]

Activity

Hide
David Nolen added a comment -

This issue needs work before it will be considered. Do not link outside to minimal cases, you need to provide all the information in the ticket.

Show
David Nolen added a comment - This issue needs work before it will be considered. Do not link outside to minimal cases, you need to provide all the information in the ticket.
Hide
Nikita Prokopov added a comment -

David, thanks for the heads up! I’ve updated the issue with standalone zip containing test case and instructions on how to build it. Unfortunately I cannot include cljs.jar in it because it’s too big (24 Mb, while JIRA only allows for 10 Mb attachments max). But I’m sure you have one somewhere on your machine

Show
Nikita Prokopov added a comment - David, thanks for the heads up! I’ve updated the issue with standalone zip containing test case and instructions on how to build it. Unfortunately I cannot include cljs.jar in it because it’s too big (24 Mb, while JIRA only allows for 10 Mb attachments max). But I’m sure you have one somewhere on your machine
Hide
David Nolen added a comment - - edited

The issue appears to be that somehow duplicate inputs are present, that is

datascript/pull_parser.cljc
will get compiled twice.

Show
David Nolen added a comment - - edited The issue appears to be that somehow duplicate inputs are present, that is
datascript/pull_parser.cljc
will get compiled twice.
Hide
Nikita Prokopov added a comment -

Is it because of how cljc is handled? I don’t see duplicate file names in that repo

Show
Nikita Prokopov added a comment - Is it because of how cljc is handled? I don’t see duplicate file names in that repo
Hide
Nikita Prokopov added a comment -

BTW confirm that 1.9.494 fixes my issue. Thanks a lot!

Show
Nikita Prokopov added a comment - BTW confirm that 1.9.494 fixes my issue. Thanks a lot!
Hide
Pavel Kopychenko added a comment -

Hi all!! I'm sorry! I have simular error when I try to compile production after update clojurescript to version 1.9.494 from 1.9.229.

Compiling "resources/public/js/c/camerton/testor/main.js" from ["src-cljc" "src-cljs-common" "src-cljs-testor"]...
Feb 25, 2017 3:40:05 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /home/maker/git/camerton/target/cljsbuild-compiler-3/cljs/core/async.js:1426: ERROR - Parse error. No newline allowed before '=>'
var inst_56769 = async(inst_56768);
                                  ^

Feb 25, 2017 3:40:05 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. No newline allowed before '=>' at /home/maker/git/camerton/target/cljsbuild-compiler-3/cljs/core/async.js line 1426 : 34
Show
Pavel Kopychenko added a comment - Hi all!! I'm sorry! I have simular error when I try to compile production after update clojurescript to version 1.9.494 from 1.9.229.
Compiling "resources/public/js/c/camerton/testor/main.js" from ["src-cljc" "src-cljs-common" "src-cljs-testor"]...
Feb 25, 2017 3:40:05 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /home/maker/git/camerton/target/cljsbuild-compiler-3/cljs/core/async.js:1426: ERROR - Parse error. No newline allowed before '=>'
var inst_56769 = async(inst_56768);
                                  ^

Feb 25, 2017 3:40:05 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_PARSE_ERROR. Parse error. No newline allowed before '=>' at /home/maker/git/camerton/target/cljsbuild-compiler-3/cljs/core/async.js line 1426 : 34
Hide
António Nuno Monteiro added a comment -

@nikita not related to cljc files in particular, just the way we sort dependencies could make us end up with duplicate entries. We now calculate `distinct` by keying on the namespace that the entry provides.

@pavel that's not related to this issue, but I'm also seeing it. Please open a new ticket. It's most likely related to the Closure Compiler upgrade.

Show
António Nuno Monteiro added a comment - @nikita not related to cljc files in particular, just the way we sort dependencies could make us end up with duplicate entries. We now calculate `distinct` by keying on the namespace that the entry provides. @pavel that's not related to this issue, but I'm also seeing it. Please open a new ticket. It's most likely related to the Closure Compiler upgrade.

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: