ClojureScript

CLJS compilation error with Closure lib dependency on Windows

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Completed
  • Affects Version/s: 1.9.293
  • Fix Version/s: None
  • Component/s: None
  • Labels:
  • Environment:
    Tested on Windows 10 with java 8 and on windows 7 with java 7 and java 8.
    Clojure version is 1.8.0 on both systems.

Description

I encountered this error while developing a cljs project that included openlayers lib in its deps.
Open layers is a Closure library and I used the latest version 3.15.1.

The error that compiler reports is:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect

I have recreated the minimal scenario, the one like on ClojureScript Quick start guide.
Steps to reproduce the problem (basically like the quick start):

  • Download standalone cljs jar from the link in the guide
  • Create the same build.cljs file as defined on the guide
  • Download openlayers-3.15.1.jar and place it in the root
  • Add require statement in core.cljs - (ns hello-world.core (:require ol.Map)) for example
  • Than in cmd run java -cp "cljs.jar;openlayers-3.15.1.jar;src" clojure.main build.clj

I have also tested with cljs master branch (created an uberjar and tested again) and i get the same error.

Attached is the example stacktrace (in that run I added :verbose true to get the 'Copying...' output).

  1. CLJS-1868.patch
    03/Feb/17 8:23 AM
    1 kB
    Dejan Josifovic
  2. CLJS-1868.patch
    03/Feb/17 6:54 AM
    1 kB
    Dejan Josifovic
  3. stacktrace.txt
    11/Dec/16 8:05 AM
    4 kB
    Dejan Josifovic

Activity

Hide
Dejan Josifovic added a comment -

I have also created a git hub repo for this problem: https://github.com/28/openlayers-cljs-compile-error-repo

Show
Dejan Josifovic added a comment - I have also created a git hub repo for this problem: https://github.com/28/openlayers-cljs-compile-error-repo
Hide
Gijs Stuurman added a comment -

This issue is present on all OS's, but breaks on Windows. I see it on Ubuntu.

The logic that already exists to shorten filenames for files from jars is not applied for the library in the openlayer.jar.

In verbose mode there is this output:

Copying jar:file:/path/openlayers/openlayers-cljs-compile-error-repo/openlayers-3.15.1.jar!/cljsjs/openlayers/development/ol/events/event.js to out/file:/path/openlayers/openlayers-cljs-compile-error-repo/openlayers-3.15.1.jar!/cljsjs/openlayers/development/ol/events/event.js

The file that gets written into the "out" directory contains a colon (:) and an exclamation point (!) in the path (at the "file:" and "jar!"). On Windows this is a Java IO Exception, because colons are not allowed in file names or paths.

The generated "main.js" file contains:

goog.addDependency("../file:/path/openlayers/openlayers-cljs-compile-error-repo/openlayers-3.15.1.jar!/cljsjs/openlayers/development/ol/events/event.js", ['ol.events.Event'], []);

The "openlayers" jar is a Google Closure compatible JavaScript library in a jar. All its files have a "goog.provides" and "goog.require"'s and the jar contains a "deps.cljs" with the keys :libs and :externs.

The following monkey patch in "build.clj" makes paths without "file:" or "jar!" in them:

(alter-var-root #'cljs.closure/rel-output-path
                (fn [orig-fn]
                  (fn [js & args]
                    (apply orig-fn (dissoc js :closure-lib) args))))

For comparison, the verbose output for the files generated for files from cljs.jar:

Copying jar:file:/path/openlayers/openlayers-cljs-compile-error-repo/cljs.jar!/goog/base.js to out/goog/base.js
Show
Gijs Stuurman added a comment - This issue is present on all OS's, but breaks on Windows. I see it on Ubuntu. The logic that already exists to shorten filenames for files from jars is not applied for the library in the openlayer.jar. In verbose mode there is this output:
Copying jar:file:/path/openlayers/openlayers-cljs-compile-error-repo/openlayers-3.15.1.jar!/cljsjs/openlayers/development/ol/events/event.js to out/file:/path/openlayers/openlayers-cljs-compile-error-repo/openlayers-3.15.1.jar!/cljsjs/openlayers/development/ol/events/event.js
The file that gets written into the "out" directory contains a colon (:) and an exclamation point (!) in the path (at the "file:" and "jar!"). On Windows this is a Java IO Exception, because colons are not allowed in file names or paths. The generated "main.js" file contains:
goog.addDependency("../file:/path/openlayers/openlayers-cljs-compile-error-repo/openlayers-3.15.1.jar!/cljsjs/openlayers/development/ol/events/event.js", ['ol.events.Event'], []);
The "openlayers" jar is a Google Closure compatible JavaScript library in a jar. All its files have a "goog.provides" and "goog.require"'s and the jar contains a "deps.cljs" with the keys :libs and :externs. The following monkey patch in "build.clj" makes paths without "file:" or "jar!" in them:
(alter-var-root #'cljs.closure/rel-output-path
                (fn [orig-fn]
                  (fn [js & args]
                    (apply orig-fn (dissoc js :closure-lib) args))))
For comparison, the verbose output for the files generated for files from cljs.jar:
Copying jar:file:/path/openlayers/openlayers-cljs-compile-error-repo/cljs.jar!/goog/base.js to out/goog/base.js
Hide
David Nolen added a comment -

Gijs thanks for the additional information. Very helpful, will take a look.

Show
David Nolen added a comment - Gijs thanks for the additional information. Very helpful, will take a look.
Hide
David Nolen added a comment -

The suggested fix isn't desirable as far as I can tell since we have a branch specifically for dealing with Closure libs that will be avoided. We need a better explanation.

Show
David Nolen added a comment - The suggested fix isn't desirable as far as I can tell since we have a branch specifically for dealing with Closure libs that will be avoided. We need a better explanation.
Hide
Dejan Josifovic added a comment -

Hi all,

the problem occurred only for Closure libraries compilation (such as openlayers).
As already commented, it manifested on both Win and Linux, but on Linux it did not
cause problems.

The problem was in the function that produced relative output paths for deps (lib-rel-path).
It tried to remove the first part of the absolute path with clojure.string/replace, but
it tried it with a wrong match (it did not reproduce the actual first part of the path)
so it effectively did nothing.

Example:
(str (io/file (System/getProperty "user.dir") lib-path) File/separator)
produced
file:\C:\Users****\Documents\Projects\openlayers-cljs-compile-error-repo\cljsjs\openlayers\development\
instead of
file:\C:\Users****\Documents\Projects\openlayers-cljs-compile-error-repo\openlayers-3.15.1.jar!\cljsjs\openlayers\development\

Attached patch produces better paths that work on both OS.
Can someone check if it is an okay solution?
I have tested it of course on both OS, run CLJS tests etc.

Thanks,
Dejan.

Show
Dejan Josifovic added a comment - Hi all, the problem occurred only for Closure libraries compilation (such as openlayers). As already commented, it manifested on both Win and Linux, but on Linux it did not cause problems. The problem was in the function that produced relative output paths for deps (lib-rel-path). It tried to remove the first part of the absolute path with clojure.string/replace, but it tried it with a wrong match (it did not reproduce the actual first part of the path) so it effectively did nothing. Example: (str (io/file (System/getProperty "user.dir") lib-path) File/separator) produced file:\C:\Users****\Documents\Projects\openlayers-cljs-compile-error-repo\cljsjs\openlayers\development\ instead of file:\C:\Users****\Documents\Projects\openlayers-cljs-compile-error-repo\openlayers-3.15.1.jar!\cljsjs\openlayers\development\ Attached patch produces better paths that work on both OS. Can someone check if it is an okay solution? I have tested it of course on both OS, run CLJS tests etc. Thanks, Dejan.
Hide
Dejan Josifovic added a comment -

This is the correct patch - first one contained an error.

Show
Dejan Josifovic added a comment - This is the correct patch - first one contained an error.
Hide
Shaun Mahood added a comment -

I've tested the newest patch on Windows 10/Java 8 and Mac OSX 10.11.6/Java 8, using the quickstart compile method with both :none and :advanced.
It fixes the compilation issue on Windows and will compile both openlayers and libphonenumber (both are Google Closure libraries, I used the JAR files from CLJSJS to compile them). The folder structure also looks much better and doesn't have the user path embedded anymore.

Show
Shaun Mahood added a comment - I've tested the newest patch on Windows 10/Java 8 and Mac OSX 10.11.6/Java 8, using the quickstart compile method with both :none and :advanced. It fixes the compilation issue on Windows and will compile both openlayers and libphonenumber (both are Google Closure libraries, I used the JAR files from CLJSJS to compile them). The folder structure also looks much better and doesn't have the user path embedded anymore.

People

Vote (0)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: