<< Back to previous view

[CLJS-2936] Wrong order of lines in compiled javascript Created: 18/Oct/18  Updated: 19/Oct/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jan Břečka Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: npm-deps


 Description   

https://github.com/auth0/jwt-decode

This javascript code (simplified lib/index.js):

'use strict';
function InvalidTokenError(message) {}

InvalidTokenError.prototype = new Error();
InvalidTokenError.prototype.name = 'InvalidTokenError';

module.exports = function (token,options) {};

module.exports.InvalidTokenError = InvalidTokenError;

is compiled to:

var module$node_modules$jwt_decode$lib$index={};
module$node_modules$jwt_decode$lib$index["default"].InvalidTokenError=function(message){};
module$node_modules$jwt_decode$lib$index["default"].InvalidTokenError.prototype=new Error;
module$node_modules$jwt_decode$lib$index["default"].InvalidTokenError.prototype.name="InvalidTokenError";
module$node_modules$jwt_decode$lib$index["default"]=function(token,options){}

It's failing with "Uncaught TypeError: Cannot set property 'InvalidTokenError' of undefined" - second line. In original code the function is exported first and then InvalidTokenError is assigned while in compiled code InvalidTokenError is assigned first to not yet ready "default" property.






[CLJS-2792] CraftyJS NPM dependency cannot be imported Created: 25/Jun/18  Updated: 22/Jan/19

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: 1.10.238
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Marty Glaubitz Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: clojurescript, npm-deps
Environment:

Operation System: Windows 10
Leiningen Version: Leiningen 2.8.1
Java Version: Java 1.8.0_60 Java HotSpot(TM) 64-Bit Server VM
ClojureScript Version: 1.10.238



 Description   

I'm using ClojureScript with Figwheel and trying to use CraftyJs in ClojureScript.
This is my project.clj

(defproject my_project "0.1.0-SNAPSHOT"
:description "FIXME: write this!"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}

:min-lein-version "2.7.1"

:dependencies [[org.clojure/clojure "1.9.0"]
[org.clojure/clojurescript "1.10.238"]
[org.clojure/core.async "0.4.474"]]

:plugins [[lein-figwheel "0.5.16"]
[lein-cljsbuild "1.1.7" :exclusions [[org.clojure/clojure]]]]

:source-paths ["src"]

:cljsbuild {:builds
[{:id "dev"
:source-paths ["src"]

;; The presence of a :figwheel configuration here
;; will cause figwheel to inject the figwheel client
;; into your build
:figwheel {:on-jsload "my_project.core/on-js-reload"
;; :open-urls will pop open your application
;; in the default browser once Figwheel has
;; started and compiled your application.
;; Comment this out once it no longer serves you.
:open-urls ["http://localhost:3449/index.html"]}

:compiler {:main my_project.core
:asset-path "js/compiled/out"
:install-deps true
:npm-deps {:craftyjs "0.8.0"}
:output-to "resources/public/js/compiled/my_project.js"
:output-dir "resources/public/js/compiled/out"
:source-map-timestamp true
;; To console.log CLJS data-structures make sure you enable devtools in Chrome
;; https://github.com/binaryage/cljs-devtools
:preloads [devtools.preload]}}
;; This next build is a compressed minified build for
;; production. You can build this with:
;; lein cljsbuild once min
{:id "min"
:source-paths ["src"]
:compiler {:output-to "resources/public/js/compiled/my_project.js"
:main my_project.core
:optimizations :advanced
:pretty-print false}}]}

:figwheel {;; :http-server-root "public" ;; default and assumes "resources" ;; :server-port 3449 ;; default ;; :server-ip "127.0.0.1" :css-dirs ["resources/public/css"] ;; watch and update CSS ;; Start an nREPL server into the running figwheel process ;; :nrepl-port 7888 ;; Server Ring Handler (optional) ;; if you want to embed a ring handler into the figwheel http-kit ;; server, this is for simple ring servers, if this ;; doesn't work for you just run your own server :) (see lein-ring) ;; :ring-handler hello_world.server/handler ;; To be able to open files in your editor from the heads up display ;; you will need to put a script on your path. ;; that script will have to take a file path and a line number ;; ie. in ~/bin/myfile-opener ;; #! /bin/sh ;; emacsclient -n +$2 $1 ;; ;; :open-file-command "myfile-opener" ;; if you are using emacsclient you can just use ;; :open-file-command "emacsclient" ;; if you want to disable the REPL ;; :repl false ;; to configure a different figwheel logfile path ;; :server-logfile "tmp/logs/figwheel-logfile.log" ;; to pipe all the output to the repl ;; :server-logfile false }

;; Setting up nREPL for Figwheel and ClojureScript dev
;; Please see:
;; https://github.com/bhauman/lein-figwheel/wiki/Using-the-Figwheel-REPL-within-NRepl
:profiles {:dev {:dependencies [[binaryage/devtools "0.9.9"]
[figwheel-sidecar "0.5.16"]
[cider/piggieback "0.3.1"]]
;; need to add dev source path here to get user.clj loaded
:source-paths ["src" "dev"]
;; for CIDER
;; :plugins [[cider/cider-nrepl "0.12.0"]]
:repl-options {:nrepl-middleware [cider.piggieback/wrap-cljs-repl]}
;; need to add the compliled assets to the :clean-targets
:clean-targets ^{:protect false} ["resources/public/js/compiled"
:target-path]}})

However when running lein figwheel i see following in the console:

Compiling build :dev to "resources/public/js/compiled/my_project.js" from ["src"]...
[eval]:85
!id.startsWith(goog;
^^^^

SyntaxError: missing ) after argument list
at createScript (vm.js:74:10)
at Object.runInThisContext (vm.js:116:10)
at Object.<anonymous> ([eval]-wrapper:6:22)
at Module._compile (module.js:624:30)
at evalScript (bootstrap_node.js:480:27)
at startup (bootstrap_node.js:177:9)
at bootstrap_node.js:626:3

Successfully compiled build :dev to "resources/public/js/compiled/my_project.js" in 19.529 seconds.
and i can't import the library from my ClojureScript, i also see this:

Uncaught Error: Undefined nameToPath for craftyjs
at visitNode (base.js:1357)
at Object.goog.writeScripts_ (base.js:1369)
at Object.goog.require [as require_figwheel_backup_] (base.js:706)
at index.html:14
I already tried to manually delete the compiled JS output folder



 Comments   
Comment by Mike Fikes [ 30/Jun/18 7:05 PM ]

Hey Marty,

I think you can't directly use CraftyJS as an NPM dependency with ClojureScript. If you look on the CraftyJS website it shows additionally using Browserify.

Here is what happens if you try to use it as an NPM dep using minimal tooling:

deps.edn
{:deps {org.clojure/clojurescript {:mvn/version "1.10.339"}}}
compiler-opts.edn
{:npm-deps {:craftyjs "0.8.0"}
 :install-deps true}
$ clj -m cljs.main -co compiler-opts.edn -r
ClojureScript 1.10.339
cljs.user=> (require 'craftyjs)
events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: Can't resolve 'fs' in '/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/craftyjs/src/graphics'
    at onError (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/Resolver.js:61:15)
    at loggingCallbackWrapper (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)
    at runAfter (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/Resolver.js:158:4)
    at innerCallback (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/Resolver.js:146:3)
    at loggingCallbackWrapper (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)
    at next (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/tapable/lib/Tapable.js:252:11)
    at innerCallback (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/Resolver.js:144:11)
    at loggingCallbackWrapper (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)
    at next (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/tapable/lib/Tapable.js:249:35)
    at resolver.doResolve.createInnerCallback (/Users/mfikes/Desktop/CLJS-2792-npm-deps/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)

Error: goog.require could not find: craftyjs
	 (goog/base.js:711:20)
	 require (clojure/browser/repl.cljs:226:33)
cljs.user=>

If you look at the CraftyJS code, it has a require('fs') call (which Browserify evidently helps sidestep.)

But, you can easy use CraftyJS as a foreign lib. To do so, set your compiler options instead to be:

compiler-opts.edn
{:foreign-libs [{:file "https://github.com/craftyjs/Crafty/releases/download/0.8.0/crafty.js"
                 :provides ["craftyjs"]}]}

Then, you can drive CraftyJS right from the REPL:

$ clj -m cljs.main -co compiler-opts.edn -r
ClojureScript 1.10.339
cljs.user=> (set! (.-innerHTML (.getElementById js/document "app")) "")
""
cljs.user=> (require 'craftyjs)

cljs.user=> (.init js/Crafty)
#object[Crafty]
cljs.user=> (def player (-> js/Crafty
(.e "2D, Canvas, Color, Fourway")
(.attr #js {:x 100 :y 100 :w 50 :h 50})
(.color "blue")
(.fourway 3)))
#'cljs.user/player
cljs.user=>

After the above, you can control the box using your arrow keys.

Comment by Marty Glaubitz [ 01/Jul/18 9:58 AM ]

Thanks for the tip! But are you sure that

:file "https://github.com/craftyjs/Crafty/releases/download/0.8.0/crafty.js"

is supposed to work? On my windows machine i can only pass a local file path there...

Comment by Mike Fikes [ 01/Jul/18 1:55 PM ]

Hi Marty. Yes :file can be a URL. See https://clojurescript.org/reference/compiler-options#foreign-libs

Comment by Timothy Pratley [ 22/Jan/19 4:45 PM ]

Is this something that could be supported in the future?

Doing `npm install craftyjs --save` creates node_modules/craftyjs/src/crafty.js which is suitable for being used as the foreign-lib (its the same as the file at the end of the url). So it seems in principle that it would be convenient to be able to say "Get me <x> from node, and treat dist/y as a foreign-lib.

Which makes me wonder what :npm-deps currently does... presumably it is using the source of the package rather than the dist. That makes sense to me for things that don't use browserify (or intermediary build tools). But there are useful packages that do use browserify and these leave us to relying on a URL to a file or some manual steps to get the package and build it first. Would it be possible to specify a dependency that should be fetched, built and use a dist file instead of src? The benefit would be that we could use both styles of npm dependencies via the same mechanism.

It might need to have a different name like :npm-libs because I think it would need more than just a version... ie something like :npm-libs {"asciidoctor.js" {:version "1.5.9", :lib "dist/browser/asciidoctor.js"}}

I arrived at this thread trying to use asciidoctor.js which uses Browserify to build node_modules/asciidoctor.js/dist/browser/asciidoctor.js. The asciidoctor.js package is interesting because it targets both NodeJS and the browser, producing two output files [which also interestingly still require other stuff, but the browser doesn't require things like `fs`].





[CLJS-2757] Failure to load is-plain-object transitive npm dep Created: 21/May/18  Updated: 09/Nov/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mike Fikes Assignee: Mike Fikes
Resolution: Unresolved Votes: 5
Labels: npm-deps
Environment:

{:deps {org.clojure/clojurescript {:mvn/version "1.10.238"}}}



 Description   

The slate npm dep depends on is-plain-object but it can't be loaded. Start up a REPL with:

clojure -m cljs.main -co '{:npm-deps {"react" "15.5.4" "react-dom" "15.5.4" "slate" "0.33.6" "immutable" "3.8.2"} :install-deps true :repl-verbose true}' -d out -r

(Note, that in the above we are going with the default browser REPL; the problem doesn't occur if you specify -re node.)

Then, after the REPL is up and running:

cljs.user=> (require 'slate)
load-namespace module$private$tmp$test-me$node-modules$slate$lib$slate-es , compiled: ((module$private$tmp$test-me$node-modules$slate$lib$slate-es))
goog.addDependency("../node_modules/slate/lib/slate.es.js", ['module$private$tmp$test_me$node_modules$slate$lib$slate_es'], ['module$is_plain_object', 'module$immutable', 'module$slate_dev_logger', 'module$direction', 'module$esrever', 'module$debug', 'module$lodash$isEqual', 'module$lodash$mergeWith', 'module$slate_schema_violations', 'module$lodash$pick', 'module$lodash$omit', 'module$is_empty']);

goog.require('module$private$tmp$test_me$node_modules$slate$lib$slate_es');
null;

"Error evaluating:" (require (quote slate)) :as "goog.require('module$private$tmp$test_me$node_modules$slate$lib$slate_es');\nnull;\n"
Error: Undefined nameToPath for module$is_plain_object
	 (out/goog/base.js:1357:26)
	 (out/goog/base.js:1369:14)
	 (out/goog/base.js:706:27)
	 require (out/clojure/browser/repl.cljs:340:25)

Note that it can't load module$is_plain_object, which is specified via

goog.require("module$is_plain_object");

inside of out/node_modules/slate/lib/slate.es.js.

But, you can load that namespace directly:

cljs.user=> (require 'is-plain-object)
load-namespace module$private$tmp$test-me$node-modules$is-plain-object$index , compiled: ([module$private$tmp$test-me$node-modules$isobject$index] (module$private$tmp$test-me$node-modules$is-plain-object$index))
goog.addDependency("../node_modules/isobject/index.js", ['module$private$tmp$test_me$node_modules$isobject$index'], []);
goog.addDependency("../node_modules/is-plain-object/index.js", ['module$private$tmp$test_me$node_modules$is_plain_object$index'], ['module$private$tmp$test_me$node_modules$isobject$index']);

goog.require('module$private$tmp$test_me$node_modules$is_plain_object$index');
null;

This appears to be the result of Closure failing to find the correct paths to the modules and simply emitting module$<symbol> instead of module$<path> for the goog.require.



 Comments   
Comment by Dieter Komendera [ 18/Jun/18 4:06 AM ]

Current master produces a different problem.

These errors are now logged:

Can't find variable: module
undefined is not an object (evaluating 'module$Users$kommen$work$clojurescript$node_modules$slate$node_modules$debug$src$debug["default"]')
TypeError: undefined is not a constructor (evaluating 'new module$Users$kommen$work$clojurescript$node_modules$immutable$dist$immutable.Map')

Juho suggested the original issue actually might have been a duplicate of CLJS-2746.

Comment by Mike Fikes [ 18/Jun/18 6:24 AM ]

Yes, CLJS-2746 is definitely in the area. Perhaps the issue described in this ticket is now resolved and we are onto completely different issues.

Comment by Mike Fikes [ 04/Jul/18 7:10 AM ]

Using current master of Closure Compiler, building the unshaded JAR via mvn -DskipTests -pl externs/pom.xml,pom-main.xml,pom-main-unshaded.xml and depending on it using

deps.edn
{:deps {org.clojure/clojurescript {:mvn/version "1.10.339"}
        com.google.javascript/closure-compiler-unshaded {:mvn/version "1.0-SNAPSHOT"}}}

then you can (require 'slate) without errors being reported. (It is not clear to me whether Closure is loading all of the code correctly; after this it is not clear to me how you would use Slate and the exported object appears to be empty.)

Comment by Garrett Hopper [ 06/Jul/18 10:40 AM ]

This may be an unrelated issue, but there seems to be a problem if the npm module uses `process.env.NODE_ENV`.

Starting REPL with snapshot `closure-compiler-unshaded`
clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.339"} com.google.javascript/closure-compiler-unshaded {:mvn/version "1.0-SNAPSHOT"}}}' -m cljs.main -co '{:npm-deps {"emotion" "9.2.4"} :install-deps true :repl-verbose true}' -d out -r
ClojureScript 1.10.339
cljs.user=> (require '[emotion])
load-namespace module$home$garrett$tmp$node-modules$emotion$dist$index-es , compiled: ([module$home$garrett$tmp$node-modules$$emotion$unitless$dist$index-es module$$emotion$unitless module$$emotion$unitless$dist$index-es] [module$home$garrett$tmp$node-modules$$emotion$hash$dist$index-es module$$emotion$hash module$$emotion$hash$dist$index-es] [module$home$garrett$tmp$node-modules$$emotion$stylis$dist$index-es module$$emotion$stylis module$$emotion$stylis$dist$index-es] [module$home$garrett$tmp$node-modules$stylis-rule-sheet$index module$stylis-rule-sheet module$stylis-rule-sheet$index] [module$home$garrett$tmp$node-modules$create-emotion$dist$index-es module$create-emotion module$create-emotion$dist$index-es] [module$home$garrett$tmp$node-modules$$emotion$memoize$dist$index-es module$$emotion$memoize module$$emotion$memoize$dist$index-es] (module$home$garrett$tmp$node-modules$emotion$dist$index-es module$emotion module$emotion$dist$index-es))
goog.addDependency("../node_modules/@emotion/unitless/dist/index.es.js", ['module$home$garrett$tmp$node_modules$$emotion$unitless$dist$index_es', 'module$$emotion$unitless', 'module$$emotion$unitless$dist$index_es'], []);
goog.addDependency("../node_modules/@emotion/hash/dist/index.es.js", ['module$home$garrett$tmp$node_modules$$emotion$hash$dist$index_es', 'module$$emotion$hash', 'module$$emotion$hash$dist$index_es'], []);
goog.addDependency("../node_modules/@emotion/stylis/dist/index.es.js", ['module$home$garrett$tmp$node_modules$$emotion$stylis$dist$index_es', 'module$$emotion$stylis', 'module$$emotion$stylis$dist$index_es'], []);
goog.addDependency("../node_modules/stylis-rule-sheet/index.js", ['module$home$garrett$tmp$node_modules$stylis_rule_sheet$index', 'module$stylis_rule_sheet', 'module$stylis_rule_sheet$index'], []);
goog.addDependency("../node_modules/create-emotion/dist/index.es.js", ['module$home$garrett$tmp$node_modules$create_emotion$dist$index_es', 'module$create_emotion', 'module$create_emotion$dist$index_es'], ['module$$emotion$memoize', 'module$$emotion$unitless', 'module$$emotion$hash', 'module$$emotion$stylis', 'module$stylis_rule_sheet']);
goog.addDependency("../node_modules/@emotion/memoize/dist/index.es.js", ['module$home$garrett$tmp$node_modules$$emotion$memoize$dist$index_es', 'module$$emotion$memoize', 'module$$emotion$memoize$dist$index_es'], []);
goog.addDependency("../node_modules/emotion/dist/index.es.js", ['module$home$garrett$tmp$node_modules$emotion$dist$index_es', 'module$emotion', 'module$emotion$dist$index_es'], ['module$create_emotion']);

goog.require('module$home$garrett$tmp$node_modules$emotion$dist$index_es');
null;
Browser Console Errors
index.es.js:11 Uncaught ReferenceError: process is not defined
    at index.es.js:11
(anonymous) @ index.es.js:11
index.es.js:5 Uncaught TypeError: module$home$garrett$tmp$node_modules$create_emotion$dist$index_es.default is not a function
    at index.es.js:5
Comment by Garrett Hopper [ 06/Jul/18 10:45 AM ]

In regards to `slate`, there are also browser console errors.

REPL
clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.339"} com.google.javascript/closure-compiler-unshaded {:mvn/version "1.0-SNAPSHOT"}}}' -m cljs.main -co '{:npm-deps {"react" "15.5.4" "react-dom" "15.5.4" "slate" "0.33.6" "immutable" "3.8.2"} :install-deps true}' -d out -r
ClojureScript 1.10.339
cljs.user=> (require '[slate])

cljs.user=> slate
#js {}
cljs.user=>
Browser Console Errors
browser.js:15 Uncaught TypeError: (0 , module$home$garrett$tmp$node_modules$slate$node_modules$debug$src$browser.default.load) is not a function
    at browser.js:15
(anonymous) @ browser.js:15
isBuffer.js:6 Uncaught ReferenceError: module is not defined
    at isBuffer.js:6
(anonymous) @ isBuffer.js:6
_nodeUtil.js:5 Uncaught ReferenceError: module is not defined
    at _nodeUtil.js:5
(anonymous) @ _nodeUtil.js:5
_cloneBuffer.js:6 Uncaught ReferenceError: module is not defined
    at _cloneBuffer.js:6
(anonymous) @ _cloneBuffer.js:6
slate.es.js:37 Uncaught TypeError: module$home$garrett$tmp$node_modules$immutable$dist$immutable.Map is not a constructor
    at slate.es.js:37
Comment by Mike Fikes [ 06/Jul/18 11:52 AM ]

Yeah, one issue (perhaps a different one, strictly speaking, given that this ticket as written will be solved when we ultimately move to a newer version of Closure), is that there seems to be some mixup between the debug module that is inside the slate module tree, and the top-level debug module.

Comment by Mike Fikes [ 08/Nov/18 5:26 PM ]

It appears that the Map constructor issue is that the hosting object involves a default field:

cljs.user=> (let [Map (.-Map (.-default js/module$private$tmp$cljs_2757$node_modules$immutable$dist$immutable))] (Map.))
#object[Map$$module$private$tmp$cljs_2757$node_modules$immutable$dist$immutable Map {}]




[CLJS-2756] Relative path issue processing node_modules file Created: 21/May/18  Updated: 21/May/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mike Fikes Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: npm-deps
Environment:

{:deps {org.clojure/clojurescript {:mvn/version "1.10.238"}}}



 Description   

The following

clojure -m cljs.main -co '{:npm-deps {"slate" "0.33.6"} :install-deps true}' -e "(require 'slate)"

generates the following exception

Exception in thread "main" java.lang.IllegalArgumentException: /Users/mfikes/node_modules/immutable/dist/immutable.js is not a relative path
	at clojure.java.io$as_relative_path.invokeStatic(io.clj:414)
	at clojure.java.io$file.invokeStatic(io.clj:426)
	at clojure.java.io$file.invoke(io.clj:418)
	at cljs.closure$write_javascript.invokeStatic(closure.clj:1857)
	at cljs.closure$write_javascript.invoke(closure.clj:1850)
	at cljs.closure$process_js_modules$fn__5815.invoke(closure.clj:2560)
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:63)
	at clojure.core.protocols$fn__7847.invokeStatic(protocols.clj:136)
	at clojure.core.protocols$fn__7847.invoke(protocols.clj:124)
	at clojure.core.protocols$fn__7807$G__7802__7816.invoke(protocols.clj:19)
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
	at clojure.core.protocols$fn__7835.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__7835.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__7781$G__7776__7794.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6748)
	at clojure.core$reduce.invoke(core.clj:6730)
	at cljs.closure$process_js_modules.invokeStatic(closure.clj:2559)
	at cljs.closure$process_js_modules.invoke(closure.clj:2518)
	at cljs.closure$handle_js_modules.invokeStatic(closure.clj:2672)
	at cljs.closure$handle_js_modules.invoke(closure.clj:2633)
	at cljs.repl$evaluate_form.invokeStatic(repl.cljc:508)
	at cljs.repl$evaluate_form.invoke(repl.cljc:484)
	at cljs.repl$eval_cljs.invokeStatic(repl.cljc:672)
	at cljs.repl$eval_cljs.invoke(repl.cljc:665)
	at cljs.repl$run_inits$fn__6480.invoke(repl.cljc:823)
	at cljs.repl$run_inits.invokeStatic(repl.cljc:821)
	at cljs.repl$run_inits.invoke(repl.cljc:814)
	at cljs.cli$default_main$fn__6799.invoke(cli.clj:340)
	at cljs.compiler$with_core_cljs.invokeStatic(compiler.cljc:1285)
	at cljs.compiler$with_core_cljs.invoke(compiler.cljc:1274)
	at cljs.compiler.api$with_core_cljs.invokeStatic(api.clj:50)
	at cljs.compiler.api$with_core_cljs.invoke(api.clj:34)
	at cljs.compiler.api$with_core_cljs.invokeStatic(api.clj:42)
	at cljs.compiler.api$with_core_cljs.invoke(api.clj:34)
	at cljs.cli$default_main.invokeStatic(cli.clj:326)
	at cljs.cli$default_main.invoke(cli.clj:299)
	at cljs.cli$null_opt.invokeStatic(cli.clj:394)
	at cljs.cli$null_opt.invoke(cli.clj:391)
	at cljs.cli$main.invokeStatic(cli.clj:612)
	at cljs.cli$main.doInvoke(cli.clj:601)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:659)
	at clojure.core$apply.invoke(core.clj:652)
	at cljs.main$_main.invokeStatic(main.clj:61)
	at cljs.main$_main.doInvoke(main.clj:52)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.core$apply.invokeStatic(core.clj:657)
	at clojure.main$main_opt.invokeStatic(main.clj:317)
	at clojure.main$main_opt.invoke(main.clj:313)
	at clojure.main$main.invokeStatic(main.clj:424)
	at clojure.main$main.doInvoke(main.clj:387)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.main.main(main.java:37)


 Comments   
Comment by Mike Fikes [ 21/May/18 2:30 PM ]

This failure was evidently related to the fact that I had a node_modules directory sitting in my home directory. Removing it caused the issue above to go away.

Perhaps this is the result of the processing marching outward (I was in a subdirectory of my home directory). If this is a possibility, maybe the ClojureScript compiler can itself handle this corner case? (Or is this just a user error?)





[CLJS-2746] Missing/Incorrect provides for node module required from ES6 module Created: 01/May/18  Updated: 16/Jun/18  Resolved: 15/Jun/18

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Corin Lawson Assignee: David Nolen
Resolution: Completed Votes: 1
Labels: foreign-libs, modules, npm-deps
Environment:

Linux 4.16.2 #23 SMP Mon Apr 30 11:01:40 AEST 2018 x86_64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz GenuineIntel GNU/Linux


Attachments: Text File CLJS-2746.patch    
Patch: Code

 Description   

Affects: 1.10.271 with CLJS-2402.patch

I have a minimal repro repo here: https://github.com/au-phiware/cljs-in-js-in-cljs

Description of problem

The compiler appears to know node modules that are required from foreign libs by two different names, but the cljs_deps.js file only uses one of them. In the repro case below, unique-random is referenced as both module$unique_random and module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index in the compiled output of the foreign lib (}target/main.out/es6/hello.js}), but {{cljs_deps.js only specifies module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index as its provides.

Steps to reproduce the problem

Consider the following source files:

src/hello/core.cljs
(ns hello.core (:require [unique-random]
                         [hello :refer [greet]]))

(greet "test")
es6/hello.js
import random from "unique-random";
export var greet = function(m) {
    document.write("Hello, " + m + random(1, 100)());
};
build.clj
(require 'cljs.build.api)

(cljs.build.api/build
  "src"
  {:main 'hello.core
   :output-to "target/main.js"
   :output-dir "target/main.out"
   :asset-path "main.out"
   :foreign-libs [{:file "es6"
                   :module-type :es6}]
   :install-deps true
   :npm-deps {:unique-random "latest"}})

Execute cljs:

java -cp cljs.jar:src clojure.main build.clj

Expected outcome

cljs should produce nothing to the standard output, exit cleanly and write the following files (approximately).

target/main.js
var CLOSURE_UNCOMPILED_DEFINES = {};
var CLOSURE_NO_DEPS = true;
if(typeof goog == "undefined") document.write('<script src="main.out/goog/base.js"></script>');
document.write('<script src="main.out/goog/deps.js"></script>');
document.write('<script src="main.out/cljs_deps.js"></script>');
document.write('<script>if (typeof goog == "undefined") console.warn("ClojureScript could not load :main, did you forget to specify :asset-path?");</script>');
document.write('<script>goog.require("process.env");</script>');
document.write('<script>goog.require("hello.core");</script>');
target/main.out/cljs_deps.js
goog.addDependency("base.js", ['goog'], []);
goog.addDependency("../cljs/core.js", ['cljs.core'], ['goog.string', 'goog.Uri', 'goog.object', 'goog.math.Integer', 'goog.string.StringBuffer', 'goog.array', 'goog.math.Long']);
goog.addDependency("../process/env.js", ['process.env'], ['cljs.core']);
goog.addDependency("../node_modules/unique-random/index.js", ['module$unique_random', 'module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index'], []);
goog.addDependency("../es6/hello.js", ['module$usr$src$cljs_in_js_in_cljs$es6$hello'], ['module$unique_random']);
goog.addDependency("../hello/core.js", ['hello.core'], ['cljs.core', 'module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index', 'module$usr$src$cljs_in_js_in_cljs$es6$hello']);
target/main.out/es6/hello.js
goog.provide("module$usr$src$cljs_in_js_in_cljs$es6$hello");
goog.require("module$unique_random");
var greet$$module$usr$src$cljs_in_js_in_cljs$es6$hello=function(m){document.write("Hello, "+m+module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index["default"](1,100)())};var module$usr$src$cljs_in_js_in_cljs$es6$hello={};module$usr$src$cljs_in_js_in_cljs$es6$hello.greet=greet$$module$usr$src$cljs_in_js_in_cljs$es6$hello
target/main.out/hello/core.js
// Compiled by ClojureScript 1.10.272 {}
goog.provide('hello.core');
goog.require('cljs.core');
goog.require('module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index');
goog.require('module$usr$src$cljs_in_js_in_cljs$es6$hello');
module$usr$src$cljs_in_js_in_cljs$es6$hello.greet("test");

Caveat

The expected outcome was derived from the actual outcome with the minimal set of changes required to produce the desired outcome (i.e. working in the browser without error).

Alternatively, all references to module$unique_random could be replaced with module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index, for the same desired outcome.

Actual outcome

The cljs_deps.js file is missing the module$unique_random provides name for the node module.

target/main.out/cljs_deps.js
goog.addDependency("base.js", ['goog'], []);
goog.addDependency("../cljs/core.js", ['cljs.core'], ['goog.string', 'goog.Uri', 'goog.object', 'goog.math.Integer', 'goog.string.StringBuffer', 'goog.array', 'goog.math.Long']);
goog.addDependency("../process/env.js", ['process.env'], ['cljs.core']);
goog.addDependency("../node_modules/unique-random/index.js", ['module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index'], []);
goog.addDependency("../es6/hello.js", ['module$usr$src$cljs_in_js_in_cljs$es6$hello'], ['module$unique_random']);
goog.addDependency("../hello/core.js", ['hello.core'], ['cljs.core', 'module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index', 'module$usr$src$cljs_in_js_in_cljs$es6$hello']);

The following is the diff output for clarity.

diff --git a/target/main.out/cljs_deps.js b/target/main.out/cljs_deps.js
index dd4df1f..de77e67 100644
--- a/target/main.out/cljs_deps.js
+++ b/target/main.out/cljs_deps.js
@@ -1,6 +1,6 @@
 goog.addDependency("base.js", ['goog'], []);
 goog.addDependency("../cljs/core.js", ['cljs.core'], ['goog.string', 'goog.Uri', 'goog.object', 'goog.math.Integer', 'goog.string.StringBuffer', 'goog.array', 'goog.math.Long']);
 goog.addDependency("../process/env.js", ['process.env'], ['cljs.core']);
-goog.addDependency("../node_modules/unique-random/index.js", ['module$unique_random', 'module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index'], []);
+goog.addDependency("../node_modules/unique-random/index.js", ['module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index'], []);
 goog.addDependency("../es6/hello.js", ['module$usr$src$cljs_in_js_in_cljs$es6$hello'], ['module$unique_random']);
 goog.addDependency("../hello/core.js", ['hello.core'], ['cljs.core', 'module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index', 'module$usr$src$cljs_in_js_in_cljs$es6$hello'])

Furthermore, if optimisations is turned on (e.g. :optimizations :simple) the following error is observed from cljs.

stderr
May 01, 2018 5:50:49 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /usr/src/cljs-in-js-in-cljs/target/main.out/es6/hello.js:2: ERROR - required "module$unique_random" namespace never provided
goog.require("module$unique_random");
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

May 01, 2018 5:50:49 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_MISSING_PROVIDE_ERROR. required "module$unique_random" namespace never provided at /usr/src/cljs-in-js-in-cljs/target/main.out/es6/hello.js line 2 : 0
Exception in thread "main" java.lang.Exception: Closure compilation failed, compiling:(/usr/src/cljs-in-js-in-cljs/build.clj:3:1)
	at clojure.lang.Compiler.load(Compiler.java:7526)
	at clojure.lang.Compiler.loadFile(Compiler.java:7452)
	at clojure.main$load_script.invokeStatic(main.clj:278)
	at clojure.main$script_opt.invokeStatic(main.clj:338)
	at clojure.main$script_opt.invoke(main.clj:333)
	at clojure.main$main.invokeStatic(main.clj:424)
	at clojure.main$main.doInvoke(main.clj:387)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:702)
	at clojure.main.main(main.java:37)
Caused by: java.lang.Exception: Closure compilation failed
	at cljs.closure$report_failure.invokeStatic(closure.clj:389)
	at cljs.closure$optimize.invokeStatic(closure.clj:1422)
	at cljs.closure$optimize.doInvoke(closure.clj:1382)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:659)
	at cljs.closure$build.invokeStatic(closure.clj:2866)
	at cljs.build.api$build.invokeStatic(api.clj:207)
	at cljs.build.api$build.invoke(api.clj:189)
	at cljs.build.api$build.invokeStatic(api.clj:195)
	at cljs.build.api$build.invoke(api.clj:189)
	at user$eval34.invokeStatic(build.clj:3)
	at user$eval34.invoke(build.clj:3)
	at clojure.lang.Compiler.eval(Compiler.java:7062)
	at clojure.lang.Compiler.load(Compiler.java:7514)
	... 9 more


 Comments   
Comment by Corin Lawson [ 03/May/18 8:59 AM ]

I found some time to perform a git bisect and the module$unique_random name appears at v1.9.1033 (72e2ab6e) "CLJS-2389: Update Closure-compiler".

Comment by Corin Lawson [ 09/May/18 12:18 PM ]

I have found that ModuleNames/fileToModuleName produces module$usr$src$cljs_in_js_in_cljs$node_modules$unique_random$index for node_modules/unique-random/index.js and .getRequires produces {{#object[com.google.common.collect.RegularImmutableList [module$unique_random]]}} for es6/hello.js.

I found the location where the goog.provide statements are being written. Ensuring that a goog.provide statement is written for each name that the module provide resolves this problem, see attached patch.

Comment by David Nolen [ 15/Jun/18 4:18 PM ]

fixed https://github.com/clojure/clojurescript/commit/6950aff480174a25a77c2308aa87a099a774a6e8

Comment by Juho Teperi [ 16/Jun/18 2:37 PM ]

Good catch. I didn't remember that we add :provides lists to npm foreign-libs which is why I didn't think about this when I wrote the problematic code.





[CLJS-2739] Optimize node_modules indexing Created: 17/Apr/18  Updated: 07/May/18  Resolved: 07/May/18

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.10.238
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Fikes Assignee: David Nolen
Resolution: Completed Votes: 5
Labels: npm-deps

Attachments: Text File CLJS-2739.patch    
Patch: Code

 Description   

Indexing node_modules can take quite a while. As an example, with the node_modules directory that is set up for React Native projects, on a 2012 Mac Pro, this can take about 45 seconds. This is long enough where, if node_modules isn't actually being used for ClojureScript, using :npm-deps false is highly motivated.

If there was a way to make this closer to instantaneous (perhaps via caching or other clever approaches), then this would benefit all users.



 Comments   
Comment by Juho Teperi [ 17/Apr/18 10:38 AM ]

Based on Mike's test this is mostly caused by index-node-modules-dir, which reads the node modules dir tree using JVM: https://gist.github.com/mfikes/c99cfac7b9f5ae48fc9644bbde492a3c

Comment by Mike Fikes [ 06/May/18 5:06 PM ]

For one of my projects the attached patch drops the time spent in index-node-modules-dir from 30 seconds to 5 seconds.

Comment by David Nolen [ 07/May/18 3:29 PM ]

fixed https://github.com/clojure/clojurescript/commit/e577d46c1e9fc5ecea8e727f6f6c31cc8e5a6652





[CLJS-2738] Can't use aws-amplify or aws-sdk with npm-deps Created: 15/Apr/18  Updated: 09/Aug/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Quang Van Assignee: Unassigned
Resolution: Unresolved Votes: 5
Labels: npm-deps


 Description   

If I put aws-amplify in :npm-deps it seems to require aws-sdk, but when I put aws-sdk in there, it outputs this error:

Compiling build :dev to "resources/public/js/main.js" from ["src"]...
events.js:182
      throw er; // Unhandled 'error' event
      ^

Error: module not found: "fs" from file /home/quang/dev/org-re-frame/node_modules/aws-amplify/node_modules/aws-sdk/lib/util.js
    at onresolve (/home/quang/dev/org-re-frame/node_modules/@cljs-oss/module-deps/index.js:181:30)
    at onResolve (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/Resolver.js:70:11)
    at innerCallback (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/Resolver.js:143:22)
    at callbackWrapper (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)
    at next (/home/quang/dev/org-re-frame/node_modules/tapable/lib/Tapable.js:249:35)
    at innerCallback (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/Resolver.js:143:22)
    at callbackWrapper (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)
    at next (/home/quang/dev/org-re-frame/node_modules/tapable/lib/Tapable.js:249:35)
    at resolver.doResolve.createInnerCallback (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:41:24)
    at callbackWrapper (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)


 Comments   
Comment by Jeffrey Stanton [ 26/Apr/18 2:51 PM ]

I've also run into the same issue. Here's a simple repro:

deps.edn
{:deps {org.clojure/clojurescript {:mvn/version "1.10.238"}}}
src/foo.cljs
(ns foo (:require ["aws-sdk" :as sdk]))
(defn -main [] (println "hello world"))

then run:

$ clj -m cljs.main -co '{:install-deps true, :npm-deps {:aws-sdk "2.229.1"}}' -m foo
...
Error: module not found: "fs" from file /Users/jeff/repro/node_modules/aws-sdk/lib/util.js
    at onresolve (/Users/jeff/repro/node_modules/@cljs-oss/module-deps/index.js:181:30)
    at onResolve (/Users/jeff/repro/node_modules/enhanced-resolve/lib/Resolver.js:70:11)
    at innerCallback (/Users/jeff/repro/node_modules/enhanced-resolve/lib/Resolver.js:143:22)
    at callbackWrapper (/Users/jeff/repro/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)
    ...
Comment by Mike Fikes [ 05/May/18 7:35 PM ]

The big picture is not completely clear to me for this one, but it appears that the NPM version of the library is not intended to be used directly from the browser runtime, and instead you need to build it so that it is packaged for the browser.

The root problem is that the code is attempting to use Node's fs code, but from the browser.

You can see that the code can be compiled and executed if you target Node: If you modify Jeffrey's minimal repro to add

(set! *main-cli-fn* -main)

to the bottom of src/foo.cljs then things will compile and execute in Node:

$ clj -m cljs.main -co '{:install-deps true, :npm-deps {:aws-sdk "2.229.1"}}' -O simple -o main.js -t node -c foo
WARNING: foo is a single segment namespace at line 1 /private/tmp/aws/src/foo.cljs
$ node main.js
hello world

For the browser case, if you look at the code in node_modules/aws-sdk/lib/util.js you can see that it has conditional branches for browser use, but I suspect that this is intended to be used as described here https://github.com/aws/aws-sdk-js#in-the-browser but if you really want to use the code directly from its NPM dependency, you have to use that NPM dependency to build the JavaScript intended to be used in the browser as detailed here: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/building-sdk-for-browsers.html

If this can be confirmed, then this issue can be resolved as a non-issue, and it is just a matter of correct use of the library.

Comment by David Whittington [ 05/May/18 8:23 PM ]

I can confirm that it works in the browser with the AWS recommended browser build process. I was hoping it would also work as a node dep, but I think you're probably right that it's just not intended to be used that way.

Comment by Christian Paul Dehli [ 09/Aug/18 9:55 PM ]

So adding `aws-sdk` the recommended way does work, however there are many AWS packages that pull in `aws-sdk` as a dependency. One example is https://github.com/awslabs/aws-mobile-appsync-sdk-js/tree/master/packages/aws-appsync. This means that these packages won't run in ClojureScript because they throw the same error as above. I have confirmed that in a regular JavaScript project, `aws-sdk` is added correctly (an example is here: https://github.com/aws-samples/aws-mobile-appsync-events-starter-react/).

Like Mike said, it looks to be coming from this line in the code: https://github.com/aws/aws-sdk-js/blob/ebe83921863f1eb020b6a07ef471f2017cd58550/lib/util.js#L39. process.browser is what's indicating to aws-sdk whether or not it's a browser environment.

My guess would be if we set `process.browser` to true for when building to non-node environments it would solve this problem (https://nolanlawson.com/2017/01/09/how-to-write-a-javascript-package-for-both-node-and-the-browser/)





[CLJS-2677] Better handling of npm-deps via REPL require when output-dir not specified Created: 20/Mar/18  Updated: 25/Apr/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Fikes Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: npm-deps
Environment:

{:deps {org.clojure/clojurescript {:mvn/version "1.10.217"}}}



 Description   

I'm speculating that the following doesn't work because Closure is not involved when requiring at the REPL.

co.edn
{:output-dir "out"
 :output-to "out/main.js"
 :optimizations :none
 :install-deps true
 :npm-deps {:react "15.6.1"
            :react-dom "15.6.1"}}
src/example/core.cljs
(ns example.core
  (:require [react :refer [createElement]]
            ["react-dom/server" :as ReactDOMServer :refer [renderToString]]))

(js/console.log (renderToString (createElement "div" nil "Hello World!")))

This works fine:

clj -m cljs.main -co co.edn -c example.core -r

logging this in the browser console

[Log] <div data-reactroot="" data-reactid="1" data-react-checksum="1309021079">Hello World!</div> (core.js, line 6)

But if instead you start off fresh (without out, node_modules, package.json and package-lock.json) and do

clj -m cljs.main -co co.edn -r

followed by this in the REPL:

(require 'example.core)

No errors are shown in the REPL console, but the JavaScript console shows lots of errors.

I think to satisfy this ticket this could either be made to "just work" somehow, or perhaps instead the problem could be detected and an ex-info thrown letting the user know that things won't work this way.

Here are the errors logged:

[Error] ReferenceError: Can't find variable: process
	Global Code (warning.js:4)
[Error] ReferenceError: Can't find variable: process
	Global Code (canDefineProperty.js:2:170)
[Error] ReferenceError: Can't find variable: process
	Global Code (emptyObject.js:2:153)
[Error] ReferenceError: Can't find variable: process
	Global Code (invariant.js:4:192)
[Error] ReferenceError: Can't find variable: process
	Global Code (lowPriorityWarning.js:3)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactBaseClasses.js:16)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactPropTypeLocationNames.js:2:185)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactDOMFactories.js:5)
[Error] ReferenceError: Can't find variable: process
	Global Code (checkPropTypes.js:10)
[Error] ReferenceError: Can't find variable: process
	createChainableTypeChecker (factoryWithTypeCheckers.js:12:305)
	createPrimitiveTypeChecker (factoryWithTypeCheckers.js:17:102)
	(anonymous function) (factoryWithTypeCheckers.js:10:535)
	(anonymous function) (factory.js:3:441)
	Global Code (ReactPropTypes.js:5:149)
[Error] ReferenceError: Can't find variable: process
	Global Code (factory.js:34:317)
[Error] ReferenceError: Can't find variable: process
	Global Code (React.js:18)
[Error] ReferenceError: Can't find variable: process
	Global Code (EventPluginRegistry.js:16:218)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactErrorUtils.js:5)
[Error] ReferenceError: Can't find variable: process
	Global Code (EventPluginUtils.js:12)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$EventPluginRegistry["default"].injectEventPluginOrder')
	Global Code (EventPluginHub.js:16:199)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$EventPluginHub["default"].getListener')
	Global Code (EventPropagators.js:9:167)
[Error] ReferenceError: Can't find variable: process
	Global Code (SyntheticEvent.js:18)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticEvent["default"].augmentClass...')
	Global Code (SyntheticCompositionEvent.js:4:393)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticEvent["default"].augmentClass...')
	Global Code (SyntheticInputEvent.js:4:203)
[Error] ReferenceError: Can't find variable: process
	Global Code (DOMProperty.js:14:471)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$DOMProperty["default"].ID_ATTRIBUTE_NAME')
	Global Code (ReactDOMComponentTree.js:7:516)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactInvalidSetStateWarningHook.js:4)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactInstrumentation.js:4)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticEvent["default"].augmentClass...')
	Global Code (SyntheticUIEvent.js:7:93)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticUIEvent["default"].augmentClass...')
	Global Code (SyntheticMouseEvent.js:9:95)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$DOMProperty["default"].injection')
	Global Code (HTMLDOMPropertyConfig.js:3:425)
[Error] ReferenceError: Can't find variable: process
	Global Code (DOMChildrenOperations.js:21)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$DOMChildrenOperations["default"].dangerouslyReplaceNodeWithMarkup')
	Global Code (ReactComponentBrowserEnvironment.js:5:349)
[Error] ReferenceError: Can't find variable: process
	Global Code (CSSPropertyOperations.js:17)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$DOMProperty["default"].ATTRIBUTE_NAME_START_CHAR')
	Global Code (DOMPropertyOperations.js:9:200)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react$lib$React["default"].isValidElement')
	Global Code (LinkedValueUtils.js:9:533)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactPropTypeLocationNames.js:2:193)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactCompositeComponent.js:21:366)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactMultiChild.js:21)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactServerRenderingTransaction.js:10:126)
[Error] ReferenceError: Can't find variable: process
	Global Code (validateDOMNesting.js:7)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$EventPluginHub["default"].deleteListener')
	Global Code (ReactDOMComponent.js:42:171)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$DOMProperty["default"].injection')
	Global Code (ReactInjection.js:13:279)
[Error] ReferenceError: Can't find variable: process
	Global Code (ReactReconcileTransaction.js:18)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticEvent["default"].augmentClass...')
	Global Code (SyntheticAnimationEvent.js:4:256)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticEvent["default"].augmentClass...')
	Global Code (SyntheticClipboardEvent.js:4:303)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticUIEvent["default"].augmentClass...')
	Global Code (SyntheticFocusEvent.js:4:214)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticUIEvent["default"].augmentClass...')
	Global Code (SyntheticKeyboardEvent.js:10:95)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticMouseEvent["default"].augmentClass...')
	Global Code (SyntheticDragEvent.js:4:214)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticUIEvent["default"].augmentClass...')
	Global Code (SyntheticTouchEvent.js:6:95)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticEvent["default"].augmentClass...')
	Global Code (SyntheticTransitionEvent.js:4:257)
[Error] TypeError: undefined is not a function (near '...module$private$tmp$island$node_modules$react_dom$lib$SyntheticMouseEvent["default"].augmentClass...')
	Global Code (SyntheticWheelEvent.js:5:98)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$lib$ReactInjection["default"].EventEmitter')
	inject (ReactDefaultInjection.js:22:383)
	Global Code (ReactDOMServer.js:6:257)
[Error] TypeError: undefined is not an object (evaluating 'module$private$tmp$island$node_modules$react_dom$server["default"].renderToString')
	Global Code (core.js:6)


 Comments   
Comment by David Nolen [ 20/Mar/18 9:59 AM ]

You must set :output-dir if you intend to use :npm-deps for now. If we somehow figure out a way to make this work, it also needs to work with Node.js if you do not supply :output-dir.





[CLJS-2622] Add support for package tarballs in :npm-deps Created: 06/Mar/18  Updated: 30/Apr/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: 1.10.238
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Jannis Pohlmann Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: npm-deps

Attachments: Text File CLJS-2622.patch    

 Description   

NPM supports installing packages from tarballs via {npm install /path/to/foo.tgz}. This can be useful for trying a local (modified) version of a package, or generally any unpublished and/or private JS project; all that's needed is a {package.json} and a {.js} file and {npm pack} can generate a tarball.

It would be nice if ClojureScript would support this as well, through {:npm-deps}.

Proposed syntax:

{:npm-deps {"/path/to/tarball.tgz" "0.1.0"
            "/path/to/bar.tgz" ""}}

The version isn't needed with tarballs, so it could be anything.

I have the code for this working but I still want to add a test before I submit a patch.



 Comments   
Comment by Jannis Pohlmann [ 07/Mar/18 7:18 AM ]

This patch implements the feature and adds tests for indexing and building with a tarball dependency.

Comment by Thomas Heller [ 30/Apr/18 2:53 PM ]

What benefit does this provide over just running npm install ./some.tgz?

I don't think this is a good idea.





[CLJS-2402] Duplicate source files passed to Closure Compiler causes ModuleLoader#resolvePaths to throw "Duplicate module path after resolving" Created: 19/Nov/17  Updated: 11/May/19

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.908
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Corin Lawson Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: modules, npm-deps

Attachments: Text File CLJS-2402.patch     File patch    
Patch: Code

 Description   

I have a small (not minimal) repo here: https://github.com/au-phiware/boot-parsets

The repo includes a es6 module and a cljs file that both depend on a node module, which produces the following error.

Writing main.cljs.edn...
Compiling ClojureScript...
• main.js
                                       java.lang.Thread.run                  Thread.java:  748
         java.util.concurrent.ThreadPoolExecutor$Worker.run      ThreadPoolExecutor.java:  617
          java.util.concurrent.ThreadPoolExecutor.runWorker      ThreadPoolExecutor.java: 1142
                        java.util.concurrent.FutureTask.run              FutureTask.java:  266
                                                        ...                                   
                        clojure.core/binding-conveyor-fn/fn                     core.clj: 2022
                              adzerk.boot-cljs/compile-1/fn                boot_cljs.clj:  160
                                   adzerk.boot-cljs/compile                boot_cljs.clj:   72
                                          boot.pod/call-in*                      pod.clj:  413
                                                        ...                                   
org.projectodd.shimdandy.impl.ClojureRuntimeShimImpl.invoke  ClojureRuntimeShimImpl.java:  102
org.projectodd.shimdandy.impl.ClojureRuntimeShimImpl.invoke  ClojureRuntimeShimImpl.java:  109
                                                        ...                                   
                                          boot.pod/call-in*                      pod.clj:  410
                                      boot.pod/eval-fn-call                      pod.clj:  359
                                         clojure.core/apply                     core.clj:  657
                                                        ...                                   
                         adzerk.boot-cljs.impl/compile-cljs                     impl.clj:  151
                                       cljs.build.api/build                      api.clj:  205
                                         cljs.closure/build                  closure.clj: 2595
                             cljs.closure/handle-js-modules                  closure.clj: 2496
                            cljs.closure/process-js-modules                  closure.clj: 2389
                            cljs.closure/convert-js-modules                  closure.clj: 1680
                com.google.javascript.jscomp.Compiler.parse                Compiler.java:  995
          com.google.javascript.jscomp.Compiler.parseInputs                Compiler.java: 1731
      com.google.javascript.jscomp.deps.ModuleLoader.<init>            ModuleLoader.java:   92
com.google.javascript.jscomp.deps.ModuleLoader.resolvePaths            ModuleLoader.java:  276
java.lang.IllegalArgumentException: Duplicate module path after resolving: /home/corin/Projects/Demos/boot-parsets/node_modules/d3/d3.js
        clojure.lang.ExceptionInfo: Duplicate module path after resolving: /home/corin/Projects/Demos/boot-parsets/node_modules/d3/d3.js
    from: :boot-cljs
        clojure.lang.ExceptionInfo: Duplicate module path after resolving: /home/corin/Projects/Demos/boot-parsets/node_modules/d3/d3.js
    line: 33

Run `boot cljs` to reproduce the issue.

The patch attach removes duplicates from the set of input source files before they are preprocessed. With this patch the repo compiles correctly.



 Comments   
Comment by Mike Fikes [ 19/Nov/17 6:05 PM ]

Hi Corin,

  1. Have you signed the CA? (Your name doesn't appear on https://clojure.org/community/contributors)
  2. Can you provide a minimal repro that doesn't employ downstream tooling? (Bug-filing details are at https://clojurescript.org/community/reporting-issues)
Comment by Corin Lawson [ 19/Nov/17 6:27 PM ]

1. I have signed the CA (I did it after filing this bug).
2. Not a problem, I'll get on to that later today. Is it okay to link to the project on GitHub, or should I upload a tarball?

Comment by Mike Fikes [ 19/Nov/17 6:48 PM ]

Hi Corin, it is not OK to link to GitHub, and any repro should not make use of any downstream tooling (Leiningen, Boot, etc.)—this means that a repro would ideally depend only on the shipping cljs.jar, executable like the examples at Quick Start https://clojurescript.org/guides/quick-start

Comment by Mike Fikes [ 19/Nov/17 6:59 PM ]

Hey Corin, you may want to submit the patch using the instructions at https://clojurescript.org/community/patches (your current patch won't apply using git am, which is what I suspect David uses in the end.

Comment by Mike Fikes [ 19/Nov/17 7:24 PM ]

lein test is failing when applying patch

FAIL in (commonjs-module-processing) (module_processing_tests.clj:54)
processed modules are added to :libs
expected: (= {:foreign-libs [], :ups-foreign-libs [], :libs [(test/platform-path "out/src/test/cljs/reactJS.js") (test/platform-path "out/src/test/cljs/Circle.js")], :closure-warnings {:non-standard-jsdoc :off}} (env/with-compiler-env cenv (closure/process-js-modules {:foreign-libs [{:file "src/test/cljs/reactJS.js", :provides ["React"], :module-type :commonjs} {:file "src/test/cljs/Circle.js", :provides ["Circle"], :module-type :commonjs, :preprocess :jsx}], :closure-warnings {:non-standard-jsdoc :off}})))
  actual: (not (= {:foreign-libs [], :ups-foreign-libs [], :libs ["out/src/test/cljs/reactJS.js" "out/src/test/cljs/Circle.js"], :closure-warnings {:non-standard-jsdoc :off}} {:foreign-libs [], :closure-warnings {:non-standard-jsdoc :off}, :libs ["out/src/test/cljs/Circle.js" "out/src/test/cljs/reactJS.js"], :ups-foreign-libs []}))
Comment by Corin Lawson [ 19/Nov/17 7:48 PM ]

Thanks Mike,

I will make an effort follow all those instructions, but I do not see how I should provide the repro (no links, no attachments)... should it be inline code blocks?

Also, I shall be sure to run lein test before submitting the next patch. I note that the difference is only the order of the items in :libs vector, can you advise if order is important?

Comment by Mike Fikes [ 20/Nov/17 8:11 AM ]

Hi Corin,

Yes, inline code blocks are great. Anything minimal that doesn't depend on more than the shipping cljs.jar to demonstrate the issue (either directly in its REPL or via compiling using the build API). Here is a recent example using the build API: https://dev.clojure.org/jira/browse/CLJS-2397?focusedCommentId=47278&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-47278

There are actually other tests as well beyond lein test. See https://clojurescript.org/community/running-tests

I haven't looked into the details of this issue, so can't speak to to whether order of items is important.

Comment by Corin Lawson [ 20/Nov/17 8:47 AM ]

Steps to reproduce the problem.

Consider the following three source files:

src/distinct_inputs/core.cljs
(ns distinct-inputs.core
  (:require [d3]
            [circle :refer [circle]]))

(-> d3
    (.select "body")
    (.append "svg")
    (.attr "width" 200)
    (.attr "height" 200)
    (.call circle "steelblue"))
es6/circle.js
import * as d3 from 'd3';

export function circle(sel, color) {
  return sel
    .append("circle")
    .attr("cx", "100px")
    .attr("cy", "100px")
    .attr("r",  "100px")
    .attr("fill", color);
}
build.clj
(require 'cljs.build.api)

(cljs.build.api/build
  "src"
  {:main 'distinct-inputs.core
   :output-to "out/main.js"
   :install-deps true
   :foreign-libs [{:file "es6"
                   :module-type :es6}]
   :npm-deps     {:d3 "3.5.16"}})

Execute cljs:

java -cp cljs.jar:src clojure.main build.clj

Expected outcome

cljs should produce nothing to the standard output, exit cleanly and write the following files (approximately).

out/main.js
var CLOSURE_UNCOMPILED_DEFINES = {};
var CLOSURE_NO_DEPS = true;
if(typeof goog == "undefined") document.write('<script src="out/goog/base.js"></script>');
document.write('<script src="out/goog/deps.js"></script>');
document.write('<script src="out/cljs_deps.js"></script>');
document.write('<script>if (typeof goog == "undefined") console.warn("ClojureScript could not load :main, did you forget to specify :asset-path?");</script>');
document.write('<script>goog.require("process.env");</script>');
document.write('<script>goog.require("distinct_inputs.core");</script>');
out/distinct_inputs/core.js
goog.provide('distinct_inputs.core');
goog.require('cljs.core');
goog.require('module$distinct_inputs$node_modules$d3$d3');
goog.require('module$distinct_inputs$es6$circle');
module$distinct_inputs$node_modules$d3$d3.select("body").append("svg").attr("width",(200)).attr("height",(200)).call(module$distinct_inputs$es6$circle.circle,"steelblue");

//# sourceMappingURL=core.js.map
out/es6/circle.js
goog.provide("module$distinct_inputs$es6$circle");goog.require("module$distinct_inputs$node_modules$d3$d3");function circle$$module$distinct_inputs$es6$circle(sel,color){return sel.append("circle").attr("cx","100px").attr("cy","100px").attr("r","100px").attr("fill",color)}module$distinct_inputs$es6$circle.circle=circle$$module$distinct_inputs$es6$circle

Actual outcome.

cljs exits with exit code 1 and produces the following standard out.

stdout
Exception in thread "main" java.lang.IllegalArgumentException: Duplicate module path after resolving: /distinct_inputs/node_modules/d3/d3.js, compiling:(/distinct_inputs/build.clj:3:1)
	at clojure.lang.Compiler.load(Compiler.java:7391)
	at clojure.lang.Compiler.loadFile(Compiler.java:7317)
	at clojure.main$load_script.invokeStatic(main.clj:275)
	at clojure.main$script_opt.invokeStatic(main.clj:335)
	at clojure.main$script_opt.invoke(main.clj:330)
	at clojure.main$main.invokeStatic(main.clj:421)
	at clojure.main$main.doInvoke(main.clj:384)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:379)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalArgumentException: Duplicate module path after resolving: /distinct_inputs/node_modules/d3/d3.js
	at com.google.javascript.jscomp.deps.ModuleLoader.resolvePaths(ModuleLoader.java:276)
	at com.google.javascript.jscomp.deps.ModuleLoader.<init>(ModuleLoader.java:92)
	at com.google.javascript.jscomp.Compiler.parseInputs(Compiler.java:1731)
	at com.google.javascript.jscomp.Compiler.parse(Compiler.java:995)
	at cljs.closure$convert_js_modules.invokeStatic(closure.clj:1680)
	at cljs.closure$process_js_modules.invokeStatic(closure.clj:2371)
	at cljs.closure$handle_js_modules.invokeStatic(closure.clj:2495)
	at cljs.closure$build.invokeStatic(closure.clj:2592)
	at cljs.build.api$build.invokeStatic(api.clj:204)
	at cljs.build.api$build.invoke(api.clj:189)
	at cljs.build.api$build.invokeStatic(api.clj:192)
	at cljs.build.api$build.invoke(api.clj:189)
	at user$eval24.invokeStatic(build.clj:3)
	at user$eval24.invoke(build.clj:3)
	at clojure.lang.Compiler.eval(Compiler.java:6927)
	at clojure.lang.Compiler.load(Compiler.java:7379)
	... 11 more

None of the aforementioned expected files are produced.

Cause of the exception.

The exception emitted by ModuleLoader#resolvePaths is a result of the same input file (i.e. node_modules/d3/d3.js) having been specified more than once to Compiler#initModules. There happens to be this note in Compiler#getAllInputsFromModules:

src/com/google/javascript/jscomp/Compiler.java
// NOTE(nicksantos): If an input is in more than one module,
        // it will show up twice in the inputs list, and then we
        // will get an error down the line.

cljs.closure/process-js-modules is provided a :foreign-libs vector which contains a repeated entry for node_modules/d3/d3.js (and also it's package.json). That vector is a result of multiple invocations of cljs.closure/node-inputs; once for out/cljs$node_modules.js (which is presumably a result of the dependency in distinct_inputs/core) and again for es6/circle.js.

In short, the dependency on D3 is pulled in by both ClojureScript source files and JavaScript module source files.

Proposed solution.

In this scenario the :foreign-libs vector contains repeated entries dispite the use of distinct within cljs.closure/node-inputs. A possible solution would to remove the use of distinct within cljs.closure/node-inputs and require the caller of cljs.closure/node-inputs to use distinct.

Solution A
From 063e35080c14d35189ab7827f25f071e958ab5b4 Mon Sep 17 00:00:00 2001
From: Corin Lawson <corin@responsight.com>
Date: Tue, 21 Nov 2017 01:31:53 +1100
Subject: [PATCH] CLJS-2402: Ensure :foreign-libs vector contains distinct
 entries.

---
 src/main/clojure/cljs/closure.clj | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/src/main/clojure/cljs/closure.clj b/src/main/clojure/cljs/closure.clj
index a686f878..74a0cc86 100644
--- a/src/main/clojure/cljs/closure.clj
+++ b/src/main/clojure/cljs/closure.clj
@@ -2219,7 +2219,7 @@
      (when env/*compiler*
        (:options @env/*compiler*))))
   ([entries opts]
-   (into [] (distinct (mapcat #(node-module-deps % opts) entries)))))
+   (into [] (mapcat #(node-module-deps % opts) entries))))
 
 (defn index-node-modules
   ([modules]
@@ -2480,14 +2480,15 @@
         output-dir (util/output-directory opts)
         opts (update opts :foreign-libs
                (fn [libs]
-                 (into (if (= target :nodejs)
-                         []
-                         (index-node-modules node-required))
-                   (into expanded-libs
-                     (node-inputs (filter (fn [{:keys [module-type]}]
-                                            (and (some? module-type)
-                                              (not= module-type :amd)))
-                                    expanded-libs))))))
+                 (distinct
+                   (into (if (= target :nodejs)
+                           []
+                           (index-node-modules node-required))
+                         (into expanded-libs
+                               (node-inputs (filter (fn [{:keys [module-type]}]
+                                                      (and (some? module-type)
+                                                           (not= module-type :amd)))
+                                                    expanded-libs)))))))
         opts (if (some
                    (fn [ijs]
                      (let [dest (io/file output-dir (rel-output-path (assoc ijs :foreign true) opts))]
-- 
2.13.0

A more general solution is cljs.closure/process-js-modules must ensure the set of input files (i.e. js-modules) is distinct. This patch would be simpler (i.e. doesn't mess with code I don't understand) and closer to the call to Google Closure Compiler.

Solution B
From 6bf11a24b93642e118e6d29c5af8a137fa01ea94 Mon Sep 17 00:00:00 2001
From: Corin Lawson <corin@responsight.com>
Date: Sun, 19 Nov 2017 20:25:31 +1100
Subject: [PATCH] CLJS-2402: Ensure input source files are distinct.

---
 src/main/clojure/cljs/closure.clj | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/main/clojure/cljs/closure.clj b/src/main/clojure/cljs/closure.clj
index a686f878..24421bde 100644
--- a/src/main/clojure/cljs/closure.clj
+++ b/src/main/clojure/cljs/closure.clj
@@ -2364,7 +2364,8 @@
   (let [;; Modules from both :foreign-libs (compiler options) and :ups-foreign-libs (deps.cljs)
         ;; are processed together, so that files from both sources can depend on each other.
         ;; e.g. commonjs module in :foreign-libs can depend on commonjs module from :ups-foreign-libs.
-        js-modules (filter :module-type (concat (:foreign-libs opts) (:ups-foreign-libs opts)))]
+        js-modules (filter :module-type (concat (:foreign-libs opts) (:ups-foreign-libs opts)))
+        js-modules (distinct js-modules)]
     (if (seq js-modules)
       (util/measure (:compiler-stats opts)
         "Process JS modules"
-- 
2.13.0

FWIW: I prefer Solution B.

Comment by Corin Lawson [ 20/Nov/17 8:49 AM ]

Attached proposed Solution B

Comment by Corin Lawson [ 20/Nov/17 9:02 AM ]

Hi Mike,

I hope this is to your's (and BDFL's) satisfaction now; I ran lein test for both proposed solutions and I do not receive any failures. I do receive errors, however, that do not occur in assertions. I assume that the cause is something peculiar (or lack thereof) in my setup. Let me know if you require anything else from me.

Cheers,
Corin.

Comment by Mike Fikes [ 20/Nov/17 12:08 PM ]

Thanks Corin. The entire test suite passes for me with your latest patch.

Comment by Mike Fikes [ 11/May/19 8:46 PM ]

CLJS-2402.patch added to Patch Tender





[CLJS-2393] process.env preload is added after all existing preloads Created: 31/Oct/17  Updated: 17/Nov/17

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.908
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Eugene Pakhomov Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: npm-deps


 Description   

When there're other preloads and the check for process-shim returns true, `process.env` is added to the end of the vector of all preloads.
That causes problems when one of the previous preloads requires something that checks `process.env` during loading.

E.g. I have a Reagent application with `:npm-deps` set to React and a few other libraries. And I have a preload exactly like this one: https://github.com/flexsurfer/re-frisk/blob/master/src/re_frisk/preload.cljs
The issue is that `re-frisk.core` requires Reagent, which requires React, which checks `process.env` - all before `process.env` was actually created.

I think that `process.env` should go before all existing preloads.



 Comments   
Comment by Hendrik Poernama [ 02/Nov/17 8:02 AM ]

Workaround is to manually add process.env to the first of :preloads vector





[CLJS-2382] circular dependency in node_modules prevents compilation Created: 13/Oct/17  Updated: 10/Feb/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.908
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Hendrik Poernama Assignee: Juho Teperi
Resolution: Unresolved Votes: 1
Labels: npm-deps


 Description   

Compiler complains of missing provides when there is circular dependency in node_modules.

Cause is the order of the 'goog.addDependency' statements in out/cljs_deps.js

goog.addDependency("../node_modules/lib1.js", ['lib1'], ['lib2']); // This line will fail since 'lib2' is not yet provided
goog.addDependency("../node_modules/lib2.js", ['lib2'], ['lib1']);

Example of affected node_modules: apollo-client 1.9.2

I'm not sure if this is a closure compiler limitation or explicitly unsupported, but it does reduce the number of node packages that can be included using node_modules.

Current workaround is to rewrite the library code to not have circular deps or to switch to cljsjs.



 Comments   
Comment by Juho Teperi [ 10/Feb/18 5:28 PM ]

I didn't notice circular dependencies when testing Apollo-client 1.9.2 with latest Cljs changes, but there is something else strange going on.

code
// Code from apollo-client references some modules by incorrect name:
goog.provide("module$home$juho$Source$reagent$test_environments$browser_node$node_modules$apollo_client$transport$networkInterface");
goog.require("module$whatwg_fetch");
goog.require("module$graphql$language$printer");

// From whatwg file:
goog.provide("module$home$juho$Source$reagent$test_environments$browser_node$node_modules$whatwg_fetch$fetch");
code

It is possible this is caused by Cljs bug, because provide name is created by us, and require names are provided by Closure.





[CLJS-2376] Add support for ES6 default imports/exports Created: 02/Oct/17  Updated: 17/Sep/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Thomas Heller Assignee: Juho Teperi
Resolution: Unresolved Votes: 17
Labels: npm-deps

Attachments: Text File CLJS-2376-1.patch    

 Description   

ES6 has special syntax for using "default" imports and there is currently no equivalent for CLJS when using imported ES6 code.

import * as name from "module-name";
(:require ["module-name" :as name])

import { export } from "module-name";
(:require ["module-name" :refer (export)])

import { export as alias } from "module-name";
(:require ["module-name" :refer (export) :rename {export alias}])

import defaultExport from "module-name";
import defaultExport, { export [ , [...] ] } from "module-name";
import defaultExport, * as name from "module-name";
;; no easy access to defaultExport

I'm proposing to add a :default to the ns :require.

(:require ["module-name" :as mod :default foo])

This makes it much more convenient to use rewritten ES6 code from CLJS. If "module-name" has a default export you currently have to write mod/default everywhere since they is no easy way to alias the default.

(:require ["material-ui/RaisedButton" :as RaisedButton])
;; :as is incorrect and the user must now use
RaisedButton/default

(:require ["material-ui/RaisedButton" :default RaisedButton])
;; would allow direct use of
RaisedButton

Internally the Closure compiler (and ES6 code rewritten by babel) will rewrite default exports to a .default property, so :default really is just a convenient way to access it.

The long version that already works is

(:require ["material-ui/RaisedButton" :refer (default) :rename {default RaisedButton}])

When ES6 becomes more widespread we should have convenient way to correctly refer to "default" exports.

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import



 Comments   
Comment by Juho Teperi [ 10/Jan/18 2:25 PM ]

First take on the implementation. This implementation rewrites :default name to {{:refer [default] :rename {default name}}} at the lowest level (parse-require-spec), so I think other functions don't need to be changed.

I didn't update the require spec validation errors yet.

Comment by David Nolen [ 12/Feb/18 6:25 AM ]

It seems to me the most desirable syntax for ES6 default imports is the following:

(:require ["material-ui/RaisedButton" :refer [RaisedButton]])

I don't see why we couldn't just make this work?

Comment by Thomas Heller [ 12/Feb/18 6:55 AM ]

I don't understand that suggestion. The default export has no name, the .default property is only used in interop by transpilers. In actual JS a live reference is used. How would you map RaisedButton back to default?

["material-ui/RaisedButton" :default foo] would be valid. The JS module may also actually have an `export { RaisedButton }` in addition to `export default ...`, which would lead to a conflict if you "guess" the wrong name. Default exports are not aliases for the module itself, that is separate.

I made a reference translation table [1] and adding the :default alias is the only way to reliably map all ES6 import/export features. I don't see a way to make :refer work that would not be totally non-intuitive and unreliable.

import defaultExport, * as name from "module-name";

[1] https://shadow-cljs.github.io/docs/UsersGuide.html#_using_npm_packages

Comment by David Nolen [ 12/Feb/18 8:55 AM ]

Let's leave rhetoric out of this discussion please and focus on the technical bits. I just don't want to include another directive in the ns form and I would prefer an approach that avoids that. From my point of view there has not been enough exploration of that alternative.

This ticket is mostly about convenience, and I think we need to stew on this one for a bit longer before deciding on anything.

Comment by jcr [ 24/Mar/18 6:45 AM ]

Is there a reason we can't use metadata to deal with it? I.e.

(:require ["material-ui/RaisedButton" :refer [^:default RaisedButton]])
(:require ["module-name" :as mod :refer [^:default foo])
(:require ["module-name" :refer [^:default foo, bar, baz])

This doesn't require additional ns directives and doesn't cause any ambiguities either.

Since this wasn't used for macros (i.e. we have :require-macros instead of plain :require + ^:macros or :refer [^:macro foo]), I assume there may be some nuances I am not aware of that prevent implementing it. Are there any?

Comment by Thomas Heller [ 26/Apr/18 5:49 PM ]

The ^:default hint seems far more complicated to me from an implementation standpoint.

There is no equivalent for default import/export in Java/JVM and Clojure. There is also no equivalent in CommonJS even. It is strictly about ES6. Therefore bolting it onto "other" methods seems incorrect to me.

The reason it should be separate IMHO is that using the .default property is not strictly correct and the fact that it exists at all is purely for compatibility reasons with CommonJS. In strict ES6 they don't exist as part of the "Object" created by :as.

;; a.js
export let a = 1;
export let b = 2;
export default 3;

;; b.js
import x, * as y from "./a.js"

// these exist
y.a
y.b
// this is NOT valid in ES6 and does not exist
y.default
// instead x must be used
x

We certainly rely on the .default property existing for the time being given that CLJS does not emit ES6. webpack already changed how the default exports are handled however and I do expect the Closure Compiler to do something similar in the future.

Comment by Thomas Heller [ 26/Apr/18 6:06 PM ]

Turns out I'm actually wrong. Re-read the spec and .default is actually defined as a getter, so it is accessible. Nevermind my previous post then.

Doesn't change my opinion about the usefulness of :default as a simpler aliasing method though.

Comment by Paulus Esterhazy [ 17/Sep/18 8:46 AM ]

I ran into ES6 default exports, which are common these days. Folks stumble over `default` in the code, e.g.: https://github.com/pesterhazy/cljs-spa-example/issues/13. I wanted to report my findings.

As explained in the description of this issue, you can use a combination of `:refer` and `:rename` to get rid of the references to `:default` in the code, moving it into the NS declaration. It's not the most elegant solution but gets the job done and works with global exports: https://github.com/pesterhazy/cljs-spa-example/pull/14/files

Note that AIUI because `default` is a strict keyword in ES3, you need to set the following compiler option for this to work:

```
:language-out :ecmascript5
```

Otherwise `default` gets munged to `default$`





[CLJS-2369] Undefined nameToPath for bootstrap` when using Twitter's Bootstrap (twbs) Created: 24/Sep/17  Updated: 19/May/18  Resolved: 19/May/18

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Corin Lawson Assignee: Mike Fikes
Resolution: Not Reproducible Votes: 2
Labels: npm-deps
Environment:

1.9.946



 Description   

How to reproduce problem

src/cljsbuild_bootstrap4/core.cljs
(ns cljsbuild-bootstrap4.core
  (:require [bootstrap]))

(enable-console-print!)

(println "Hello world!")

(-> ".alert" js/jQuery (.alert "close"))
index.html
<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<title>cljsbuild-bootstrap4</title>
	<link rel="stylesheet" href="node_modules/bootstrap/dist/css/bootstrap.css"/>
</head>
<body>
	<div class="alert alert-warning alert-dismissible fade show" role="alert">
		<button type="button" class="close" data-dismiss="alert" aria-label="Close">
			<span aria-hidden="true">&times;</span>
		</button>
		<strong>Holy guacamole!</strong> I should be fading.
	</div>
	<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
	<script src="out/main.js"></script>
</body>
</html>
build.clj
(require 'cljs.build.api)

(cljs.build.api/build "src" {:output-to "out/main.js"
                             :main 'cljsbuild-bootstrap4.core
                             :install-deps true
                             :npm-deps {:bootstrap "4.0.0-beta"}})
  1. Copy the 1.9.946 cljs.jar to the current working directory
  2. Run
    java -cp cljs.jar:src clojure.main build.clj
  3. open index.html
  4. Observe console messages.

Expected result

  1. Console should show the following messages:
    Error: Bootstrap dropdown require Popper.js (https://popper.js.org)
    Hello world!
  2. The Bootstrap Alert component should fade from the page resulting in a blank page.
  3. Compiler should produce:
    out/cljs_deps.js
    goog.addDependency("base.js", ['goog'], []);
    goog.addDependency("../cljs/core.js", ['cljs.core'], ['goog.string', 'goog.object', 'goog.math.Integer', 'goog.string.StringBuffer', 'goog.array', 'goog.math.Long']);
    goog.addDependency("../process/env.js", ['process.env'], ['cljs.core']);
    goog.addDependency("../node_modules/bootstrap/dist/js/bootstrap.js", ['bootstrap'], []); 
    goog.addDependency("../cljsbuild_bootstrap4/core.js", ['cljsbuild_bootstrap4.core'], ['cljs.core', 'bootstrap']);

Actual result

  1. Console shows
    Error: Undefined nameToPath for bootstrap
  2. The Bootstrap Alert component fails to fade from the page and remains on the page.
  3. Compiler produces:
    out/cljs_deps.js
    goog.addDependency("base.js", ['goog'], []);
    goog.addDependency("../cljs/core.js", ['cljs.core'], ['goog.string', 'goog.object', 'goog.math.Integer', 'goog.string.StringBuffer', 'goog.array', 'goog.math.Long']);
    goog.addDependency("../process/env.js", ['process.env'], ['cljs.core']);
    goog.addDependency("../cljsbuild_bootstrap4/core.js", ['cljsbuild_bootstrap4.core'], ['cljs.core', 'bootstrap']);


 Comments   
Comment by Hendrik Poernama [ 28/Sep/17 7:44 PM ]

I came across the same problem and did some tracing. It looks like 'load-foreign-library' correctly populated the 'provides' key, but 'library-graph-node' failed to get the 'provides'. I also observed that the output of library-graph-node is the one that gets passed to cljs_deps.js

load-foreign-library: /Users/Hendrik/Projects/bugreports/cljsbuild-bootstrap4/node_modules/bootstrap/dist/js/bootstrap.js
provides: ["bootstrap" "bootstrap/dist/js/bootstrap.js" "bootstrap/dist/js/bootstrap"]

load-foreign-library: /Users/Hendrik/Projects/bugreports/cljsbuild-bootstrap4/node_modules/bootstrap/package.json
provides: []

Copying file:/Users/Hendrik/Projects/bugreports/cljsbuild-bootstrap4/node_modules/bootstrap/dist/js/bootstrap.js to out/node_modules/bootstrap/dist/js/bootstrap.js

load-library: out/node_modules/bootstrap/dist/js/bootstrap.js

library-graph-node: /Users/Hendrik/Projects/bugreports/cljsbuild-bootstrap4/out/node_modules/bootstrap/dist/js/bootstrap.js
{:requires [], :provides [],
:url #object[java.net.URL 0x6a0659ac "file:/<snip>/cljsbuild-bootstrap4/out/node_modules/bootstrap/dist/js/bootstrap.js"],
:closure-lib true, :lib-path "out/node_modules/bootstrap/dist/js/bootstrap.js"}

Comment by Hendrik Poernama [ 29/Sep/17 3:31 AM ]

I am also getting the same error when trying to use 'apollo-client'. 'cljs_deps.js' would be missing the 'whatwg-fetch' dependency.

I suspect this is because 'whatwg-fetch' is a polyfill that does not explicitly export anything. I was able to workaround the issue by appending the following lines to the bottom of '<projectdir>/node_modules/whatwg-fetch/fetch.js'

const whatwg_fetch = 1;
export { whatwg_fetch };

I don't know enough to say that this is the same problem, other than the error message being identical. I have not tried adding the same hack to bootstrap.js

Comment by Derek Chiang [ 08/Jan/18 2:03 AM ]

Getting the same error for `ethereumjs-abi` too.

Log from console:

```
base.js:1357 Uncaught Error: Undefined nameToPath for ethereumjs_abi
at visitNode (base.js:1357)
at visitNode (base.js:1355)
at visitNode (base.js:1355)
at Object.goog.writeScripts_ (base.js:1369)
at Object.goog.require (base.js:706)
at (index):46
```

Comment by Mike Fikes [ 19/May/18 4:08 PM ]

Corin, I can't repro. I've also updated the description so that anyone can attempt to repro. (Minimal per https://clojurescript.org/community/reporting-issues)

I get a slightly different out/cljs_deps.js (below), but otherwise, I get the expected behavior as per the ticket description.

out/cljs_deps.js
goog.addDependency("base.js", ['goog'], []);
goog.addDependency("../node_modules/bootstrap/dist/js/bootstrap.js", ['module$Users$mfikes$Desktop$minimal$node_modules$bootstrap$dist$js$bootstrap'], []);
goog.addDependency("../cljs/core.js", ['cljs.core'], ['goog.string', 'goog.Uri', 'goog.object', 'goog.math.Integer', 'goog.string.StringBuffer', 'goog.array', 'goog.math.Long']);
goog.addDependency("../process/env.js", ['process.env'], ['cljs.core']);
goog.addDependency("../cljsbuild_bootstrap4/core.js", ['cljsbuild_bootstrap4.core'], ['module$Users$mfikes$Desktop$minimal$node_modules$bootstrap$dist$js$bootstrap', 'cljs.core']);
Comment by Mike Fikes [ 19/May/18 4:22 PM ]

Ahh I see what is going on. This is a problem with 1.9.946. No longer reproducible with 1.10.238.





[CLJS-2353] use portable `node-module-dep?` function in analyze-deps Created: 10/Sep/17  Updated: 11/Sep/17  Resolved: 11/Sep/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.908
Fix Version/s: 1.10.238

Type: Defect Priority: Major
Reporter: António Nuno Monteiro Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: npm-deps

Attachments: Text File CLJS-2353.patch    
Patch: Code
Approval: Accepted

 Comments   
Comment by David Nolen [ 11/Sep/17 6:33 AM ]

fixed https://github.com/clojure/clojurescript/commit/fe28810661256907f1402ac1c7c8556db1cf616a





[CLJS-2325] konan cannot handle `export ... from` imports Created: 16/Aug/17  Updated: 16/Aug/17  Resolved: 16/Aug/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.908

Type: Defect Priority: Critical
Reporter: David Nolen Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: modules, npm-deps

Approval: Vetted

 Description   

We use a very small dependency Node.js `konan` to supply import detection. Unfortunately `konan` only handles `require` and `import`, it doesn't check for `export ... from`. Given how trivial this code is we should just probably write our own logic for this in `module_deps.js`.



 Comments   
Comment by David Nolen [ 16/Aug/17 1:31 PM ]

fixed https://github.com/clojure/clojurescript/commit/56a880cb09d57e287a3eba4839c4f5688958850f





[CLJS-2290] Node packages using require('assert') fail compilation Created: 31/Jul/17  Updated: 01/Aug/17  Resolved: 01/Aug/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.908

Type: Defect Priority: Critical
Reporter: Will Cohen Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Attachments: Text File CLJS-2290.patch    
Patch: Code and Test
Approval: Vetted

 Description   

When including packages that contain "require('assert')" as a node dependency – for example, https://github.com/uber/react-map-gl – Clojurescript fails to compile the module with the following error (note that importing react-map-gl currently requires closure compiler master because Mapbox GL has dependencies which require the fix from https://github.com/google/closure-compiler/pull/2579):

WARNING: JSC_JS_MODULE_LOAD_WARNING. Failed to load module "assert" at [...]/node_modules/react-map-gl/dist/utils/map-state.js line 14 : 4
WARNING: JSC_JS_MODULE_LOAD_WARNING. Failed to load module "assert" at [...]/node_modules/react-map-gl/dist/utils/map-state.js line 14 : 14
WARNING: JSC_JS_MODULE_LOAD_WARNING. Failed to load module "assert" at [...]/node_modules/viewport-mercator-project/dist/perspective-mercator-viewport.js line 24 : 14
WARNING: JSC_JS_MODULE_LOAD_WARNING. Failed to load module "assert" at [...]/node_modules/viewport-mercator-project/dist/perspective-mercator-viewport.js line 24 : 4
WARNING: JSC_JS_MODULE_LOAD_WARNING. Failed to load module "assert" at [...]/node_modules/viewport-mercator-project/dist/viewport.js line 52 : 14
WARNING: JSC_JS_MODULE_LOAD_WARNING. Failed to load module "assert" at [...]/node_modules/viewport-mercator-project/dist/viewport.js line 52 : 4

For reference, the relevant js files:
https://unpkg.com/react-map-gl@3.0.1/dist/utils/map-state.js
https://unpkg.com/viewport-mercator-project@4.1.1/dist/perspective-mercator-viewport.js
https://unpkg.com/viewport-mercator-project@4.1.1/dist/viewport.js

Naively adding https://github.com/defunctzombie/commonjs-assert to package.json or :npm-deps alone does not resolve the issue.



 Comments   
Comment by António Nuno Monteiro [ 31/Jul/17 10:38 PM ]

Attached patch with fix.

Comment by David Nolen [ 01/Aug/17 7:19 AM ]

fixed https://github.com/clojure/clojurescript/commit/b17c18604daf97fa6ca54d226bdfedcfd7ad6a17





[CLJS-2289] Node packages with browser entry don't work with Closure Created: 31/Jul/17  Updated: 15/Sep/17  Resolved: 15/Sep/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.10.238

Type: Enhancement Priority: Major
Reporter: Juho Teperi Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Approval: Vetted

 Description   

Closure doesn't yet support package.json browser field: https://github.com/google/closure-compiler/issues/2433

However, our, module_deps.js / brower-resolve, and use this field.

One package that has problems is https://github.com/PaulLeCam/react-leaflet which depends on https://github.com/BerkeleyTrue/warning which uses browser entry.
index-node-modules-dir reuturns a list of all JS files in node_modules, this includes both warning/warning.js and warning/browser.js. index-node-modules only includes warning/browser.js. Because Closure doesn't understand package.json browser field, it can't resolve the warning require when processing React-leaflet: https://github.com/PaulLeCam/react-leaflet/blob/master/lib/Pane.js#L47

(cljs.closure/index-node-modules-dir)
;; =>
{:file "/home/juho/Source/x/y/node_modules/warning/warning.js", :module-type :es6, :provides ["warning/warning.js" "warning/warning" "warning"]}
{:file "/home/juho/Source/x/y/node_modules/warning/browser.js", :module-type :es6, :provides ["warning/browser.js" "warning/browser"]}
(cljs.closure/index-node-modules ["react-leaflet"])
;; =>
{:file "/home/juho/Source/x/y/node_modules/warning/browser.js", :module-type :es6, :provides ["warning" "warning/browser.js" "warning/browser"]}


 Comments   
Comment by António Nuno Monteiro [ 03/Aug/17 5:31 PM ]

This is now fixed in Closure Compiler master (https://github.com/google/closure-compiler/pull/2598).





[CLJS-2281] module_deps.js cannot compute inputs for ES6 sources Created: 28/Jul/17  Updated: 30/Jul/17  Resolved: 30/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.908

Type: Defect Priority: Major
Reporter: David Nolen Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Attachments: Text File CLJS-2281.patch    
Patch: Code and Test
Approval: Accepted

 Description   

Currently module_deps.js cannot follow requires for ES6 sources because it expects that it will parse CommonJS. module-deps supports transforms prior to CommonJS require resolution for this specific reason. We should leverage this functionality. A good benchmark is if we can compile Google's fully Closure compatible Material Components project without issue.



 Comments   
Comment by Jeremy Schoffen [ 29/Jul/17 10:27 AM ]

Hello, here is a minimal repo concerning this issue, hope this helps testing.

https://github.com/JeremS/class-test-npm

Cheers,

Comment by António Nuno Monteiro [ 29/Jul/17 7:03 PM ]

Attached patch with fix.

Unfortunately `module-deps`'s transforms are not applied to files in `node_modules`. As per the README: "When you call mdeps() with an opts.transform, the transformations you specify will not be run for any files in node_modules/. This is because modules you include should be self-contained and not need to worry about guarding themselves against transformations that may happen upstream."

The attached patch employs a different strategy. I forked `module-deps` to https://github.com/cljs-oss/module-deps which includes the changes in this pull request: https://github.com/substack/module-deps/pull/63, allowing us to use konan (https://www.npmjs.com/package/konan) instead of `detective` (https://www.npmjs.com/package/detective) to parse both ES6 and CommonJS imports.

This way, we have a minimal patch on top of `module-deps` and don't need to worry about writing transformations that could prove buggy, relying instead on a tested library that parses both imports and ES6. We can always change to the original module-deps if the aforementioned PR is merged.

The repo linked in the comment above works with this patch after fixing bugs in the repro itself: account for the ES6 default export, a typo where MDCCheckbox is used but the imported module is a text field and the usage of `defaultAdapter` as a function with it's a property.

Comment by Thomas Heller [ 30/Jul/17 3:51 AM ]

From a tool perspective (ie. shadow-cljs) I really did not like launching a node process for module_deps.js. Instead I rewrote all of it in Clojure plus a little Java Class that parses .js files and looks for require and import/export using the Closure Compiler. It is fairly simple code and completely eliminates the need for detective and others. Replicating everything module_deps.js does is a bit more work but the central piece is the extractor for the requires.

Not saying that you should do this as well but it is an option.

The .java code looks like this:
https://gist.github.com/thheller/d5761ee8adbe6f857bf6e05a71efda06

I have not merged the code into shadow-cljs yet but it does work quite well.

Comment by David Nolen [ 30/Jul/17 8:15 AM ]

fixed https://github.com/clojure/clojurescript/commit/da9dd66458aed0bed95f2c0f4d9cd2d6ba36c2df





[CLJS-2255] Clean up :npm-deps Created: 17/Jul/17  Updated: 25/Jul/17  Resolved: 25/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Task Priority: Critical
Reporter: David Nolen Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: npm-deps

Attachments: Text File CLJS-2255.patch    

 Description   

We should separate module installation from building and REPLs. I propose the following new public build fns:

cljs.build.api/install-deps!
cljs.build.api/get-deps
cljs.build.api/print-deps


 Comments   
Comment by David Nolen [ 17/Jul/17 2:00 AM ]

`get-deps` should return a vector of {:name ... :version ... :type ...} entries

Comment by António Nuno Monteiro [ 17/Jul/17 8:37 PM ]

Attached a tentative patch adding `cljs.build.api/install-node-deps!` and `cljs.build.api/get-node-deps` functions.

Looking for feedback if this is the satisfies the requirements for what you had in mind. IMO printing dependencies is just a call away from `get-node-deps`.

Comment by David Nolen [ 18/Jul/17 12:36 AM ]

I agree about printing deps. These functions should probably take compiler options since they are user facing and that's the easiest interface from that standpoint.

We should also remove automatic calls to dep installation in build as well as REPLs. Installing dependencies on each build, REPL start just doesn't seem like a good idea.

Comment by David Nolen [ 25/Jul/17 1:19 AM ]

After thinking about it some more I think the supplied patch is OK.

Comment by David Nolen [ 25/Jul/17 1:26 AM ]

fixed https://github.com/clojure/clojurescript/commit/8972224b4b4617a98a9fdd497af1aeb91a29ed2a





[CLJS-2254] Module Indexing: Provide relative paths for a package's main module Created: 16/Jul/17  Updated: 17/Jul/17  Resolved: 17/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Enhancement Priority: Critical
Reporter: António Nuno Monteiro Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Attachments: Text File CLJS-2254.patch    
Patch: Code and Test
Approval: Accepted

 Comments   
Comment by António Nuno Monteiro [ 16/Jul/17 8:14 PM ]

The attached patch also provides tests for previously untested `module_deps.js` indexing to make sure it's at parity with the indexing on the Clojure side.

Comment by David Nolen [ 17/Jul/17 12:53 AM ]

fixed https://github.com/clojure/clojurescript/commit/7139c4c17932b1e11e7a2b665914b32936f02644





[CLJS-2238] Perf regression with node module indexing Created: 14/Jul/17  Updated: 14/Jul/17  Resolved: 14/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Defect Priority: Blocker
Reporter: Mike Fikes Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps, performance

Attachments: Text File CLJS-2238.patch    
Patch: Code
Approval: Accepted

 Description   

On master, I'm observing that, using Figwheel, changes to source take about 12 seconds to appear in a React Native app running in an iOS simulator, whereas with the 1.9.671 such changes take about 2 seconds.

I apologize for not having a minimal repro available. I suspect it is easily reproducible locally using a setup generated using re-natal init FutureApp (see https://github.com/drapanjanas/re-natal)

Of note: I don't have :npm-deps set. My compiler options are:

{:output-to     "target/ios/not-used.js"
 :main          "env.ios.main"
 :output-dir    "target/ios"
 :optimizations :none
 :checked-arrays false 
 :warnings      true}

A git bisect shows that this was introduced with the change in CLJS-2212.



 Comments   
Comment by Mike Fikes [ 14/Jul/17 1:26 PM ]

For reference, in handle-js-modules the top-level set ends up having 14748 elements, and it does appear to take about 12 seconds to produce that number.

My requires set has 77 elements and the node-required (the intersection) has 0 elements.

Comment by David Nolen [ 14/Jul/17 2:03 PM ]

fixed https://github.com/clojure/clojurescript/commit/b17b83121486d6e8ffe9887208cdf105993b73cd

Comment by Mike Fikes [ 14/Jul/17 2:04 PM ]

With the patch, changes appear in about 2 seconds.





[CLJS-2212] Replace missing-js-modules with new index-node-module-dir Created: 11/Jul/17  Updated: 11/Jul/17  Resolved: 11/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Enhancement Priority: Blocker
Reporter: Juho Teperi Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Attachments: Text File CLJS-2212.patch    
Patch: Code
Approval: Accepted

 Comments   
Comment by David Nolen [ 11/Jul/17 5:32 PM ]

fixed https://github.com/clojure/clojurescript/commit/1eabfe9aced4b48271be93d367104d407e8d6bf3





[CLJS-2206] Refactor :npm-deps functionality to use indexed node_modules instead of handling missing modules in special pass Created: 10/Jul/17  Updated: 13/Jul/17  Resolved: 13/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Task Priority: Critical
Reporter: David Nolen Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: js-modules, nodejs, npm-deps

Approval: Vetted

 Description   

Doing this will have the added benefit in supporting enhancements to the Node.js target. By indexing `node_modules` up front we can later determine how a particular library should be required - either `goog.require` or CommonJS `require`.



 Comments   
Comment by David Nolen [ 11/Jul/17 4:55 PM ]

Depends on CLJS-2211

Comment by David Nolen [ 11/Jul/17 5:35 PM ]

Partially addressed by CLJS-2212. We should audit / remove any vestigial missing-js-modules logic if there is any.





[CLJS-2202] String requires not working from files in classpath Created: 09/Jul/17  Updated: 09/Jul/17  Resolved: 09/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Defect Priority: Blocker
Reporter: Juho Teperi Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Attachments: Text File CLJS-2202.patch    
Patch: Code and Test
Approval: Accepted

 Comments   
Comment by David Nolen [ 09/Jul/17 3:13 PM ]

fixed https://github.com/clojure/clojurescript/commit/2fb7ba9a8d39442ca361b8d2722f3f40e188cf4b





[CLJS-2199] String requires broken after recompile Created: 09/Jul/17  Updated: 09/Jul/17  Resolved: 09/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.854
Fix Version/s: 1.9.854

Type: Defect Priority: Blocker
Reporter: Juho Teperi Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: js-modules, npm-deps

Approval: Vetted

 Description   

String requires uses js-module-exists? to check if module needs to loaded: https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/analyzer.cljc#L722

It checks the compiler state for :js-module-index.

The first time e.g. react-dom/server is not found.

If using the same compiler state for recompile, the second time the module is already present in compiler-state.

This means that the module is not indexed into foreign-libs: https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/closure.clj#L2043

Result is that cljs_deps.js won't have the line for react-dom/sever or No such namespace exception about react-dom$server.



 Comments   
Comment by David Nolen [ 09/Jul/17 12:21 PM ]

fixed https://github.com/clojure/clojurescript/commit/2ade039f9b0af8ae18d46b57c75669114076d4ac





[CLJS-2193] :npm-deps dependencies are implicit Created: 08/Jul/17  Updated: 08/Jul/17  Resolved: 08/Jul/17

Status: Closed
Project: ClojureScript
Component/s: None
Affects Version/s: 1.9.671
Fix Version/s: 1.9.854

Type: Defect Priority: Blocker
Reporter: David Nolen Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: npm-deps

Attachments: Text File CLJS-2193.patch    
Patch: Code
Approval: Accepted

 Description   

Currently the Node dependencies for `:npm-deps` to work are implicit. Looking `cljs/module_deps.js` it's clear that at least `module-deps`, `resolve`, and `browserResolve` must exist. This should probably be handled transparently for the user.



 Comments   
Comment by David Nolen [ 08/Jul/17 9:52 AM ]

On my local machine at least I had to install these manually. I think this should probably be handled for the user?

Comment by David Nolen [ 08/Jul/17 12:09 PM ]

Even after manually installing all the necessary deps if I try to run just the `test-npm-deps` test by itself on master I get the following stacktrace from Node.js:

module.js:471
    throw err;
    ^

Error: Cannot find module 'resolve'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at [eval]:3:19
    at ContextifyScript.Script.runInThisContext (vm.js:25:33)
    at Object.runInThisContext (vm.js:97:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:570:32)
    at evalScript (bootstrap_node.js:353:27)
Comment by David Nolen [ 08/Jul/17 2:08 PM ]

fixed https://github.com/clojure/clojurescript/commit/bd6ff4ff7c772e629c6cb66bf81e7a96577a3099





Generated at Tue May 21 07:38:56 CDT 2019 using JIRA 4.4#649-r158309.