[CLJS-2738] Can't use aws-amplify or aws-sdk with npm-deps Created: 15/Apr/18  Updated: 09/Aug/18

Status: Open
Project: ClojureScript
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Quang Van Assignee: Unassigned
Resolution: Unresolved Votes: 5
Labels: npm-deps


If I put aws-amplify in :npm-deps it seems to require aws-sdk, but when I put aws-sdk in there, it outputs this error:

Compiling build :dev to "resources/public/js/main.js" from ["src"]...
      throw er; // Unhandled 'error' event

Error: module not found: "fs" from file /home/quang/dev/org-re-frame/node_modules/aws-amplify/node_modules/aws-sdk/lib/util.js
    at onresolve (/home/quang/dev/org-re-frame/node_modules/@cljs-oss/module-deps/index.js:181:30)
    at onResolve (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/Resolver.js:70:11)
    at innerCallback (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/Resolver.js:143:22)
    at callbackWrapper (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)
    at next (/home/quang/dev/org-re-frame/node_modules/tapable/lib/Tapable.js:249:35)
    at innerCallback (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/Resolver.js:143:22)
    at callbackWrapper (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)
    at next (/home/quang/dev/org-re-frame/node_modules/tapable/lib/Tapable.js:249:35)
    at resolver.doResolve.createInnerCallback (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:41:24)
    at callbackWrapper (/home/quang/dev/org-re-frame/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)

Comment by Jeffrey Stanton [ 26/Apr/18 2:51 PM ]

I've also run into the same issue. Here's a simple repro:

{:deps {org.clojure/clojurescript {:mvn/version "1.10.238"}}}
(ns foo (:require ["aws-sdk" :as sdk]))
(defn -main [] (println "hello world"))

then run:

$ clj -m cljs.main -co '{:install-deps true, :npm-deps {:aws-sdk "2.229.1"}}' -m foo
Error: module not found: "fs" from file /Users/jeff/repro/node_modules/aws-sdk/lib/util.js
    at onresolve (/Users/jeff/repro/node_modules/@cljs-oss/module-deps/index.js:181:30)
    at onResolve (/Users/jeff/repro/node_modules/enhanced-resolve/lib/Resolver.js:70:11)
    at innerCallback (/Users/jeff/repro/node_modules/enhanced-resolve/lib/Resolver.js:143:22)
    at callbackWrapper (/Users/jeff/repro/node_modules/enhanced-resolve/lib/createInnerCallback.js:10:21)
Comment by Mike Fikes [ 05/May/18 7:35 PM ]

The big picture is not completely clear to me for this one, but it appears that the NPM version of the library is not intended to be used directly from the browser runtime, and instead you need to build it so that it is packaged for the browser.

The root problem is that the code is attempting to use Node's fs code, but from the browser.

You can see that the code can be compiled and executed if you target Node: If you modify Jeffrey's minimal repro to add

(set! *main-cli-fn* -main)

to the bottom of src/foo.cljs then things will compile and execute in Node:

$ clj -m cljs.main -co '{:install-deps true, :npm-deps {:aws-sdk "2.229.1"}}' -O simple -o main.js -t node -c foo
WARNING: foo is a single segment namespace at line 1 /private/tmp/aws/src/foo.cljs
$ node main.js
hello world

For the browser case, if you look at the code in node_modules/aws-sdk/lib/util.js you can see that it has conditional branches for browser use, but I suspect that this is intended to be used as described here https://github.com/aws/aws-sdk-js#in-the-browser but if you really want to use the code directly from its NPM dependency, you have to use that NPM dependency to build the JavaScript intended to be used in the browser as detailed here: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/building-sdk-for-browsers.html

If this can be confirmed, then this issue can be resolved as a non-issue, and it is just a matter of correct use of the library.

Comment by David Whittington [ 05/May/18 8:23 PM ]

I can confirm that it works in the browser with the AWS recommended browser build process. I was hoping it would also work as a node dep, but I think you're probably right that it's just not intended to be used that way.

Comment by Christian Paul Dehli [ 09/Aug/18 9:55 PM ]

So adding `aws-sdk` the recommended way does work, however there are many AWS packages that pull in `aws-sdk` as a dependency. One example is https://github.com/awslabs/aws-mobile-appsync-sdk-js/tree/master/packages/aws-appsync. This means that these packages won't run in ClojureScript because they throw the same error as above. I have confirmed that in a regular JavaScript project, `aws-sdk` is added correctly (an example is here: https://github.com/aws-samples/aws-mobile-appsync-events-starter-react/).

Like Mike said, it looks to be coming from this line in the code: https://github.com/aws/aws-sdk-js/blob/ebe83921863f1eb020b6a07ef471f2017cd58550/lib/util.js#L39. process.browser is what's indicating to aws-sdk whether or not it's a browser environment.

My guess would be if we set `process.browser` to true for when building to non-node environments it would solve this problem (https://nolanlawson.com/2017/01/09/how-to-write-a-javascript-package-for-both-node-and-the-browser/)

Generated at Wed Apr 24 00:58:25 CDT 2019 using JIRA 4.4#649-r158309.