ClojureScript

Aliased keywords are still considered invalid tokens by cljs.reader/read-keyword

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

Despite the changes in CLJS-488 aliased keywords (::foo/bar) still don't work (and, in fact, raise an error while compiling cljs -> js)

Specifically, there is a "Invalid token" error raised by cljs.reader/read-keyword when keywords prefixed by "::" are encountered (see reader.cljs#L337)

Sizable portion of that error:

14:46:44.483 [clojure-agent-send-pool-5] INFO io.pedestal.app-tools.compile - {:line 129, :task :compiling, :js-path "hammock_cafe/ui/orders/emitters/status.js"}
14:46:44.483 [clojure-agent-send-pool-5] INFO io.pedestal.app-tools.compile - {:line 129, :task :compiling, :js-path "hammock_cafe/ui/orders/messages.js"}
14:46:45.488 [clojure-agent-send-pool-5] ERROR io.pedestal.app-tools.build - {:line 222}
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Invalid token: ::msg/type
at clojure.lang.LispReader.read(LispReader.java:220) ~[clojure-1.5.0.jar:na]
at clojure.core$read.invoke(core.clj:3407) ~[clojure-1.5.0.jar:na]
at clojure.core$read.invoke(core.clj:3405) ~[clojure-1.5.0.jar:na]
at cljs.compiler$forms_seq$fn__2160.invoke(compiler.clj:721) ~[na:na]
at cljs.compiler$forms_seq.invoke(compiler.clj:721) ~[na:na]
at cljs.compiler$forms_seq$fn__2162.invoke(compiler.clj:722) ~[na:na]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.0.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.0.jar:na]
at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.0.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.0.jar:na]
at cljs.compiler$compile_file_STAR_$fn_2173$fn_2174.invoke(compiler.clj:754) ~[na:na]
at cljs.compiler$compile_file_STAR_$fn__2173.invoke(compiler.clj:751) ~[na:na]
at cljs.compiler$compile_file_STAR_.invoke(compiler.clj:746) ~[na:na]
at cljs.compiler$compile_file.invoke(compiler.clj:794) ~[na:na]
at cljs.closure$compile_file.invoke(closure.clj:356) ~[na:na]
at cljs.closure$eval2598$fn__2599.invoke(closure.clj:398) ~[na:na]
at cljs.closure$eval2523$fn_2524$G2514_2531.invoke(closure.clj:267) ~[na:na]
at io.pedestal.app_tools.compile$build_sources_BANG_$reify_7197$fn_7199.invoke(compile.clj:132) ~[na:na]
...

Activity

Hide
David Nolen added a comment -

We need a bit more information. At the Clojure REPL just entering ::foo/bar into the REPL doesn't work either. ::foo/bar implies that a namespace alias foo exists.

Show
David Nolen added a comment - We need a bit more information. At the Clojure REPL just entering ::foo/bar into the REPL doesn't work either. ::foo/bar implies that a namespace alias foo exists.
Hide
Ryan Neufeld added a comment -

That is correct David, in our particular case we have required io.pedestal.app.messages :as msg and are wanting to use a namespaced keyword like ::msg/topic or ::msg/type. Our current workaround is to (def topic ::topic) inside of io.pedestal.app.messages and use msg/topic in place of ::msg/topic.

Show
Ryan Neufeld added a comment - That is correct David, in our particular case we have required io.pedestal.app.messages :as msg and are wanting to use a namespaced keyword like ::msg/topic or ::msg/type. Our current workaround is to (def topic ::topic) inside of io.pedestal.app.messages and use msg/topic in place of ::msg/topic.
Hide
David Nolen added a comment -

Ok, seems like namespace aliases need to mutate some global that cljs.reader can use?

Show
David Nolen added a comment - Ok, seems like namespace aliases need to mutate some global that cljs.reader can use?
Hide
Ryan Neufeld added a comment -

I couldn't say for certain–my exposure to cljs.reader only came about because of this particular bug. If I had to hazard a guess I would hope it should be as simple as removing the check on L337 that causes a keyword prefixed by "::" to raise an error. You'll know better than I what is entailed.

Show
Ryan Neufeld added a comment - I couldn't say for certain–my exposure to cljs.reader only came about because of this particular bug. If I had to hazard a guess I would hope it should be as simple as removing the check on L337 that causes a keyword prefixed by "::" to raise an error. You'll know better than I what is entailed.
Hide
David Nolen added a comment - - edited

I'm still not following, cljs.reader is the runtime reader, whereas the earlier error you reported was a result of trying to read a aliased namespaced keyword for a namespace that does not exist at compile time. For example (cljs.reader/read-string "::foo/bar") returns ":foo/bar". This of course isn't what you want, thus my suggested solution.

I realized that we never actually added a test case for the fix we came up with at Clojure/West. Trying it at the REPL works great but I do see that something weird happens during compilation. Will look into it.

Show
David Nolen added a comment - - edited I'm still not following, cljs.reader is the runtime reader, whereas the earlier error you reported was a result of trying to read a aliased namespaced keyword for a namespace that does not exist at compile time. For example (cljs.reader/read-string "::foo/bar") returns ":foo/bar". This of course isn't what you want, thus my suggested solution. I realized that we never actually added a test case for the fix we came up with at Clojure/West. Trying it at the REPL works great but I do see that something weird happens during compilation. Will look into it.
Hide
Ryan Neufeld added a comment -

I think I've gotten us out of sync. I can push a worker (or failing, rather) example tomorrow morning to show precisely what is occuring.

In the mean time, for what it is worth, I'll try to be more clear with a short example. Say I have a Pedestal behavior.clj file, and it looks something like this:

(ns io.rkn.my-great-app.behavior
(:require [io.pedestal.app.messages as msg])

;; ...

(defn process-a-msg [msg]
(case (::msg/type msg)
;; ...
))

;; ... Then the rest of the file

Pedestal's io.pedestal.app-tools.compile/build-sources function raises the above included error when it first encounters ::msg/type. Interestingly enough, the resultant .js file is emitted with JavaScript up to the place ::msg/type first occurs, with the remainder of the namespace missing.

I hope this helps, if not, disregard and await a sample tomorrow.

Show
Ryan Neufeld added a comment - I think I've gotten us out of sync. I can push a worker (or failing, rather) example tomorrow morning to show precisely what is occuring. In the mean time, for what it is worth, I'll try to be more clear with a short example. Say I have a Pedestal behavior.clj file, and it looks something like this:
(ns io.rkn.my-great-app.behavior (:require [io.pedestal.app.messages as msg]) ;; ... (defn process-a-msg [msg] (case (::msg/type msg) ;; ... )) ;; ... Then the rest of the file
Pedestal's io.pedestal.app-tools.compile/build-sources function raises the above included error when it first encounters ::msg/type. Interestingly enough, the resultant .js file is emitted with JavaScript up to the place ::msg/type first occurs, with the remainder of the namespace missing. I hope this helps, if not, disregard and await a sample tomorrow.
Hide
David Nolen added a comment -

fixed, http://github.com/clojure/clojurescript/commit/4a04114aedbd1073c4a7c58cee122fcfd0ef1eb4

for the runtime cljs.reader case we probably need another ticket.

Show
David Nolen added a comment - fixed, http://github.com/clojure/clojurescript/commit/4a04114aedbd1073c4a7c58cee122fcfd0ef1eb4 for the runtime cljs.reader case we probably need another ticket.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: