NodeJS support for ClojureScript
Description
Environment
Activity
Herwig Hochleitner January 24, 2019 at 10:19 AM
Awesome!
I've prototyped parsing on node with synchronous and asynchronous streams here: https://github.com/clojure/data.xml/commit/719af453ab2c90352cf72a84f2e42161bf3a8e49
Asynchronous streams can made into a kind of reducible - to - promise event source.
Synchronous streams even allow the same lazy-seq based parser as on JVM.
Both can already be parsed into an element tree.
I'd like to also adapt this for XMLHttpRequest - streams in the browser, update the API for it, and then it can be merged.
I'll ping back here, when that happens.
import January 16, 2019 at 12:12 PM
Comment made by: alza
Hi ,
I'd be happy to help with "end user" testing of the top-level api, in fact I have a real-world app that already uses data.xml in both Clojure and ClojureScript: https://github.com/digital-dj-tools/dj-data-converter
I'd be particularly interested in the "streamline pull <-> push crossover and common architecture", since I'll need to support Clojure (pull) and ClojureScript (push) api's in the same project.
Thanks!
Herwig Hochleitner January 12, 2019 at 7:43 PM
> I was just wondering if this new push approach will allow for efficiently skipping whitespace during the parsing?
Yes, as well as other transformations (e.g. https://dev.clojure.org/jira/browse/DXML-50)
On the sax branch, I've added a protocol called PushHandler (think variadic transducer) and added support for it to most relevant places in the code. You can see it in action here: https://github.com/clojure/data.xml/blob/ac9aa0f711861ee8152ddf18e89a18c1d3538b00/src/main/clojure/clojure/data/xml/js/push.cljs#L184
With this you can can build super-efficient transducer-style xml transformers.
Still there are many bits and pieces to do:
lots of cleanup and documentation and testing
API
streamline pull <-> push crossover and common architecture
port process.clj to this style of processing
Provide lazy pull interface on top of blocking nodejs streams
I'd be glad for any testing or feedback!
import January 11, 2019 at 4:58 PM
Comment made by: alza
Hi ,
I was just wondering if this new push approach will allow for efficiently skipping whitespace during the parsing? Since with the current ClojureScript xmldom implementation, I have to scan through the parsed xml again to remove it, which is yet another step that slows down the parsing overall compared to the Clojure equivalent, which seems to support skipping whitespace during the parsing process with the "skip-whitespace" option (although I couldn't see this mentioned in the docs).
Thanks!
Alex.
Herwig Hochleitner November 25, 2018 at 2:37 PM
I've started work on this here https://github.com/clojure/data.xml/tree/sax
In case anybody wants to take a look and / or help
Details
Assignee
Ryan SeniorRyan SeniorReporter
importimportPriority
Major
Details
Details
Assignee
Ryan SeniorReporter
importPriority
Major
It would be great to add NodeJS support for ClojureScript! It seems data.xml uses DOMParser, which will work in browsers but not in NodeJS.
I looked at the original issue DXML-29, and NodeJS is mentioned there (using xmldom).. but I guess support for NodeJS was never implemented in the end?
I had a look in the code and it seems xmldom is "swapped in" for DOMParser when running the ClojureScript tests, so based on that it should work? Although the tests are running on Nashorn rather than NodeJS. However I couldn't work out where xmldom is coming from.
Possible solutions:
1) Use some logic in the code, e.g. "try create DOMParser, if error try create xmldom".
2) Use xmldom everywhere (apparently it's possible to use it outside NodeJS) but I don't know enough about it. Some info here: https://github.com/jindw/xmldom/wiki/How-to-use-xmldom-in-non-node.js-JavaScript-platforms-like-Rhino-or-SpiderMonkey
Maybe option 1) is preferable.
Unknowns (at least to me): how to "export" the xmldom npm dependency so that it is available for downstream projects using the library..