<< Back to previous view

[DXML-35] clojure.data.xml/indent should append a newline after the xml preamble Created: 18/Sep/16  Updated: 18/Sep/16

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Hans Hübner Assignee: Ryan Senior
Resolution: Unresolved Votes: 0
Labels: None
Environment:

clojure.data.xml 0.1.0-beta2



 Description   

Currently, indent and indent-str put the root element directly behind the preamble, without separating the two with a newline. This should be changed so that a newline is inserted.






[DXML-13] Support for preserving whitespace between tags Created: 10/Feb/13  Updated: 03/Oct/16

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Kevin Albrecht Assignee: Ryan Senior
Resolution: Unresolved Votes: 3
Labels: None

Attachments: Text File DXML-13.patch    

 Description   

XML parsers can support preserving white space nodes, but clojure.data.xml does not seem to support this functionality.

For example, the following should be able to return true (perhaps with an option to parse-str):

Desired Behavior
(= (clojure.data.xml/element :x {}
                             (clojure.data.xml/element :a {} "foo")
                             " "
                             (clojure.data.xml/element :a {} "bar"))
   (clojure.data.xml/parse-str
     (str "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
          "<x>"
          "<a>foo</a>"
          " "
          "<a>bar</a>"
          "</x>")))
;=> true

This is the current behavior:

Current Behavior
(= (clojure.data.xml/element :x {}
                             (clojure.data.xml/element :a {} "foo")
                             (clojure.data.xml/element :a {} "bar"))
   (clojure.data.xml/parse-str
     (str "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
          "<x>"
          "<a>foo</a>"
          " "
          "<a>bar</a>"
          "</x>")))
;=> true


 Comments   
Comment by Aron Nopanen [ 18/Aug/13 3:49 PM ]

Seconded.

The issue lies with the '.isWhiteSpace' check in this section of function pull-seq:

XMLStreamConstants/CHARACTERS
(if-let [text (and (not (.isWhiteSpace sreader))
(.getText sreader))]
(cons (event :characters nil nil text)
(pull-seq sreader))
(recur))

While the 'props' argument to parse/parse-str currently only holds XMLInputFactory options, perhaps a ':maintain-whitespace' option could be added that affects this behavior? It would be straightforward to pass the props into pull-seq to conditionally perform the .isWhiteSpace check.

Comment by Aron Nopanen [ 20/Aug/13 12:47 AM ]

I have attached a patch to support a :maintain-whitespace property to parse and parse-str. If set to 'true', whitespace-only nodes will not be stripped during the parsing process.

Comment by Ryan Senior [ 10/Nov/13 10:38 PM ]

Hi Aron,

Thanks for the patch. Have sent in a contributor agreement? I didn't see you name here: http://clojure.org/contributing. Submitting patches to Clojure contrib libraries requires this.

Comment by Jason Gilman [ 08/Apr/14 6:51 AM ]

I'm running into this problem as well. Can this be fixed without using the contributed patch?

Comment by Jan-Paul Bultmann [ 15/Dec/14 5:15 PM ]

Aron Nopanen, the author of the patch is now in the contributors list after signing the agreement.
Is it still applicable or does it have to be adapted due to its age?

Comment by Jan-Paul Bultmann [ 09/Jan/15 8:28 AM ]

It seems that this is already resolved in Herwig's implementation.

So resolving DXML-4 will resolve this issue as well.

Comment by Herwig Hochleitner [ 03/Oct/16 6:09 PM ]

Taking a look. This is still present, after landing DXML-4. I'd be willing to get behind a :preserve-whitespace flag, but we need to take xml:space attributes into consideration. How should those interact with our flag?





[DXML-32] data.xml fails when used with Java 9 Created: 27/May/16  Updated: 22/Nov/16

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Toby Crawley Assignee: Ryan Senior
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Java 9's module system (Jigsaw) appears to prevent loading of some of the XML classes. Java 9 isn't due to be released until March 2017, so there isn't a huge rush to fix this, but I wanted to make you aware. Unfortunately, I don't have enough knowledge of the module system to suggest the proper fix/workaround.

tcrawley@katahdin ~/w/c/data.xml> mvn test
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building data.xml 0.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ data.xml ---
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-clojure-source-dirs) @ data.xml ---
[INFO] Source directory: /home/tcrawley/work/clojure/data.xml/src/main/clojure added.
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-resource (add-clojure-source-dirs) @ data.xml ---
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-test-source (add-clojure-test-source-dirs) @ data.xml ---
[INFO] Test Source directory: /home/tcrawley/work/clojure/data.xml/src/test/clojure added.
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-test-resource (add-clojure-test-source-dirs) @ data.xml ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ data.xml ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/tcrawley/work/clojure/data.xml/src/main/resources
[INFO] Copying 13 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ data.xml ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- clojure-maven-plugin:1.3.13:compile (clojure-compile) @ data.xml ---
Compiling clojure.data.xml.name to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.node to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.jvm.name to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.jvm.emit to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.prxml to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.impl to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.tree to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.jvm.pprint to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.process to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.protocols to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.jvm.parse to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml to /tmp/classes8428993250598039353.dir
Compiling clojure.data.xml.event to /tmp/classes8428993250598039353.dir
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ data.xml ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ data.xml ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ data.xml ---
[INFO] 
[INFO] --- clojure-maven-plugin:1.3.13:test (clojure-test) @ data.xml ---
Exception in thread "main" java.lang.IllegalAccessException: class clojure.lang.Reflector cannot access class com.sun.xml.internal.stream.XMLInputFactoryImpl (in module java.xml) because module java.xml does not export com.sun.xml.internal.stream to unnamed module @1704f67f, compiling:(test_emit.clj:18)
	at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3387)
	at clojure.lang.Compiler$DefExpr.eval(Compiler.java:398)
	at clojure.lang.Compiler.eval(Compiler.java:6516)
	at clojure.lang.Compiler.load(Compiler.java:6952)
	at clojure.lang.RT.loadResourceScript(RT.java:359)
	at clojure.lang.RT.loadResourceScript(RT.java:350)
	at clojure.lang.RT.load(RT.java:429)
	at clojure.lang.RT.load(RT.java:400)
	at clojure.core$load$fn__4890.invoke(core.clj:5415)
	at clojure.core$load.doInvoke(core.clj:5414)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invoke(core.clj:5227)
	at clojure.core$load_lib.doInvoke(core.clj:5264)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invoke(core.clj:603)
	at clojure.core$load_libs.doInvoke(core.clj:5298)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invoke(core.clj:603)
	at clojure.core$require.doInvoke(core.clj:5381)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at user$eval1038.invoke(run-test9074553612640357429.clj:5)
	at clojure.lang.Compiler.eval(Compiler.java:6511)
	at clojure.lang.Compiler.load(Compiler.java:6952)
	at clojure.lang.Compiler.loadFile(Compiler.java:6912)
	at clojure.main$load_script.invoke(main.clj:283)
	at clojure.main$script_opt.invoke(main.clj:343)
	at clojure.main$main.doInvoke(main.clj:427)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:415)
	at clojure.lang.AFn.applyToHelper(AFn.java:161)
	at clojure.lang.Var.applyTo(Var.java:532)
	at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalAccessException: class clojure.lang.Reflector cannot access class com.sun.xml.internal.stream.XMLInputFactoryImpl (in module java.xml) because module java.xml does not export com.sun.xml.internal.stream to unnamed module @1704f67f
	at jdk.internal.reflect.Reflection.throwIllegalAccessException(java.base@9-ea/Reflection.java:405)
	at jdk.internal.reflect.Reflection.throwIllegalAccessException(java.base@9-ea/Reflection.java:396)
	at jdk.internal.reflect.Reflection.ensureMemberAccess(java.base@9-ea/Reflection.java:98)
	at java.lang.reflect.AccessibleObject.slowCheckMemberAccess(java.base@9-ea/AccessibleObject.java:355)
	at java.lang.reflect.AccessibleObject.checkAccess(java.base@9-ea/AccessibleObject.java:347)
	at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:525)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
	at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
	at clojure.data.xml.jvm.parse$make_stream_reader.invoke(parse.clj:116)
	at clojure.data.xml$event_seq.invoke(xml.clj:57)
	at clojure.data.xml$parse.doInvoke(xml.clj:67)
	at clojure.lang.RestFn.invoke(RestFn.java:410)
	at clojure.core$comp$fn__4034.invoke(core.clj:2279)
	at clojure.lang.AFn.applyToHelper(AFn.java:161)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3382)
	... 31 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.133 s
[INFO] Finished at: 2016-05-27T17:48:41-04:00
[INFO] Final Memory: 13M/300M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.3.13:test (clojure-test) on project data.xml: Clojure failed. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
tcrawley@katahdin ~/w/c/data.xml> java -version
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+119)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+119, mixed mode)


 Comments   
Comment by Toby Crawley [ 22/Nov/16 10:02 AM ]

This is caused by http://dev.clojure.org/jira/browse/CLJ-2066

Comment by Herwig Hochleitner [ 22/Nov/16 9:38 PM ]

Unless the reflector is fixed on time for a java 9 release, we'll have to work around this in data.xml by adding appropriate annotations to avoid reflection.





[DXML-27] Provide serialization for XML Schema built-in data types Created: 15/Jun/15  Updated: 24/Nov/16

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Adam Neilson Assignee: Herwig Hochleitner
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Clojure version 1.7
data.xml 0.8
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)



 Description   

data.xml doesn't currently serialize all the data types specified in https://www.w3.org/TR/xmlschema-2/#built-in-datatypes

data.xml should provide automatic serialization for all well-known clojure/jvm types, that have an xmlschema built-in mapping.

Current support

Numbers

ints and floats probably work as expected, ratios need to serialize to decimals.

Dates/Times/Durations

Not currently covered. XML Schema defines duration, dateTime, time and date based on ISO 8601. Thus, clojure's formatting of dates should be reusable. Additionally, the new java.time package should be considered.

URIs/binary data

uris need to be supported, binary data could be serialized as base64Binary or hexBinary

QNames

QNames don't currently support being serialized standalone. It makes a lot of sense, though, to support serializing qnames with the prefix they have in their xml environment.

Was:

Element java.util.Date throws IllegalArgumentException

This seems very similar to DXML-14 but given the lib has been in the wild a while I would have expected this to have been reported already. Can't find a bug report though so submitted here for the record.

I ran this the first time and it seemed fine but all subsequent executions result in this error:

(emit-str (element :something {} #inst "2015-06-15T16:53:18.470-00:00"))
IllegalArgumentException No implementation of method: :gen-event of protocol: #'clojure.data.xml/EventGeneration found for class: java.util.Date  clojure.core/-cache-protocol-fn (core_deftype.clj:544)


 Comments   
Comment by Adam Neilson [ 15/Jun/15 12:18 PM ]

Actually clojure version is 1.6.0

Comment by Herwig Hochleitner [ 24/Nov/16 11:28 PM ]

I was about to decline this ticket due to scope creep, then I had a look at xml schema and I think the built-in datatypes there establish a well-defined boundary of what to expect data.xml to serialize by default.





[DXML-22] Adding hiccup generation function for elements Created: 24/Feb/14  Updated: 28/Mar/14

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Chris Zheng Assignee: Ryan Senior
Resolution: Unresolved Votes: 0
Labels: None
Environment:

N/a



 Description   

This is for completeness really. See pull request https://github.com/clojure/data.xml/pull/10

I would like to:

  • generate an element using hiccup (already exists)
  • generate hiccup using an element (proposed)


 Comments   
Comment by Chris Zheng [ 28/Mar/14 7:22 AM ]

I'm hoping someone can at least give some feedback to this ticket.

Comment by Ryan Senior [ 28/Mar/14 7:53 AM ]

Hi Chris,

Thanks for the reminder on this. I'll have more time to dig in this weekend, but off the top of my head I think more will need to be done on this, both on implementation and on testing. I think what you have now won't work with comments or cdata. One way to flesh some of that out is to create round trip types of tests in src/test/clojure/clojure/data/xml/test_sexp.clj.





[DXML-29] ClojureScript support Created: 13/Nov/15  Updated: 08/Aug/16

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Artem Yarulin Assignee: Herwig Hochleitner
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Parsing XML(and HTML) is very often task and currently it requires a lot of effort and use of different libraries and wrappers around JS libs.

We can dramatically simplify it if we migrate data.xml to ClojureScript.

I'm maintaining koh library which has proof-of-concept XML support for Node and Browsers https://github.com/artemyarulin/koh/blob/master/src/koh/xml.cljs

The general idea is that we can utilise DOMParser object in order to parse string to XML tree. It has a decent browser support https://developer.mozilla.org/en-US/docs/Web/API/DOMParser and has a NodeJS implementation which follows the same API https://github.com/jindw/xmldom

What do you think about this feature?
I can pretty much follow the same guidelines from http://dev.clojure.org/jira/browse/DZIP-5 and implement it in the same way

Thanks



 Comments   
Comment by Herwig Hochleitner [ 04/Aug/16 2:40 PM ]

I'm working on clojurescript support in a separate branch: https://github.com/clojure/data.xml/tree/cljs
It can extend browser-native DOM objects to work with clojure data protocols.
Let me know what you think.

Comment by Artem Yarulin [ 05/Aug/16 4:18 AM ]

Oh, that's super cool! If you need any testing - feel free to ping me, I'll be happy to help

Comment by Herwig Hochleitner [ 08/Aug/16 1:37 PM ]

Thanks for the offer! In fact, the clojurescript support is in a testable state right now and I'm utilizing it for a cljs webdav library: https://github.com/webnf/davstore/blob/master/src/server/webnf/davstore/dav/xml.cljc

The main thing preventing it from being ready is, that I've utilized .cljc and reader tags. As of my last information, our CI doesn't support .cljc yet, for the test matrix and also it's unclear how to target clojure versions < 1.7

I'd be happy to the clojurescript support tested, though. We can still copy+paste into separate clj and cljs files, when we decide to release, if cljc support for contrib projects isn't ready by then.





[DXML-25] Emit Empty Elements using EmptyElementTag Created: 09/Jul/14  Updated: 03/Oct/16

Status: Open
Project: data.xml
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alexander Kiel Assignee: Ryan Senior
Resolution: Unresolved Votes: 0
Labels: enhancement, patch
Environment:

Does not apply


Patch: Code and Test

 Description   

Currently data.xml emits empty elements (elements without content) using start and end tags. The XML spec also allows special empty tags like <foo/>.

I need to serialize XML using such special empty tags because a device, I want to communicate with, does require empty tags. The device is just not able to parse XML messages using start and end tags.

I created a branch on GitHub where I implemented empty tags in the emit function. I'm not familiar how to create a patch. So for now here is the link to the compare view.

As I wrote in my commit message we should discuss, whether a option to the emit function would be a better solution.



 Comments   
Comment by Herwig Hochleitner [ 03/Oct/16 6:42 PM ]

I think, this is a worthwhile change, not because we want to support broken xml receivers, but because some xml dialects lean heavily on lots of attr-only tags for byte savings, even going so far as to encode whole polymorphic data structures in attr string values, as seen in svg.

For that reason, of not wanting to take on complexity arising from broken receivers, I'd vote against an emitter flag.

Would you be willing to sign a contributor's agreement and rebase your changes?





Generated at Sat Dec 03 18:15:36 CST 2016 using JIRA 4.4#649-r158309.