<< Back to previous view

[DFRS-3] Lists do not round trip Created: 06/Apr/14  Updated: 29/Jul/14  Resolved: 29/Jul/14

Status: Closed
Project: data.fressian
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Stuart Halloway
Resolution: Declined Votes: 0
Labels: None


 Description   
(fress/read (fress/write '())) -> []

Note: Moved from https://github.com/clojure/data.fressian/issues/3



 Comments   
Comment by Stuart Halloway [ 29/Jul/14 9:51 AM ]

Roundtripping is not an objective. User control, however, is an objective, and the real issue here is Fressian handling of list.

See comments on http://dev.clojure.org/jira/browse/DFRS-1.





[DFRS-1] add clojure.lang.PersistentVector encoding/decoding Created: 03/Dec/13  Updated: 29/Jul/14  Resolved: 29/Jul/14

Status: Closed
Project: data.fressian
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Max Countryman Assignee: Stuart Halloway
Resolution: Declined Votes: 0
Labels: None

Attachments: Text File 0001-add-clojure.lang.PersistentVector-encoding-decoding.patch    
Patch: Code and Test

 Description   

Previously, clojure.lang.PersistentVector was not special-cased on writing
and reading. This resulted in fressian-encoded streams, containing Clojure
vectors, being coerced on reads into java.lang.ArrayList. Coercion here is
a bit surprising and potentially causes unexpected errors.

Here we explicitly mark clojure.lang.PersistentVector instances with the
vec tag on writes and upon reads decode these back to Clojure's vector type
by casting them with a call to vec.



 Comments   
Comment by Stuart Halloway [ 29/Jul/14 9:48 AM ]

The real problem here is that List extensibility is not exposed in Fressian itself. We would need to look into either

  1. user specified handlers where FressianReader calls getHandler
  2. documenting the ConvertList interface

or

  1. making the list extension point more feel like the other collection types for customization
Comment by Stuart Halloway [ 29/Jul/14 9:48 AM ]

Needs Fressian enhancement instead.





[DFRS-4] Better document how to extend with custom readers and writers Created: 05/May/14  Updated: 06/May/14  Resolved: 06/May/14

Status: Resolved
Project: data.fressian
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Completed Votes: 0
Labels: docstring, documentation

Approval: Ok

 Description   

This was requested over email as it was not particularly clear how to integrate custom handlers with the existing handler maps when creating readers and writers.

In particular, it was confusing how to properly create the handler maps using the provided utilities.



 Comments   
Comment by Alex Miller [ 05/May/14 8:38 PM ]

Created wiki page with an example and more info:

https://github.com/clojure/data.fressian/wiki/Creating-custom-handlers

Comment by Alex Miller [ 06/May/14 11:36 AM ]

docstrings updated for create-reader and create-writer.





[DFRS-2] Make writing footer checksums less expensive or optional Created: 17/Dec/13  Updated: 18/Dec/13

Status: Open
Project: data.fressian
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Ghadi Shayban Assignee: Stuart Halloway
Resolution: Unresolved Votes: 0
Labels: None

Approval: Incomplete

 Description   

Problem:
JVM profiler indicates checksums as implemented are a significant bottleneck.

Cause:
impl.RawOutput wraps the provided OutputStream with a CheckedOutputStream. Every time a rawInt is written, CheckedOutputStream calls on its checksum to update itself.

Adler32's update method happens to be native, which may not be germane to the problem.
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/zip/Adler32.java#91

The read side of data.fressian already exposes a knob for checksums to be ignored in RawInput. No such knob exists on the write side.

Checksums are used in the footer methods. They may be extremely useful for data at rest, but may be redundant with other out-of-band mechanisms.

Possible solutions
Buffering so that checksums don't recalculate frequently.
Exposing a knob to control whether write checksums are enabled. This would potentially involve changes with the footer.



 Comments   
Comment by Stuart Halloway [ 18/Dec/13 8:33 AM ]

It is definitely possible that the checksum calculation dings perf. (And if so, another possible solution is just removing checksums entirely from Fressian.)

That said, I don't want to trust a profiler. To move this forward, would like to see a benchmark of a real-world use case without the profiler in play.





Generated at Wed Oct 01 09:32:48 CDT 2014 using JIRA 4.4#649-r158309.