data.csv

data.csv does not handle BOMs

Details

  • Type: Defect Defect
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Declined
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    Usually Windows (but also Linux)

Description

Sometimes BOMs are prepended to files in Microsoft Land. Data.csv does not handle this edge case, which causes the first field in the header of a csv file to be incorrect. This can be hard to detect, since \ufeff is usually invisible.

http://www.rgagnon.com/javadetails/java-handle-utf8-file-with-bom.html
http://www.fileformat.info/info/unicode/char/feff/index.htm

Activity

Hide
Jonas Enlund added a comment -

This isn't really a csv specific problem. I've encountered files with a byte order mark and then I have simply executed (.skip reader 1) before handing the reader over to read-csv. Is this not a good enough solution?

Show
Jonas Enlund added a comment - This isn't really a csv specific problem. I've encountered files with a byte order mark and then I have simply executed (.skip reader 1) before handing the reader over to read-csv. Is this not a good enough solution?
Hide
Jonas Enlund added a comment -

Instead of adding support for this, I added some docs on how to achieve it without changing data.csv

Show
Jonas Enlund added a comment - Instead of adding support for this, I added some docs on how to achieve it without changing data.csv

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: