http://dev.clojure.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+DCSV+AND+resolution+%3D+Unresolved+ORDER+BY+assignee+ASC%2C+priority+DESC An XML representation of a search request en-us 4.4 649 25-07-2011 http://dev.clojure.org/jira/browse/DCSV-4 data.csv <p>This regards the gray area of being "more forgiving." If I understand RFC 4180 correctly, I want to suggest substituting one bit of forgiveness for another: rather than supporting unquoted, multi-line cell values, I suggest supporting CSVs with just \return as the record-separator. Would you accept a patch for that?</p> <p>A file with \return as record-separator is interpreted by read-csv as a single row like (<span class="error">&#91;&quot;Header1&quot; &quot;Header2\rval1&quot; &quot;val2&quot;&#93;</span>). I believe the RFC only allows fields to contain CR and LF when they're <tt>escaped</tt> (i.e., surrounded in double quotes). See the ABNF at the end of <a href="http://tools.ietf.org/html/rfc4180#section-2">section 2</a>.</p> <p>As far as implementation, I believe this would require wrapping any Reader w/o markSupported in one that does, so that the LF following a CR can be consumed when present.</p> <p><span class="error">&#91;I&#39;ve classified this as a major defect because I ran into a \return-delimited file as soon as I passed a CSV from a Linux machine to a Windows machine, so I&#39;m guessing these files are common. Feel free to reclassify.&#93;</span></p> DCSV-4

\return as record separator with unquoted fields is read as part of the field

Defect Major Open Unresolved Jonas Enlund John Hume Wed, 24 Oct 2012 13:55:57 -0500 Wed, 24 Oct 2012 15:00:49 -0500 0 0 <p>&gt; rather than supporting unquoted, multi-line cell values, I suggest supporting CSVs with just \return as the record-separator. Would you accept a patch for that?</p> <p>Sounds good to me.</p> <p>&gt; As far as implementation, I believe this would require wrapping any Reader w/o markSupported in one that does</p> <p>I think that's ok, since BufferedReader supports it.</p> Global Rank http://dev.clojure.org/jira/browse/DCSV-5 data.csv <p>I imagine a very common use case for parsing CSVs is to get the output as a sequence of maps. I'm happy to provide a patch for this but wanted to make sure I had the right design.</p> <p>My initial idea is to add another option to read-csv with the name :headers which can be a sequence of values, or a flag such as :first-row. Presumably though we ought to also support using the first row as keywords rather than strings, so I'm not sure whether that ought to be another option or a different flag (e.g., :first-row-keywords).</p> DCSV-5

No option for parsing into maps

Enhancement Major Open Unresolved Jonas Enlund Gary Fredericks Tue, 21 May 2013 11:30:25 -0500 Fri, 24 May 2013 12:41:50 -0500 0 0 <p>I've seen this feature request before so I think that something like this should be added. One approach would be to provide a helper function:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">(defn csv-data-&gt;maps [vecs] (map zipmap (repeat (first vecs)) (<span class="code-keyword">rest</span> vecs))) (csv-data-&gt;maps (read-csv reader))</pre> </div></div> <p>I've ran into the same question and prepared a small library to do my csv processing.<br/> It uses data.csv as a workinghorse, but puts some additional functionality on top of it, such as:<br/> 1. csv-to-map: which does the same as the code above, but also maps strings in the first line to keywords. Furthermore, you can choose to translate the keys to lowercase, which is often needed when submitting the csv-data to a database<br/> 2. csv-columnMap: which does a selection of a subset of columns, renaming of these columns (aka renaming the first line of csv-data.<br/> 3. read-csv: my primary entry point using data.csv + csv-to-map + csv-columnMap<br/> 4. read-csv-lazy: A lazy variant which takes a processing function to be used in the inner loop (to allow large csv-datasets) <br/> 5. read-csv-to-db: pumping a csv into a database<br/> 6. map-seq-to-csv: mapping a uniform sequence of hashmaps to a dataset that can be written to a csv (first line contains the keys)</p> <p>Feel free to reuse parts of the code. You can find the code here:</p> <p><a href="https://github.com/cvkem/vinzi.tools/blob/master/vinzi.tools/src/main/clojure/vinzi/tools/vCsv.clj">https://github.com/cvkem/vinzi.tools/blob/master/vinzi.tools/src/main/clojure/vinzi/tools/vCsv.clj</a></p> Global Rank http://dev.clojure.org/jira/browse/DCSV-6 data.csv <p>When whitespace is present after the closing \" the clojure reader crashes with a weird error.<br/> It took me some time to notice it was a white-space issue as whitespace is .... not visible.</p> <p>See an example of the error below.</p> <p>=&gt; (read-csv (java.io.StringReader. "\"a\" " ))<br/> Exception CSV error (unexpected character: ) clojure.data.csv/read-quoted-cell (csv.clj:36)<br/> =&gt; (read-csv (java.io.StringReader. "\"a\"" ))<br/> (<span class="error">&#91;&quot;a&quot;&#93;</span>)</p> DCSV-6

read-csv can not handle white-space at end of line

Defect Major Open Unresolved Jonas Enlund Cees van Kemenade Fri, 24 May 2013 12:50:04 -0500 Fri, 24 May 2013 16:35:40 -0500 0 0 <p>To take the issue a little further, the same holds for whitespace in the middle of a line between the closing-quote and the separator, see:<br/> =&gt; (read-csv (java.io.StringReader. "\"a\" , 5\n \"b,b\",\"6\"" ))<br/> Exception CSV error (unexpected character: ) clojure.data.csv/read-quoted-cell (csv.clj:36)</p> <p>This raises the question what happens if you put a space between the separator and the opening quote (first the default case):<br/> =&gt; (read-csv (java.io.StringReader. "\"a\", 5\n\"b\",\"6\"" ))<br/> (<span class="error">&#91;&quot;a&quot; &quot; 5&quot;&#93;</span> <span class="error">&#91;&quot;b&quot; &quot;6&quot;&#93;</span>)</p> <p>Now adding one additional space:<br/> =&gt; (read-csv (java.io.StringReader. "\"a\", 5\n \"b\",\"6\"" ))<br/> (<span class="error">&#91;&quot;a&quot; &quot; 5&quot;&#93;</span> <span class="error">&#91;&quot; \&quot;b\&quot;&quot; &quot;6&quot;&#93;</span>)</p> <p>Interesting, the white-space is considered to be the start of the string and the quote that follows is considered to be part of the tekst-value that is read.<br/> The main reason for using quotes is to allow separators in text, so let us see that happens if we extend the string by putting a separator in it.<br/> =&gt; (read-csv (java.io.StringReader. "\"a\", 5\n \"b,b\",\"6\"" ))<br/> (<span class="error">&#91;&quot;a&quot; &quot; 5&quot;&#93;</span> <span class="error">&#91;&quot; \&quot;b&quot; &quot;b\&quot;&quot; &quot;6&quot;&#93;</span>)</p> <p>Now we see that the separator is not quoted anymore and as expect, the line is interpreted to contain three values instead of two values.</p> <p>When using standard libraries the issues mentioned above usually do not appear. However, in custom code that emits csv-files or when doing small manual fixes in a csv it is easy to introduce such an issue/error and subsequently it is quit tough to analyse this issue correctly. <br/> Therefore I would opt for a mode of operation where white-space before an opening-quote or after a closing quote are considered to be void (unless it is an escaped quote like "").</p> Global Rank http://dev.clojure.org/jira/browse/DCSV-1 data.csv <p>If you build data.csv alone with the current pom.xml you get a couple of warnings and test are not executed. With the recent versions of Maven, these warnings can break the build.</p> <p>A fixed (I hope!) version is attached.</p> Apache Maven 3.0.3 (r1075438; 2011-02-28 18:31:09+0100) <br/> Maven home: /usr/share/maven <br/> Java version: 1.6.0_29, vendor: Apple Inc. <br/> Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home <br/> Default locale: it_IT, platform encoding: MacRoman <br/> OS name: &quot;mac os x&quot;, version: &quot;10.7.2&quot;, arch: &quot;x86_64&quot;, family: &quot;mac&quot; DCSV-1

pom.xml directives

Enhancement Minor Open Unresolved Jonas Enlund Giorgio Valoti Fri, 10 Feb 2012 04:59:36 -0600 Fri, 10 Feb 2012 04:59:36 -0600 0 0 Global Rank Patch Fixed