Clojure

Add transient keyword to cached toString() value in _str

Details

  • Type: Defect Defect
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: Release 1.6
  • Fix Version/s: Release 1.7
  • Component/s: None
  • Labels:
  • Patch:
    Code and Test
  • Approval:
    Screened

Description

_str field in Keyword and Symbol classes lazily caches result of toString(). Because this field is not transient, serializing (using Java serialization) any keyword or symbol before and after calling toString() for the first time yields different results:

(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream  ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

;; keyword example

(def k1 (serialize :k))
(println :k)
(def k2 (serialize :k))

(= k1 k2) ;;=> false 

;; symbol example

(def sym 'a)

(def s1 (serialize sym))
(println sym)
(def s2 (serialize sym))

(= s1 s2) ;;=> false

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key :k and then print :k, I can no longer find such key.

Approach: Add transient keyword to _str field in Keyword and Symbol classes

Patch: CLJ-1408-3.patch

Screened by: Brenton Ashworth

  1. CLJ-1408.patch
    19/Apr/14 4:53 AM
    1 kB
    Tomasz Nurkiewicz
  2. CLJ-1408-2.patch
    09/May/14 3:55 PM
    1 kB
    Tomasz Nurkiewicz
  3. CLJ-1408-3.patch
    22/Aug/14 11:00 PM
    2 kB
    Alex Miller

Activity

Tomasz Nurkiewicz made changes -
Field Original Value New Value
Attachment CLJ-1408.patch [ 12955 ]
Hide
Alex Miller added a comment -

Hi Tomasz, it would be good to fix this, can you sign the CLA?

Show
Alex Miller added a comment - Hi Tomasz, it would be good to fix this, can you sign the CLA?
Hide
Tomasz Nurkiewicz added a comment -

Thanks, I'll sign and send CLA ASAP.

Show
Tomasz Nurkiewicz added a comment - Thanks, I'll sign and send CLA ASAP.
Hide
Tomasz Nurkiewicz added a comment -

My contributor greement arrived, please merge this patch whenever you find suitable.

Show
Tomasz Nurkiewicz added a comment - My contributor greement arrived, please merge this patch whenever you find suitable.
Hide
Alex Miller added a comment -

Hi Tomasz, I noticed you added the private keyword - please remove that and update the patch.

Show
Alex Miller added a comment - Hi Tomasz, I noticed you added the private keyword - please remove that and update the patch.
Alex Miller made changes -
Patch Code [ 10001 ]
Approval Triaged [ 10120 ]
Hide
Tomasz Nurkiewicz added a comment -

Removed `private` keyword

Show
Tomasz Nurkiewicz added a comment - Removed `private` keyword
Tomasz Nurkiewicz made changes -
Attachment CLJ-1408-2.patch [ 12975 ]
Rich Hickey made changes -
Approval Triaged [ 10120 ] Vetted [ 10003 ]
Rich Hickey made changes -
Fix Version/s Release 1.7 [ 10250 ]
Alex Miller made changes -
Approval Vetted [ 10003 ] Screened [ 10004 ]
Description {{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

Attaching patch against current master, will sign CLA if you agree with this issue.
{{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Patch:* CLJ-1408-2.patch

*Screened by:* Alex Miller - no test but not sure that's essential for this. If needed, can turn example here into a test.
Labels interop
Hide
Alex Miller added a comment -

On second thought, it looks like we have most of the infrastructure for serialization testing anyways, so would appreciate an updated patch with the example turned into a serialization test. Please see test/clojure/test_clojure/serialization.clj for a place to put this (using existing roundtrip function).

Show
Alex Miller added a comment - On second thought, it looks like we have most of the infrastructure for serialization testing anyways, so would appreciate an updated patch with the example turned into a serialization test. Please see test/clojure/test_clojure/serialization.clj for a place to put this (using existing roundtrip function).
Alex Miller made changes -
Approval Screened [ 10004 ] Incomplete [ 10006 ]
Description {{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Patch:* CLJ-1408-2.patch

*Screened by:* Alex Miller - no test but not sure that's essential for this. If needed, can turn example here into a test.
{{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Patch:* CLJ-1408-2.patch

*Screened by:*
Hide
Andy Fingerhut added a comment -

Tomasz, in addition to Alex's previous comment, it appears that a commit made to Clojure master earlier today causes your patches to no longer apply cleanly. I haven't looked to see whether updating the patches would be easy, but likely it is.

Show
Andy Fingerhut added a comment - Tomasz, in addition to Alex's previous comment, it appears that a commit made to Clojure master earlier today causes your patches to no longer apply cleanly. I haven't looked to see whether updating the patches would be easy, but likely it is.
Hide
Alex Miller added a comment -

Updated the patch for latest master and added the obvious test.

Show
Alex Miller added a comment - Updated the patch for latest master and added the obvious test.
Alex Miller made changes -
Patch Code [ 10001 ] Code and Test [ 10002 ]
Approval Incomplete [ 10006 ] Vetted [ 10003 ]
Description {{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Patch:* CLJ-1408-2.patch

*Screened by:*
{{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Patch:* CLJ-1408-3.patch

*Screened by:*
Attachment CLJ-1408-3.patch [ 13253 ]
Brenton Ashworth made changes -
Assignee Brenton Ashworth [ brentonashworth ]
Brenton Ashworth made changes -
Description {{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

(def s1 (serialize :k))
;(println :k)
(def s2 (serialize :k))
(println (= s1 s2))
{code}

Uncomment {{(println :k)}} and suddenly {{s1}} and {{s2}} are no longer equal.

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Patch:* CLJ-1408-3.patch

*Screened by:*
{{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword or symbol before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

;; keyword example

(def k1 (serialize :k))
(println :k)
(def k2 (serialize :k))

(= k1 k2) ;;=> false

;; symbol example

(def sym 'a)

(def s1 (serialize sym))
(println sym)
(def s2 (serialize sym))

(= s1 s2) ;;=> false
{code}

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Approach:* Add {{transient}} keyword to {{_str}} field in {{Keyword}} and {{Symbol}} classes

*Patch:* CLJ-1408-3.patch

*Screened by:*
Brenton Ashworth made changes -
Approval Vetted [ 10003 ] Screened [ 10004 ]
Description {{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword or symbol before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

;; keyword example

(def k1 (serialize :k))
(println :k)
(def k2 (serialize :k))

(= k1 k2) ;;=> false

;; symbol example

(def sym 'a)

(def s1 (serialize sym))
(println sym)
(def s2 (serialize sym))

(= s1 s2) ;;=> false
{code}

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Approach:* Add {{transient}} keyword to {{_str}} field in {{Keyword}} and {{Symbol}} classes

*Patch:* CLJ-1408-3.patch

*Screened by:*
{{_str}} field in {{Keyword}} and {{Symbol}} classes lazily caches result of {{toString()}}. Because this field is not {{transient}}, serializing (using Java serialization) any keyword or symbol before and after calling {{toString()}} for the first time yields different results:

{code}
(import (java.io ByteArrayOutputStream ObjectOutputStream
                 ByteArrayInputStream ObjectInputStream))

(defn- serialize [obj]
  (with-open [bos (ByteArrayOutputStream.)
              stream (ObjectOutputStream. bos)]
    (.writeObject stream obj)
    (-> bos .toByteArray seq)))

;; keyword example

(def k1 (serialize :k))
(println :k)
(def k2 (serialize :k))

(= k1 k2) ;;=> false

;; symbol example

(def sym 'a)

(def s1 (serialize sym))
(println sym)
(def s2 (serialize sym))

(= s1 s2) ;;=> false
{code}

This issue came up when I was trying to use keywords as key in [Hazelcast](https://github.com/hazelcast/hazelcast) map. Hazelcast uses serialized keys in various scenarios, thus if I first put something to map under key {{:k}} and then print {{:k}}, I can no longer find such key.

*Approach:* Add {{transient}} keyword to {{_str}} field in {{Keyword}} and {{Symbol}} classes

*Patch:* CLJ-1408-3.patch

*Screened by:* Brenton Ashworth

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated: