ClojureScript

Code Size Issue - Constructors

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. The total KLOC of pretty printed advanced compiled code using master is ~3800.

Quick testing shows that the following ctors are present in spectral norm:

PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq

We removed the runtime function dependency of cljs.core.PersistentTreeSet/EMPTY, all ctors related to PTSs disappeared.

Removing the runtime constructions of the other EMPTY collections seems to have no further effect on code size, probably because we have avoided dependencies elsewhere.

Activity

David Nolen made changes -
Field Original Value New Value
Description When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?
When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}
David Nolen made changes -
Description When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}
When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.
David Nolen made changes -
Description When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.
When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns types, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.
David Nolen made changes -
Description When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns types, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.
When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns types, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

A minimal case just tested under advanced compilation:

{code}
/**
* @constructor
*/
Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

bar will be eliminated but *not* Foo!
David Nolen made changes -
Description When compiling something like spectral norm where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns types, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

A minimal case just tested under advanced compilation:

{code}
/**
* @constructor
*/
Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

bar will be eliminated but *not* Foo!
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns types, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

A minimal case just tested under advanced compilation:

{code}
/**
* @constructor
*/
Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

bar will be eliminated but *not* Foo!
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. It's unclear what exactly causes this. After a long series of experiments, I note that removing declares (as they now actually generate code) results in a small decrease in code size.

Using CLJS master if we advance compile spectral norm we get a file that's 77K. If we remove all declares spectral norm advance compiles to 66K. Perhaps there's something similar to declare that causes issues with Closure advanced compilation?

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns types, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

A minimal case just tested under advanced compilation:

{code}
/**
* @constructor
*/
Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

bar will be eliminated but *not* Foo!
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

A minimal case just tested under advanced compilation:

{code}
/**
* @constructor
*/
Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

bar will be eliminated but *not* Foo!
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

A minimal case just tested under advanced compilation:

{code}
/**
* @constructor
*/
Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

bar will be eliminated but *not* Foo!
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

Attempting a minimal case with the Closure Compiler has not revealed any issues, for example something similar to what we output for variadic fns.

{code}
/**
* @constructor
*/
var Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

It really appears as if rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something similar to this when playing around with other optimizations early on.

This explains why StringBufferWriter appears even though spectral norm *never* uses anything along the printing path.

Attempting a minimal case with the Closure Compiler has not revealed any issues, for example something similar to what we output for variadic fns.

{code}
/**
* @constructor
*/
var Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

I suspect rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something strangely similar to this when playing around with other optimizations early on.

However I've been unable to find a minimal case with the Closure Compiler, for example something similar to what we output for variadic fns.

{code}
/**
* @constructor
*/
var Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

All code will get eliminated.
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

I suspect rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something strangely similar to this when playing around with other optimizations early on.

However I've been unable to find a minimal case with the Closure Compiler, for example something similar to what we output for variadic fns.

{code}
/**
* @constructor
*/
var Foo = function(x) {
  this.x = x;
};

var bar = (function() {
  var baz = function() {
    return (new Foo()).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

All code will get eliminated.
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

I suspect rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something strangely similar to this when playing around with other optimizations early on.

However I've been unable to find a minimal case with the Closure Compiler, for example something similar to what we output for variadic fns.

{code}
goog.provide("ns.test");

/**
* @constructor
*/
ns.test.Foo = function(x) {
  this.x = x;
};

ns.test.bar = (function() {
  var baz = function() {
    return (new ns.test.Foo(1)).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

All code will get eliminated.
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove sorted-map & sorted-map-by both which refer to cljs.core.PersistentTreeMap then all ctors related to PTMs disappear and spectral norm then takes only 60K. Both of these are variadic.

I suspect rest fns are at the root of the problem. Anything referred in a variadic fn appears to be necessary to the closure compiler, other fns, ctors, etc. I encountered something strangely similar to this when playing around with other optimizations early on.

However I've been unable to find a minimal case with the Closure Compiler, for example something similar to what we output for variadic fns.

{code}
goog.provide("ns.test");

/**
* @constructor
*/
ns.test.Foo = function(x) {
  this.x = x;
};

ns.test.bar = (function() {
  var baz = function() {
    return (new ns.test.Foo(1)).x
  };
  var woz = function() {
    return "hello";
  };
  woz.baz = baz;
  woz.prop = "hello world";
  return woz;
})();
{code}

All code will get eliminated.
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove the runtime construction of cljs.core.PersistentTreeSet/EMPTY then all ctors related to PTSs disappear and spectral norm compiles down to 60K.

However removing the runtime constructions of the other EMPTY collections seems to have no further effect on code size.
Hide
David Nolen added a comment - - edited

Another issue looks like recursive relationships between constructors, this occurs with hash map upgrading as well as with transient. Getting ridding rid of the IEditableCollection implementations removes all the transient related constructors from appearing. With this change on master we around ~3580 KLOCs.

Coupled with the pr code removed we're down to ~2700 KLOCs. Many constructors magically disappear if we remove the printing code.

After some more investigation, it's only necessary to remove pr-str and that puts us at ~2700 KLOCs.

It appears my speculation about the variadic functions is way off. Recursive relationships between the core types and functions seems to be the real source of the issue.

Show
David Nolen added a comment - - edited Another issue looks like recursive relationships between constructors, this occurs with hash map upgrading as well as with transient. Getting ridding rid of the IEditableCollection implementations removes all the transient related constructors from appearing. With this change on master we around ~3580 KLOCs. Coupled with the pr code removed we're down to ~2700 KLOCs. Many constructors magically disappear if we remove the printing code. After some more investigation, it's only necessary to remove pr-str and that puts us at ~2700 KLOCs. It appears my speculation about the variadic functions is way off. Recursive relationships between the core types and functions seems to be the real source of the issue.
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output.
Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove the runtime construction of cljs.core.PersistentTreeSet/EMPTY then all ctors related to PTSs disappear and spectral norm compiles down to 60K.

However removing the runtime constructions of the other EMPTY collections seems to have no further effect on code size.
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. The total KLOC of pretty printed advanced compiled code using master is ~3800.

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove the runtime construction of cljs.core.PersistentTreeSet/EMPTY then all ctors related to PTSs disappear and spectral norm compiles down to 60K.

However removing the runtime constructions of the other EMPTY collections seems to have no further effect on code size.
Hide
David Nolen added a comment - - edited

There are many complex dependencies between functions and types currently. For example when writing a type it's tempting to rely on the standard library, but this has repercussions. For example calling map from a deftype implementation means you need to pull in the chunked types!

Another example is how PersistentArrayMap and ObjMap both call reduce, in this case reduce doesn't pull in dependencies but it shows a kind of thinking that should probably be avoided in the collections at the heart of ClojureScript. The reduce cases should be probably be replaced with a loop/recur.

Show
David Nolen added a comment - - edited There are many complex dependencies between functions and types currently. For example when writing a type it's tempting to rely on the standard library, but this has repercussions. For example calling map from a deftype implementation means you need to pull in the chunked types! Another example is how PersistentArrayMap and ObjMap both call reduce, in this case reduce doesn't pull in dependencies but it shows a kind of thinking that should probably be avoided in the collections at the heart of ClojureScript. The reduce cases should be probably be replaced with a loop/recur.
David Nolen made changes -
Description When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. The total KLOC of pretty printed advanced compiled code using master is ~3800.

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

If we remove the runtime construction of cljs.core.PersistentTreeSet/EMPTY then all ctors related to PTSs disappear and spectral norm compiles down to 60K.

However removing the runtime constructions of the other EMPTY collections seems to have no further effect on code size.
When compiling something like spectral norm http://github.com/swannodette/cljs-stl/blob/master/src/cljs_stl/spectral/demo.cljs where no ClojureScript data structures are used we still see many constructors in the advanced compiled output. The total KLOC of pretty printed advanced compiled code using master is ~3800.

Quick testing shows that the following ctors are present in spectral norm:

{code}
PersistentTreeMap
RedNode
BlackNode
PersistentTreeMapSeq
TransientHashMap
PersistentHashMap
ArrayNodeSeq
NodeSeq
HashCollisionNode
ArrayNode
BitmapIndexedNode
TransientArrayMap
PersistentArrayMap
ObjMap
NeverEquiv
TransientVector
ChunkedSeq
PersistentVector
VectorNode
ChunkedCons
ArrayChunk
ChunkBuffer
LazySeq
Keyword
Cons
EmptyLIst
List
IndexedSeq
{code}

We removed the runtime function dependency of cljs.core.PersistentTreeSet/EMPTY, all ctors related to PTSs disappeared.

Removing the runtime constructions of the other EMPTY collections seems to have no further effect on code size, probably because we have avoided dependencies elsewhere.
David Nolen made changes -
Comment [ If we remove the pr stuff (all variadic fns), the pr-writer code (non-local implementations), and all the extension of JS natives via the lookup tables we knock off another 1000 lines from the advanced compiled code.

I suspect the variadic fn stuff really does cause problems. Maybe because of the function unwrapping by Closure? Perhaps this happens in a early pass before dead code elimination causing trouble at the dead code elimination phase?

The set! stuff where we set properties on the collections like EMPTY and helper fns don't seem to be the source of any issues which makes things a bit mysterious as to why we see so many unnecessary constructors. ]
Hide
David Nolen added a comment -

Instead of guessing it might be useful to examine the call graph of the AST - there some instructions on how to do so here http://stackoverflow.com/questions/1385335/how-to-generate-function-call-graphs-for-javascript/12220307#12220307

Show
David Nolen added a comment - Instead of guessing it might be useful to examine the call graph of the AST - there some instructions on how to do so here http://stackoverflow.com/questions/1385335/how-to-generate-function-call-graphs-for-javascript/12220307#12220307
Hide
David Nolen added a comment - - edited

One way to control the amount of mutual reference, move that functionality out into a separate function, this way the usage of a specifc function triggers inclusions of data structures instead of it being implicit to using a particular data structure. A specific example is switching data structures based on size i.e. PAM -> PHM.

Show
David Nolen added a comment - - edited One way to control the amount of mutual reference, move that functionality out into a separate function, this way the usage of a specifc function triggers inclusions of data structures instead of it being implicit to using a particular data structure. A specific example is switching data structures based on size i.e. PAM -> PHM.
Hide
David Nolen added a comment -

After attempting to limit the inclusion of transients collections in ClojureScript, Closure is indeed very accurate. The real problem is the reliance on convenience fns within deftypes this includes map/reduce/into/etc. for example map will bring in chunked types, into will bring in all transient collections. deftype should really only be written with the language primitives - this is better from a performance perspective anyhow. We should review the deftypes and eliminate the various conveniences where possible.

Show
David Nolen added a comment - After attempting to limit the inclusion of transients collections in ClojureScript, Closure is indeed very accurate. The real problem is the reliance on convenience fns within deftypes this includes map/reduce/into/etc. for example map will bring in chunked types, into will bring in all transient collections. deftype should really only be written with the language primitives - this is better from a performance perspective anyhow. We should review the deftypes and eliminate the various conveniences where possible.
Hide
David Nolen added a comment -

Other offenders are the top level extend-type/protocol on JS natives. extend-type nil calls hashcode, IPrintWithWriter includes all the printing machinery, string & array load primseq arrayseq.

We should probably adopt the Clojure JVM and dispatch on the types where we can in the library fns themselves at not at the protocol level.

Show
David Nolen added a comment - Other offenders are the top level extend-type/protocol on JS natives. extend-type nil calls hashcode, IPrintWithWriter includes all the printing machinery, string & array load primseq arrayseq. We should probably adopt the Clojure JVM and dispatch on the types where we can in the library fns themselves at not at the protocol level.
Hide
David Nolen added a comment -

This has been addressed in master in a number of commits. (.log js/console "Hello world!") is now ~100 lines of code. There are more enhancements that need further investigation like removing the various transient data structures if unused but this needs more investigation.

Show
David Nolen added a comment - This has been addressed in master in a number of commits. (.log js/console "Hello world!") is now ~100 lines of code. There are more enhancements that need further investigation like removing the various transient data structures if unused but this needs more investigation.
David Nolen made changes -
Resolution Completed [ 1 ]
Status Open [ 1 ] Resolved [ 5 ]
David Nolen made changes -
Status Resolved [ 5 ] Closed [ 6 ]

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: