tools.namespace

Reload on a function-by-function basis for improved performance in large projects

Details

  • Type: Enhancement Enhancement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Declined
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:

Description

CURRENT SITUATION

Currently tools.namespace reloads portions of a given Clojure project on a file-by-file (really, namespace-by-namespace) basis. While it works great and performs quite well for most projects — ones which feature namespaces which compile quickly, dependency graphs which don't have too many nodes, and/or dependency graphs which aren't very deep — developers experience a (potentially) significant slowdown the less the project conforms to these three features.

POTENTIAL SOLUTION

A potential solution is this:

  • Walk the forms in all the project's namespaces (significant up-front cost but amortized with incremental reloads)
  • Create a dependency graph of (namespaced, non-local) vars, not just namespaces
  • Detect whether a var's compile-time value (e.g. in the case of a `defn`, its metadata, params, and unevaled body) has changed
  • If so:
  • Reload all the vars which depend on it (according to the dependency graph previously created)
  • Compile it
  • Update the dependency graph
  • Detect whether a var has been added/created
  • If so:
  • Compile it
  • Update the dependency graph

As for tests, it would be nice to re-run only the tests affected by a change to a var, not all the tests in that namespace every time the namespace changes at all.

Essentially, this feature would increase reload granularity (and reload speed) in the same way that namespace-reloading increases granularity (and speed) from recompiling the entire project.

Activity

Hide
Alex Gunnarson added a comment -

Sorry, the nested lists got messed up. Should be:

  • Walk the forms in all the project's namespaces (significant up-front cost but amortized with incremental reloads)
  • Create a dependency graph of (namespaced, non-local) vars, not just namespaces
  • Detect whether a var's compile-time value (e.g. in the case of a `defn`, its metadata, params, and unevaled body) has changed
    • If so:
      • Reload all the vars which depend on it (according to the dependency graph previously created)
      • Compile it
      • Update the dependency graph
  • Detect whether a var has been added/created
    • If so:
      • Compile it
      • Update the dependency graph
Show
Alex Gunnarson added a comment - Sorry, the nested lists got messed up. Should be:
  • Walk the forms in all the project's namespaces (significant up-front cost but amortized with incremental reloads)
  • Create a dependency graph of (namespaced, non-local) vars, not just namespaces
  • Detect whether a var's compile-time value (e.g. in the case of a `defn`, its metadata, params, and unevaled body) has changed
    • If so:
      • Reload all the vars which depend on it (according to the dependency graph previously created)
      • Compile it
      • Update the dependency graph
  • Detect whether a var has been added/created
    • If so:
      • Compile it
      • Update the dependency graph
Hide
Stuart Sierra added a comment -

This is a fascinating idea, one which I would be happy to see developed in the future. However, it is sufficiently different from tools.namespace that it would best be pursued in a completely separate project with a different name.

Show
Stuart Sierra added a comment - This is a fascinating idea, one which I would be happy to see developed in the future. However, it is sufficiently different from tools.namespace that it would best be pursued in a completely separate project with a different name.
Hide
Alex Gunnarson added a comment -

That makes sense. Thanks for your input!

Show
Alex Gunnarson added a comment - That makes sense. Thanks for your input!

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: