tools.namespace

Ability to track files that are not namespaces

Details

  • Type: Enhancement Enhancement
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

Support tracking non-clojure files when performing a refresh. The idea here is to make the reloaded workflow work well for libraries like HugSQL (which may depend on non-clojure files in the project).

See https://github.com/layerware/hugsql/issues/72 for some additional context

Activity

Hide
Stuart Sierra added a comment -

I have been aware of the value of this feature for a long time now, but I expect it would require significant, breaking changes to the internal data structures used by tools.namespace.

Show
Stuart Sierra added a comment - I have been aware of the value of this feature for a long time now, but I expect it would require significant, breaking changes to the internal data structures used by tools.namespace.
Hide
Ray Huang added a comment - - edited

I've been thinking about the internal data structures for a bit, I think it may be possible to track this without breaking the internal data structures. The insight is that we need to determine `modified` clojure source files by taking the max modified timestamp of the clojure src file and any of it's external dependencies.

This would require three changes:

1. a way of defining external file dependencies (perhaps as metadata on the namespaces). Example:

(ns my.testing.ns
  {:clojure.tools.namespace.files/filedeps #{"resources/foo1.properties" 
                                             "resources/foo2.properties"})

2. a new entry in tracker named ::files/filedeps that maps source files -> other files.

3. update the `modified-files` predicate to take in `file-deps`, roughly like so:

(defn- modified? [tracker ^File file]
  (let [filedeps  (get-in tracker [::file/filedeps file])
        time (::time tracker 0)]
    (or (< time (.lastModified file))
        (< time (apply max (map #(.lastModified %) filedeps)))
        (not-every? #(.exists %) filedeps))))

(defn- modified-files [tracker files]
  (filter #(modified? tracker %) files))
Show
Ray Huang added a comment - - edited I've been thinking about the internal data structures for a bit, I think it may be possible to track this without breaking the internal data structures. The insight is that we need to determine `modified` clojure source files by taking the max modified timestamp of the clojure src file and any of it's external dependencies. This would require three changes: 1. a way of defining external file dependencies (perhaps as metadata on the namespaces). Example:
(ns my.testing.ns
  {:clojure.tools.namespace.files/filedeps #{"resources/foo1.properties" 
                                             "resources/foo2.properties"})
2. a new entry in tracker named ::files/filedeps that maps source files -> other files. 3. update the `modified-files` predicate to take in `file-deps`, roughly like so:
(defn- modified? [tracker ^File file]
  (let [filedeps  (get-in tracker [::file/filedeps file])
        time (::time tracker 0)]
    (or (< time (.lastModified file))
        (< time (apply max (map #(.lastModified %) filedeps)))
        (not-every? #(.exists %) filedeps))))

(defn- modified-files [tracker files]
  (filter #(modified? tracker %) files))
Hide
Ray Huang added a comment -

I took a stab at implementing this, would love your thoughts. The docs do need a bit of work to explain how this works though.

Show
Ray Huang added a comment - I took a stab at implementing this, would love your thoughts. The docs do need a bit of work to explain how this works though.
Hide
Ray Huang added a comment -

Following Stuart's advice, I've created a fork with some instructions for how to set it up! https://github.com/rymndhng/tools.namespace

Show
Ray Huang added a comment - Following Stuart's advice, I've created a fork with some instructions for how to set it up! https://github.com/rymndhng/tools.namespace
Hide
Daniel Compton added a comment -

I'm trying this out now and it seems to do just what I want. The only thing I'm noticing is that there is no indication that a filedep has changed in the `:reloading` message, it only shows the namespaces. Not sure how/if you want to show that though.

Show
Daniel Compton added a comment - I'm trying this out now and it seems to do just what I want. The only thing I'm noticing is that there is no indication that a filedep has changed in the `:reloading` message, it only shows the namespaces. Not sure how/if you want to show that though.

People

Vote (2)
Watch (2)

Dates

  • Created:
    Updated: