ClojureScript

cljs.closure/build file locks

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Declined
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    Windows 7, 8, sublime text 3, lein-cljsbuild 1.0.3

Description

I suspect that the following issue of the lein-cljsbuild (https://github.com/emezeske/lein-cljsbuild/issues/322) is caused by clojurescript itself and not the plugin. In short: Is it possible that the cljs.closure/build function keeps file locks on the source files? This would explain why running `lein cljsbuild auto` prevents editors like sublime to save (overwrite) the files.

Activity

Hide
Thomas Heller added a comment -

cljs.closure/build does no file locking. The file is opened, read completely and then closed.

https://github.com/clojure/clojurescript/blob/95122e1206d4a64f392cff03bfd73712ab7f3a33/src/clj/cljs/closure.clj#L256

Show
Thomas Heller added a comment - cljs.closure/build does no file locking. The file is opened, read completely and then closed. https://github.com/clojure/clojurescript/blob/95122e1206d4a64f392cff03bfd73712ab7f3a33/src/clj/cljs/closure.clj#L256
Hide
joerupen added a comment -

Okay, but how is possible that when I comment out the call to the build function, then lein-cljsbuild maintains no locks on the source files? Without the call, lein-cljsbuild picks up any modifications that I do to my source files and invokes the build (which in that case just does nothing, see https://github.com/emezeske/lein-cljsbuild/issues/322).

I opened this issue because it's really disturbing the development workflow when you cannot use `lein cljsbuild auto`. If it's not due to ClojureScript you can close this issue.

Show
joerupen added a comment - Okay, but how is possible that when I comment out the call to the build function, then lein-cljsbuild maintains no locks on the source files? Without the call, lein-cljsbuild picks up any modifications that I do to my source files and invokes the build (which in that case just does nothing, see https://github.com/emezeske/lein-cljsbuild/issues/322). I opened this issue because it's really disturbing the development workflow when you cannot use `lein cljsbuild auto`. If it's not due to ClojureScript you can close this issue.
Hide
David Nolen added a comment -

Never heard this before, sounds possibly like a host issue that we might not be aware of?

Show
David Nolen added a comment - Never heard this before, sounds possibly like a host issue that we might not be aware of?
Hide
joerupen added a comment -

It happens on win7/8 with jdk 7. This problem seems related to sublime 3, as with e.g. lighttable I have no problem. You can see that java.exe holds open file handles on the source files (sysinternals process explorer). Closing the handles manually works but is just a workaround. The reason is that sublime is actually doing a file 'move' operation. When you run `lein cljsbuild auto` and try to overwrite a source file from windows explorer you get the same problem, however you can still open & modify the file in various editors. So it's not a real file lock, but a file lock that prevents a file-move operation. I really don't know if it's ClojureScript or the plugin itself.

Show
joerupen added a comment - It happens on win7/8 with jdk 7. This problem seems related to sublime 3, as with e.g. lighttable I have no problem. You can see that java.exe holds open file handles on the source files (sysinternals process explorer). Closing the handles manually works but is just a workaround. The reason is that sublime is actually doing a file 'move' operation. When you run `lein cljsbuild auto` and try to overwrite a source file from windows explorer you get the same problem, however you can still open & modify the file in various editors. So it's not a real file lock, but a file lock that prevents a file-move operation. I really don't know if it's ClojureScript or the plugin itself.
Hide
Thomas Heller added a comment -

Not sure how lein-cljsbuild handles the file watching, maybe that keeps files open?

Try replacing your (spit ...) test with (Thread/sleep 5000) or something that takes time, since spit is probably "too fast". Is it happening then?

Show
Thomas Heller added a comment - Not sure how lein-cljsbuild handles the file watching, maybe that keeps files open? Try replacing your (spit ...) test with (Thread/sleep 5000) or something that takes time, since spit is probably "too fast". Is it happening then?
Hide
joerupen added a comment -

I just tried to intercept the arguments in the call to cljs.closure/build and then invoked the build function in a repl using the same args. There was no lock this time, but also the generated js files were wrong. There might be some thread bindings in lein-cljsbuild that I am missing, but it looks like the build function by itself does not lock. I also replaced the code with Thread/sleep but the effect is the same. Only when I comment out the build function I can save my file. I also tried to run the build function in an agent, but without success, locks keep existing. I'll keep exploring some other potential causes in lein-cljsbuild and let you know if I find something useful.

Show
joerupen added a comment - I just tried to intercept the arguments in the call to cljs.closure/build and then invoked the build function in a repl using the same args. There was no lock this time, but also the generated js files were wrong. There might be some thread bindings in lein-cljsbuild that I am missing, but it looks like the build function by itself does not lock. I also replaced the code with Thread/sleep but the effect is the same. Only when I comment out the build function I can save my file. I also tried to run the build function in an agent, but without success, locks keep existing. I'll keep exploring some other potential causes in lein-cljsbuild and let you know if I find something useful.
Hide
joerupen added a comment -

By introducing a (Thread/sleep 5000) right after the build function I give the JVM some time to relax right after compiling. The locks seem to disappear (or become very rare). Doesn't that should like a finalizer waiting to be called by the garbage collector? If the polling starts immediatly, the locks occur almost instantly. I know that functions like slurp are pretty safe, so maybe this whole issue is caused by some internals of the JVM itself. Since I don't know all the internals of cljs.closure/build I can't possibly exclude it.

@Thomas: I had a look at the lein-cljsbuild source and from what I understood was that every 100ms,

My conclusion is that lein-cljsbuild does not open any file for reading (at least not in my simple project, no macros, no crossovers, just a simple hello world if you will). It only does directory listings and queries for modified timestamps. Under the hood, querying a file's attributes might very well translate into a call for opening & reading the file and this is done very often. If you have some ideas how I could attempt to locate the root cause, a hint would be appreciated As a workaround I can only recommend to switch to lighttable.

Show
joerupen added a comment - By introducing a (Thread/sleep 5000) right after the build function I give the JVM some time to relax right after compiling. The locks seem to disappear (or become very rare). Doesn't that should like a finalizer waiting to be called by the garbage collector? If the polling starts immediatly, the locks occur almost instantly. I know that functions like slurp are pretty safe, so maybe this whole issue is caused by some internals of the JVM itself. Since I don't know all the internals of cljs.closure/build I can't possibly exclude it. @Thomas: I had a look at the lein-cljsbuild source and from what I understood was that every 100ms, My conclusion is that lein-cljsbuild does not open any file for reading (at least not in my simple project, no macros, no crossovers, just a simple hello world if you will). It only does directory listings and queries for modified timestamps. Under the hood, querying a file's attributes might very well translate into a call for opening & reading the file and this is done very often. If you have some ideas how I could attempt to locate the root cause, a hint would be appreciated As a workaround I can only recommend to switch to lighttable.
Hide
Thomas Heller added a comment -

100ms seems a little overkill to me since no human will make meaningful changes in that time, but that should not matter. A pending finalizer should not be the cause since cljs.closure/build uses with-open which closes the reader when done.

But I use neither Windows, Sublime Text or lein-cljsbuild so I'm just guessing anyways.

If you are feeling adventurous you could try https://github.com/thheller/shadow-build (shameless-plug) but I never tried it on Windows so it might actually blow up or something.

Show
Thomas Heller added a comment - 100ms seems a little overkill to me since no human will make meaningful changes in that time, but that should not matter. A pending finalizer should not be the cause since cljs.closure/build uses with-open which closes the reader when done. But I use neither Windows, Sublime Text or lein-cljsbuild so I'm just guessing anyways. If you are feeling adventurous you could try https://github.com/thheller/shadow-build (shameless-plug) but I never tried it on Windows so it might actually blow up or something.
Hide
joerupen added a comment -

On https://github.com/emezeske/lein-cljsbuild/issues/322 I have posted a workaround. I basically copy the source cljs files to a temp folder and run `cljs.closure/build` from that folder. Interestingly I can always delete that folder after invoking build. This indicates that this issue only occurs in combination with the lein-cljsbuild plugin (due to the watch behavior). As far as the build function is concerned I'd recommend to close this issue in jira.

Show
joerupen added a comment - On https://github.com/emezeske/lein-cljsbuild/issues/322 I have posted a workaround. I basically copy the source cljs files to a temp folder and run `cljs.closure/build` from that folder. Interestingly I can always delete that folder after invoking build. This indicates that this issue only occurs in combination with the lein-cljsbuild plugin (due to the watch behavior). As far as the build function is concerned I'd recommend to close this issue in jira.
Hide
David Nolen added a comment -

Not a ClojureScript thing specifically.

Show
David Nolen added a comment - Not a ClojureScript thing specifically.

People

  • Assignee:
    Unassigned
    Reporter:
    joerupen
Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: