<< Back to previous view

[TCLI-6] Merge optparse-clj and increase modularity Created: 20/Nov/13  Updated: 10/Dec/13  Resolved: 10/Dec/13

Status: Resolved
Project: tools.cli
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Sung Pae Assignee: Sung Pae
Resolution: Completed Votes: 0
Labels: enhancement


 Description   

On Sun 25 Aug 2013 at 09:05:15PM -0500, gaz jones wrote:

> Hey, i am the current maintainer of tools.cli - i have very little
> time to make any changes to it at the moment (kids ). I'm not sure
> what the process is for adding you as a developer or transferring
> ownership etc but if I'm happy to do so as I have no further plans for
> working on it.

Hello Gareth,

Sorry for delay in action. I submitted my Clojure CA that week and have
been casually awaiting confirmation ever since.

Only today have I noticed that my name has appeared on
http://clojure.org/contributing, so I suppose that is confirmation
enough.

This is my proposal for tools.cli:

  • Merge the CLI arguments lexer from github.com/guns/optparse-clj to
    support GNU option parsing conventions.

Examples:
https://github.com/guns/optparse-clj#features

guns.cli.optparse/tokenize-arguments:
https://github.com/guns/optparse-clj/blob/master/src-cljx/guns/cli/optparse.cljx#L25-74

GNU options parsing spec:
https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html

  • Adapt tools.cli/cli to use the arguments lexer, then freeze it to
    maintain backwards compatibility.
  • Create a new function tools.cli/parse-opts, based largely on the
    design of guns.cli.optparse/parse, that supports the following
    features:
  • Granular options specification map.

Given the following setup:

(ns my.ns
...)

(def cli-options
[["s" "-server HOSTNAME" "Remote server"
:default (java.net.InetAddress/getByName \"example.com\")
:default-desc "example.com"
:parse-fn #(java.net.InetAddress/getByName %)
:assert-fn (partial instance? Inet4Address)
:assert-msg "%s is not an IPv4 host"]
[...]
])

A call to (clojure.tools.cli/compile-option-specs cli-options)
will result in the following PersistentArrayMap:

{option-id ; :server
{:short-opt String ; "-s"
:long-opt String ; "--server"
:required String ; "HOSTNAME"
:desc String ; "Remote server"
:default Object ; #<Inet4Address example.com/93.184.216.119>
:default-desc String ; "example.com"
:parse-fn IFn ; #(InetAddress/getByName %)
:assoc-fn IFn ; assoc
:assert-fn IFn ; (partial instance? Inet4Address)
:assert-msg String ; "%s is not an IPv4 host"
}}

The optspec compiler will verify uniqueness of option-id,
:short-opt, and :long-opt values and throw an AssertionError on
failure.

The optspec map is a PAM to preserve options ordering for summary
generation.

  • Customizable options summary.

tools.cli/parse-opts will return an options summary string to
the caller. Printing the summary with a banner will be the
responsibility of the caller.

The default options summary will look like:

-p, --port NUMBER 8080 Remote port
-s, --server HOSTNAME example.com Remote server
--detach Detach and run in the background
-h, --help

The above format can be changed by supplying an optional :summary-fn
flag that will receive the optspec map values from above and return
a summary string. The default summary-fn will be a public var.

This addresses TCLI-3.

  • Optional in-order options processing, with trailing options parsed
    by default.

This is necessary for managing different option sets for
subcommands. Indirectly addresses TCLI-5.

  • No runtime exceptions.

While parse-opts will throw an AssertionError for duplicate
option-id, :short-opt, and :long-opt values during compilation,
option parsing errors will no longer throw exceptions.

Instead, a map of {option-id error-string} will be provided, or nil
if there are no errors. Correspondingly, parse-opts will have the
following function prototype:

parse-opts:
[argument-seq [& option-vectors] & compiler-flags]
->
{:options PersistentArrayMap ; optspec above
:arguments PersistentVector ; non-optarg arguments
:summary String ; options summary produced by summary-fn
:errors {Keyword String} ; error messages by option
}

The expected usage of this function will look like:

(def usage-banner
"Usage: my-program [options] arg1 arg2\n\nOptions:\n")

(defn exit [status msg]
(println msg)
(System/exit status))

(defn -main [& argv]
(let [{:keys [options arguments summary errors]}
(cli/parse-opts argv cli-options :in-order true)]
(when (:help options)
(exit 0 (str usage-banner summary)))
(when (not= (count arguments) 2)
(exit 1 (str usage-banner summary)))
(when errors
(exit 1 (string/join "\n" (cons "The following errors have occured:\n"
(vals errors)))))
(apply my-program! options arguments)))

  • ClojureScript support.

github.com/guns/optparse-clj currently supports CLJS/node.js via
the cljx preprocessor and an extern file for the Closure Compiler
`process` namespace.

If this is desirable for tools.cli, a similar setup will be applied,
except that I will attempt to avoid cljx for easier hacking.

Comments are appreciated, and I am eager to amend this proposal to gain
community acceptance.

Cheers,
Sung Pae



 Comments   
Comment by Sung Pae [ 20/Nov/13 6:27 PM ]

A more nicely formatted version of the above is available here:

https://gist.github.com/guns/7573819/raw

EDIT:

There is an error in the function prototype above. The actual return value should be:

```
{:options {Keyword Object} ; {:server "my-server.com"}
:arguments PersistentVector ; non-optarg arguments
:summary String ; options summary produced by summary-fn
:errors {Keyword String} ; error messages by option
}
```

Specifically, :options will of course return a PHM of option keywords to their parsed values.

Comment by Gareth Jones [ 20/Nov/13 9:22 PM ]

Sounds good. One thing to add - there is an open issue on github to allow a single dash as an argument (https://github.com/clojure/tools.cli/issues/17) e.g. to indicate you wish to use stdin to read from. This seems to be used as a convention in several *nix cli utilities (such as tar for example). I was intending on adding a patch to support that by simply allowing '-' to be a valid argument name. If you can work that (or something equivalent) into your changes too that would be great!

Thanks,

Gareth

Comment by Sung Pae [ 22/Nov/13 3:57 PM ]

Sorry for the delayed response; I am finding JIRA a bit confusing.

In my experience with options parsers from other languages, '-' as
stdin is typically left to the discretion of the caller as there are
situations where you might want '-' to be a normal argument (e.g. for
describing arithmetic expressions or ranges).

Therefore I believe the proper answer to issue 17 is for the user to
call

`(replace {"-" *in*} rest-args)`

on the remaining arguments vector, and not to bake this behavior into
tools.cli.

I will be happy to make this argument to kyptin on Github if you find
this reasoning persuasive.

Meanwhile, I will wait patiently for someone on the core team to respond
to your post on clojure-dev.

Thank you!

Sung Pae

Comment by Sung Pae [ 10/Dec/13 1:20 PM ]

clojure.tools.cli/parse-opts completes the conceptual merge of optparse-clj.

Generated at Thu Oct 02 03:55:13 CDT 2014 using JIRA 4.4#649-r158309.