tools.deps

Cannot resolve Maven artifacts through proxy

Details

  • Type: Enhancement Enhancement
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    Clojure 1.9.0-RC2

Description

In environments where the internet is accessed via a proxy, clj fails to resolve (download) Maven artifacts and instead exits with an opaque error message.

$ echo $http_proxy $https_proxy
http://proxy.myorg.com:3128 http://proxy.myorg.com:3128
$ clj
Error building classpath. Failed to read artifact descriptor for org.clojure:test.check:jar:0.9.0

Proxies should be supported out of the box as they are common in corporate environments.

Support for configuring a proxy server could be added in deps.edn, or automatically by inspecting http_proxy, https_proxy etc. environment variables.

Maven settings.xml can also include proxy information, and this is ignored.

  1. maven-proxy-settings.patch
    11/Jul/18 2:50 AM
    6 kB
    David Bürgin
  2. proxy.patch
    05/Jul/18 11:16 AM
    13 kB
    David Bürgin
  3. tdeps-20-3.patch
    15/Dec/18 10:02 AM
    5 kB
    David Bürgin

Activity

Hide
David Bürgin added a comment -

With the recent tools.deps release I’m able to provide a stacktrace.
This is with Java 9.0.4 (the same as with an up-to-date JDK 8).

$ cat deps.edn
{:deps {org.clojure/test.check {:mvn/version "0.9.0"}}}
$ clj
Error building classpath. Failed to read artifact descriptor for org.clojure:test.check:jar:0.9.0
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:test.check:jar:0.9.0
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:282)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:253)
        at clojure.tools.deps.alpha.extensions.maven$eval748$fn__750.invoke(maven.clj:52)
        at clojure.lang.MultiFn.invoke(MultiFn.java:243)
        at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:147)
        at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:129)
        at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:183)
        at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:174)
        at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:59)
        at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:35)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.core$apply.invokeStatic(core.clj:657)
        at clojure.main$main_opt.invokeStatic(main.clj:317)
        at clojure.main$main_opt.invoke(main.clj:313)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.clojure:test.check:pom:0.9.0 from/to central (https://repo1.maven.org/maven2/): Connection refused (Connection refused)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:422)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:224)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:201)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:267)
        ... 20 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.clojure:test.check:pom:0.9.0 from/to central (https://repo1.maven.org/maven2/): Connection refused (Connection refused)
        at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:52)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:365)
        at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:583)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:259)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:498)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:399)
        ... 23 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
        at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
        at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
        at java.base/java.net.Socket.connect(Socket.java:591)
        at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:542)
        at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:414)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:164)
        at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTransporter.java:296)
        at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTransporter.java:252)
        at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(AbstractTransporter.java:67)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:453)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:360)
        ... 28 more
Show
David Bürgin added a comment - With the recent tools.deps release I’m able to provide a stacktrace. This is with Java 9.0.4 (the same as with an up-to-date JDK 8).
$ cat deps.edn
{:deps {org.clojure/test.check {:mvn/version "0.9.0"}}}
$ clj
Error building classpath. Failed to read artifact descriptor for org.clojure:test.check:jar:0.9.0
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:test.check:jar:0.9.0
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:282)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:253)
        at clojure.tools.deps.alpha.extensions.maven$eval748$fn__750.invoke(maven.clj:52)
        at clojure.lang.MultiFn.invoke(MultiFn.java:243)
        at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:147)
        at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:129)
        at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:183)
        at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:174)
        at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:59)
        at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:35)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.core$apply.invokeStatic(core.clj:657)
        at clojure.main$main_opt.invokeStatic(main.clj:317)
        at clojure.main$main_opt.invoke(main.clj:313)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.clojure:test.check:pom:0.9.0 from/to central (https://repo1.maven.org/maven2/): Connection refused (Connection refused)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:422)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:224)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:201)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:267)
        ... 20 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.clojure:test.check:pom:0.9.0 from/to central (https://repo1.maven.org/maven2/): Connection refused (Connection refused)
        at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:52)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:365)
        at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:583)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:259)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:498)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:399)
        ... 23 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
        at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
        at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
        at java.base/java.net.Socket.connect(Socket.java:591)
        at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:542)
        at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:414)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:164)
        at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTransporter.java:296)
        at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTransporter.java:252)
        at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(AbstractTransporter.java:67)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:453)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:360)
        ... 28 more
Hide
David Bürgin added a comment -

No proxies is a bit of a blocker for me. I cannot use the CLI tools on my work machine and there seems to be no workaround.

Attached is a patch to at least get a conversation started. Please provide feedback! I would really like to have something done about this issue.

Notes about the patch:

  • Patch examines well-known environment variables to provide a no-config experience in the best case. deps.edn configuration can be added later.
  • Proxy authentication is not implemented because I have no environment to test it.
  • Implementation: In the Maven Resolver API, a ProxySelector needs to be configured on the RepositorySystemSession, ie it is scoped to the session. RepositorySystem/newResolutionRepositories is then used to augment RemoteRepository instances with the proxy config. Other approaches are possible but this approach seemed most obvious.
  • Manual tests are successful in both proxied and non-proxied environments.

The shell environment I tested this in is as follows (Squid proxy):

http_proxy=http://proxy.myorg.com:3128
https_proxy=http://proxy.myorg.com:3128
no_proxy=localhost,127.0.0.1,.myorg.com,192.168.0.0/16,10.0.0.0/8
Show
David Bürgin added a comment - No proxies is a bit of a blocker for me. I cannot use the CLI tools on my work machine and there seems to be no workaround. Attached is a patch to at least get a conversation started. Please provide feedback! I would really like to have something done about this issue. Notes about the patch:
  • Patch examines well-known environment variables to provide a no-config experience in the best case. deps.edn configuration can be added later.
  • Proxy authentication is not implemented because I have no environment to test it.
  • Implementation: In the Maven Resolver API, a ProxySelector needs to be configured on the RepositorySystemSession, ie it is scoped to the session. RepositorySystem/newResolutionRepositories is then used to augment RemoteRepository instances with the proxy config. Other approaches are possible but this approach seemed most obvious.
  • Manual tests are successful in both proxied and non-proxied environments.
The shell environment I tested this in is as follows (Squid proxy):
http_proxy=http://proxy.myorg.com:3128
https_proxy=http://proxy.myorg.com:3128
no_proxy=localhost,127.0.0.1,.myorg.com,192.168.0.0/16,10.0.0.0/8
Hide
Alex Miller added a comment -

Hey David, I’m out this week but I’ll take a look next week. Thanks for the patch.

Show
Alex Miller added a comment - Hey David, I’m out this week but I’ll take a look next week. Thanks for the patch.
Hide
Alex Miller added a comment -

In general, I would prefer to stick to standard Maven configuration approaches as much as possible (ie use settings.xml, not env vars), similar to what we did for authenticated repos. See: https://maven.apache.org/guides/mini/guide-proxies.html

Also, seems like this patch favors http over https if both exist, but reverse seems better?

And finally, I would also like a doc patch that goes in tandem with this for https://github.com/clojure/clojure-site so that we can update the docs at the same time.

Show
Alex Miller added a comment - In general, I would prefer to stick to standard Maven configuration approaches as much as possible (ie use settings.xml, not env vars), similar to what we did for authenticated repos. See: https://maven.apache.org/guides/mini/guide-proxies.html Also, seems like this patch favors http over https if both exist, but reverse seems better? And finally, I would also like a doc patch that goes in tandem with this for https://github.com/clojure/clojure-site so that we can update the docs at the same time.
Hide
David Bürgin added a comment -

Brief explanation of why the env vars approach: I took this approach to try and provide a ‘no config needed’ experience, using existing well-known env vars. This is how curl, Leiningen, and others get away with no manual proxy setup in the ideal case. (On my work machine, the env vars were set up by infrastructure, not by me, with the expectation that many programs would ‘just work’.)

The patch does not prefer HTTP over HTTPS; the ProxySelector selects the right proxy by looking at the target URL. The "http://" fallback bit is only about connection to the proxy server, not to the target (plain host defaults to HTTP proxy, same as curl).

Thank you for the feedback, Alex. I’ll see if I can prepare a new patch and doc using settings.xml.

Show
David Bürgin added a comment - Brief explanation of why the env vars approach: I took this approach to try and provide a ‘no config needed’ experience, using existing well-known env vars. This is how curl, Leiningen, and others get away with no manual proxy setup in the ideal case. (On my work machine, the env vars were set up by infrastructure, not by me, with the expectation that many programs would ‘just work’.) The patch does not prefer HTTP over HTTPS; the ProxySelector selects the right proxy by looking at the target URL. The "http://" fallback bit is only about connection to the proxy server, not to the target (plain host defaults to HTTP proxy, same as curl). Thank you for the feedback, Alex. I’ll see if I can prepare a new patch and doc using settings.xml.
Hide
David Bürgin added a comment -

New patch maven-proxy-settings.patch uses the settings in ~/.m2/settings.xml. There is a bit of ugliness necessary to support the nonProxyHosts host selection, but overall simpler.

Show
David Bürgin added a comment - New patch maven-proxy-settings.patch uses the settings in ~/.m2/settings.xml. There is a bit of ugliness necessary to support the nonProxyHosts host selection, but overall simpler.
Hide
Joshua Tilles added a comment -

I'm really happy to see progress being made on this issue!

A couple things:

  1. Note that relying on $M2_HOME/settings.xml for specifying a proxy would not affect dependencies resolved via Git. (JGit consults the http_proxy and https_proxy environment variables and the http.proxyHost, http.proxyPort, https.proxyHost, and https.proxyPort system properties.) It seems sensible to me to just call out the difference in documentation, but I wanted to raise the point in case Alex Miller feels otherwise.
  2. The code as-is does not yet support authenticated proxies, but I've tested that a small addition fixes that:
    diff --git a/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj b/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
    index 1e60950..dc0a7b6 100644
    --- a/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
    +++ b/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
    @@ -76,7 +76,11 @@
                   (.. (DefaultProxySelector.)
                     (add (Proxy. (.getProtocol proxy-setting)
                                  (.getHost proxy-setting)
    -                             (.getPort proxy-setting))
    +                             (.getPort proxy-setting)
    +                             (.. (AuthenticationBuilder.)
    +                               (addUsername (.getUsername proxy-setting))
    +                               (addPassword (.getPassword proxy-setting))
    +                               build))
                          (.getNonProxyHosts proxy-setting))
                     (getProxy repo)))))
         first))

    Assuming it's not a bad addition, should the change just be melded into one of David Bürgin's patches? (I'm not feeling picky about attribution/credit.) Or should I create a new patch that assumes David Bürgin's has already been applied? Maybe even create a distinct JIRA issue to address proxy authentication?

Show
Joshua Tilles added a comment - I'm really happy to see progress being made on this issue! A couple things:
  1. Note that relying on $M2_HOME/settings.xml for specifying a proxy would not affect dependencies resolved via Git. (JGit consults the http_proxy and https_proxy environment variables and the http.proxyHost, http.proxyPort, https.proxyHost, and https.proxyPort system properties.) It seems sensible to me to just call out the difference in documentation, but I wanted to raise the point in case Alex Miller feels otherwise.
  2. The code as-is does not yet support authenticated proxies, but I've tested that a small addition fixes that:
    diff --git a/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj b/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
    index 1e60950..dc0a7b6 100644
    --- a/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
    +++ b/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
    @@ -76,7 +76,11 @@
                   (.. (DefaultProxySelector.)
                     (add (Proxy. (.getProtocol proxy-setting)
                                  (.getHost proxy-setting)
    -                             (.getPort proxy-setting))
    +                             (.getPort proxy-setting)
    +                             (.. (AuthenticationBuilder.)
    +                               (addUsername (.getUsername proxy-setting))
    +                               (addPassword (.getPassword proxy-setting))
    +                               build))
                          (.getNonProxyHosts proxy-setting))
                     (getProxy repo)))))
         first))
    Assuming it's not a bad addition, should the change just be melded into one of David Bürgin's patches? (I'm not feeling picky about attribution/credit.) Or should I create a new patch that assumes David Bürgin's has already been applied? Maybe even create a distinct JIRA issue to address proxy authentication?
Hide
Andrea Richiardi added a comment - - edited

Thanks for working on this, it seems very similar to an issue I have had on Travis with OpenJDK 9. See here.

Show
Andrea Richiardi added a comment - - edited Thanks for working on this, it seems very similar to an issue I have had on Travis with OpenJDK 9. See here.
Hide
Alex Miller added a comment -

FYI, I don't think the prior comment has anything to do with this.

Show
Alex Miller added a comment - FYI, I don't think the prior comment has anything to do with this.
Hide
Avi Flax added a comment -

Is there a known workaround/hack that can enable using an HTTP proxy with the current version of tool.deps without having to patch it?

Show
Avi Flax added a comment - Is there a known workaround/hack that can enable using an HTTP proxy with the current version of tool.deps without having to patch it?
Hide
Nolan added a comment -

Hello, this problem is also affecting me. Quite annoying. Most systems I have used respect the well-known environment variables.

Show
Nolan added a comment - Hello, this problem is also affecting me. Quite annoying. Most systems I have used respect the well-known environment variables.
Hide
David Bürgin added a comment -

I have updated the Maven settings patch with Joshua Tilles’s bit that enables proxy authentication (I’ve listed you in a Co-authored-by: trailer in the commit).

Show
David Bürgin added a comment - I have updated the Maven settings patch with Joshua Tilles’s bit that enables proxy authentication (I’ve listed you in a Co-authored-by: trailer in the commit).
Hide
David Bürgin added a comment -

Reviewers: Here are some instructions on how to test patch tdeps-20-3.patch. It’s easy to do, you don’t need an actual proxy environment.

mitmproxy

Use the mitmproxy tool to run a local proxy server. Go to https://mitmproxy.org and download the binary package. On Linux this is just a tarball containing executable binaries.

You can directly start a proxy server in the extracted archive:

./mitmproxy -p 3128

# Or with user:password authentication:
./mitmproxy -p 3128 --proxyauth me:secret

Trust the mitmproxy certificate

Because mitmproxy is, well, a MITM on TLS connections to Maven repos, you must temporarily add its CA certificate to the Java trust store. On Ubuntu 18.04, for example, you can use keytool to import the mitmproxy CA certificate (generated by mitmproxy on first run). Only root may write to the system trust store, so use sudo:

sudo keytool -importcert -trustcacerts -alias mitmproxy -file ~/.mitmproxy/mitmproxy-ca-cert.pem -keystore /etc/ssl/certs/java/cacerts -storepass changeit

After testing, remove the certificate again with the following command:

sudo keytool -delete -alias mitmproxy -keystore /etc/ssl/certs/java/cacerts -storepass changeit

Maven proxy settings

With the proxy set up and running, add a proxies configuration section in the Maven settings.xml file at ~/.m2/settings.xml. (If the file doesn’t exist, create it.)

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">

  <proxies>
    <proxy>
      <id>mitmproxy</id>
      <host>localhost</host>
      <port>3128</port>
      <!-- <username>me</username> -->
      <!-- <password>secret</password> -->
    </proxy>
  </proxies>

</settings>

Uncomment the username/password settings when using the authenticating proxy.

With all of this set up, tools.deps/Clojure tools should resolve artifacts through mitmproxy; you will see the requests flowing through in the mitmproxy UI.

Show
David Bürgin added a comment - Reviewers: Here are some instructions on how to test patch tdeps-20-3.patch. It’s easy to do, you don’t need an actual proxy environment. mitmproxy Use the mitmproxy tool to run a local proxy server. Go to https://mitmproxy.org and download the binary package. On Linux this is just a tarball containing executable binaries. You can directly start a proxy server in the extracted archive:
./mitmproxy -p 3128

# Or with user:password authentication:
./mitmproxy -p 3128 --proxyauth me:secret
Trust the mitmproxy certificate Because mitmproxy is, well, a MITM on TLS connections to Maven repos, you must temporarily add its CA certificate to the Java trust store. On Ubuntu 18.04, for example, you can use keytool to import the mitmproxy CA certificate (generated by mitmproxy on first run). Only root may write to the system trust store, so use sudo:
sudo keytool -importcert -trustcacerts -alias mitmproxy -file ~/.mitmproxy/mitmproxy-ca-cert.pem -keystore /etc/ssl/certs/java/cacerts -storepass changeit
After testing, remove the certificate again with the following command:
sudo keytool -delete -alias mitmproxy -keystore /etc/ssl/certs/java/cacerts -storepass changeit
Maven proxy settings With the proxy set up and running, add a proxies configuration section in the Maven settings.xml file at ~/.m2/settings.xml. (If the file doesn’t exist, create it.)
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">

  <proxies>
    <proxy>
      <id>mitmproxy</id>
      <host>localhost</host>
      <port>3128</port>
      <!-- <username>me</username> -->
      <!-- <password>secret</password> -->
    </proxy>
  </proxies>

</settings>
Uncomment the username/password settings when using the authenticating proxy. With all of this set up, tools.deps/Clojure tools should resolve artifacts through mitmproxy; you will see the requests flowing through in the mitmproxy UI.
Hide
Alex Miller added a comment -

Committed and released in tools.deps.alpha 0.6.496

Show
Alex Miller added a comment - Committed and released in tools.deps.alpha 0.6.496

People

Vote (11)
Watch (8)

Dates

  • Created:
    Updated:
    Resolved: