http://dev.clojure.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+JDBC+AND+resolution+%3D+Unresolved+AND+fixVersion+is+EMPTY+ORDER+BY+priority+DESC An XML representation of a search request en-us 4.4 649 25-07-2011 http://dev.clojure.org/jira/browse/JDBC-48 java.jdbc <p>JDBC's <tt>CallableStatement</tt> provides support for calling stored procedures. More specifically, it allows you to register OUT parameters which will become the statements (possibly many) <tt>ResultSet</tt> objects. A <tt>CallableStatement</tt> <em>is a</em> <tt>PreparedStatement</tt>, so I am hoping there wont be too much involved with regard to executing them. The main difference is being able to register and consume OUT parameters.</p> <p>I'll be hacking on this, so patches are forthcoming. Any input is appreciated.</p> JDBC-48

Support stored procedures with CallableStatement

Enhancement Major Open Unresolved Sean Corfield Jeremy Heiler Fri, 15 Mar 2013 13:50:23 -0500 Fri, 15 Mar 2013 22:51:30 -0500 0 1 <p>I've never used stored procs (I don't like the complexity that I've seen them add to version control, change management and deployment) so I'm afraid I can't offer any input - but I really appreciate you taking this on! Thank you!</p> Global Rank http://dev.clojure.org/jira/browse/JDBC-37 java.jdbc <p>The current design of java.jdbc prevents its use as a participant in a distributed XA transaction, where transactional control is delegated to a TransactionManager. It only works with local transactions and absorbs all nested transactions into the outermost one. It'd be nice to have a clean way to override this default behavior.</p> JDBC-37

Provide support for alternate transaction strategies

Enhancement Major Open Unresolved Sean Corfield Jim Crossley Tue, 31 Jul 2012 17:48:54 -0500 Fri, 3 May 2013 12:50:19 -0500 0 0 <p>I'll try to work up a straw-man solution and submit a pull-request.</p> <p>Thanx Jim. I agree the current setup isn't ideal in that area. As for "pull-request", I assume you mean a patch attached to this JIRA ticket (since Clojure and contrib projects cannot accept pull requests). Please also make sure you get a Contributor's Agreement on file per <a href="http://clojure.org/contributing">http://clojure.org/contributing</a></p> <p>Sure thing, Sean</p> <p>Sean, here's a first whack introducing a new dynamic var for transaction strategy. </p> <p>I know the desire is for a new API with explicit parameter passing, and when that vision congeals, I'm happy to help migrate, but I'd like to always have the option of the dynamic binding as well.</p> <p>My thinking is that if a tx strategy function is passed as a parameter, it'll override whatever may be set in the dynamic var, but how it gets passed is still unclear to me. I considered adding an optional key to the db-spec, but wanted to run that by you first.</p> <p>The Agreement is in the mail. I appreciate your feedback.</p> <p>There have been a lot of code changes lately and this patch no longer applies cleanly. Can you submit a new patch against the latest master? Thanx!</p> <p>Sean, I'm not sure I'm totally smitten with the new "transaction?" boolean parameter. At first glance, this seems an awkward way to define a transaction consisting of multiple statements. Can you provide an example usage with the new API of say, inserting, updating and deleting data within a single transaction? I'm hoping an example will clear up my confusion and I can propose a way of parameterizing a particular strategy for executing any transaction.</p> <p>See "Using Transactions" here <a href="https://github.com/clojure/java.jdbc/blob/master/doc/clojure/java/jdbc/UsingSQL.md">https://github.com/clojure/java.jdbc/blob/master/doc/clojure/java/jdbc/UsingSQL.md</a></p> <p>Yes, I saw that, and it seemed to confirm that my original patch should work with minor tweaking. And then I was surprised to see the "transactional?" option in the source. I was curious how you expect it to be used.</p> <p>Folks have asked for the ability to run various functions without an implicit transaction inside them - in fact for some DBs, certain commands cannot be run inside a transaction which was a problem with the old API where you couldn't turn that off. It allows users to have more explicit control over transactions and it's also a convenient "implementation artifact" for nesting calls.</p> <p>So, bottom line: I expect very few users to actually use it explicitly, unless they specifically need to turn off the implicit transaction wrapping.</p> <p>And for most of the API that users will interact with, they don't even need to worry about it.</p> <p>Does that help?</p> <p>Addressing your question about your patch: Clojure/core specifically wanted java.jdbc to move away from dynamically bound variables, which the new API / implementation achieves (given that all the old API that depends on <b>dynamic-vars</b> is deprecated now and will be completely removed before 1.0.0).</p> <p>If all you need is the ability to specify how the transaction function does its job, via a HOF, then I'll have a look at what that would take in the context of the new 'world'...</p> <p>Regarding dynamically bound variables, I think it's very common and accepted &#8211; even canonical? &#8211; to use them (or some ThreadLocal-like variant) to implement transactions. I would hate to make the api awkward just to avoid them.</p> <p>But to answer the core question, yes, I think it's important to provide an alternative to the assumptions encoded into db-transaction*, e.g. "Any nested transactions are absorbed into the outermost transaction." I might prefer a strategy in which a nested transaction suspends the current one and creates another, assuming the driver supports it.</p> <p>But my primary reason for this, as you know, is to somehow inject a "null strategy" to support distributed transactions, delegating the commit/rollback choice to an external "transaction manager".</p> <p>One question: what do you mean by "implementation artifact" for nesting calls?</p> <p>Sean, how do you feel about turning the <tt>:transactional?</tt> option to a function instead of a boolean? And that function represents the <tt>:tx-strategy</tt> used, which could assume the value of a dynamically bound value <tt>&#42;tx-strategy&#42;</tt> by default, and its value would be <tt>db-transaction*</tt> by default. And folks could set it to nil to turn off transactions, i.e. <tt>:tx-strategy nil</tt> or perhaps <tt>:tx-strategy :none</tt> would equate to <tt>:transactional? false</tt>. I think that may satisfy core's recommendation for dynamic variables not being the <b>only</b> way to alter behavior. Make sense at all? </p> <p>I was looking at the code again last night and came to much the same conclusion! I'll take a run at that this weekend (but I'm not adding a <b>dynamic</b> variable - Clojure/core were very clear about their reasons for not wanting those in code except in extremely rare situations in code that is guaranteed to be single-threaded).</p> <p>You're killing me! <img class="emoticon" src="http://dev.clojure.org/jira/images/icons/emoticons/smile.gif" height="20" width="20" align="absmiddle" alt="" border="0"/></p> <p>Without the dynamic var, I can't see any way to transparently allow the db code to participate in a distributed transaction. Can we at least agree that transactional code is guaranteed to be <b>effectively</b> single-threaded? And by this I mean that a transaction must be associated with a single connection, so any thread using that connection must have exclusive access. Do you really want to force folks using distributed transactions to pass the tx strategy in with every call? I don't think adding the dynamic var and the option to override it violates the spirit of this guideline: "If you present an interface that implicitly passes a parameter via dynamic binding (e.g. db in sql), also provide an identical interface but with the parameter passed explicitly."</p> <p>What was the specific feedback you received that contradicts that?</p> <p>Sean, I came up with a different solution for using java.jdbc with an XA connection that obviates this issue. So even though I think it's useful to provide both a dynamic var and a function option as a means to override the logic of <tt>db-transaction*</tt>, I no longer have a need for it.</p> <p>Keep up the good work on java.jdbc!</p> <p>I like problems that go away of their own accord but I still like the idea of making the transaction strategy a function so I'll look at that anyway as a possible (breaking) change for alpha2.</p> <p>Something else you might consider: define a protocol function that encapsulates your <tt>commit/rollback/setAutoCommit</tt> logic inside <tt>db-transaction*</tt> and extend it to <tt>java.sql.Connection</tt>. That way, folks could extend their more specific types, e.g. <tt>XAConnection</tt>, to your protocol (and avoid making those calls that aren't allowed by XA). </p> Global Rank http://dev.clojure.org/jira/browse/JDBC-53 java.jdbc <p>create-table &amp; drop-table doesn't take the current naming strategy into account.</p> JDBC-53

create-table & drop-table doesn't honor naming strategy

Defect Major Open Unresolved Sean Corfield Roman Scherer Wed, 24 Apr 2013 08:47:12 -0500 Sun, 12 May 2013 20:56:38 -0500 1 1 <p>Just to note: the naming strategy stuff, based on dynamically bound globals, is all deprecated in 0.3.0. Alternatives to create-table and drop-table, which respect the new entities function approach, will be provided in an upcoming alpha build of 0.3.0.</p> <p>Any update on this? I noticed the deprecations while reading the code. Could you say a few words or add a few docs on the 0.3 way of doing naming strategies?</p> <p>The replacement is entities and identifiers (as keyword arguments in java.jdbc functions and as macros in java.jdbc.sql). You can see some examples here:</p> <p><a href="http://clojure.github.io/java.jdbc/doc/clojure/java/jdbc/NameMapping.html">http://clojure.github.io/java.jdbc/doc/clojure/java/jdbc/NameMapping.html</a></p> <p>Replacements for create-table / drop-table will be added before beta1. I'm not sure right now how many more alphas we'll have. There's a lot of churn in the code right now and I want it to be feature complete and fairly settled before the first beta.</p> <p>Thanks for the update; looking forward to it!</p> Global Rank http://dev.clojure.org/jira/browse/JDBC-40 java.jdbc <p>Hi, I am using this nice package with a large (government) Teradata machine. I found that I can not insert nil values using <tt>insert-rows</tt>: the Teradata JDBC driver tells me to use <tt>.setNull</tt> or the form of <tt>.setObject</tt> including an SQL type code.</p> <p>The following replacement definition of <tt>set-parameters</tt> was the only change needed to get this to work.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">(defn- set-parameters <span class="code-quote">"Add the parameters to the given statement."</span> [^PreparedStatement stmt params] (let [pmd (.getParameterMetaData stmt)] (dorun (map-indexed (fn [ix value] (let [typ (.getParameterType pmd (inc ix))] (<span class="code-keyword">if</span> (nil? value) (.setNull stmt (inc ix) typ) (.setObject stmt (inc ix) value typ)))) params))))</pre> </div></div> <p>I am not sure whether the use of <tt>.getParameterMetaData</tt> is a generally appropriate because the docs <a href="http://docs.oracle.com/javase/7/docs/api/java/sql/ParameterMetaData.html">http://docs.oracle.com/javase/7/docs/api/java/sql/ParameterMetaData.html</a> say:</p> <blockquote> <p>For some queries and driver implementations, the data that would be returned by a ParameterMetaData object may not be available until the PreparedStatement has been executed.</p></blockquote> <p>Anyway it does work for Teradata. Maybe it could be an option?</p> <p>It just occurred to me that it is probably inefficient to retrieve the parameter metadata on every call to <tt>set-parameters</tt>. It could be done in <tt>insert-rows</tt> instead. On the other hand, I didn't actually notice any performance hit when inserting 40,000 rows.</p> Teradata 14 JDBC-40

insert nulls using .setNull (e.g. for Teradata)

Enhancement Major Reopened Unresolved Sean Corfield Felix Andrews Fri, 12 Oct 2012 06:07:37 -0500 Mon, 22 Apr 2013 04:26:18 -0500 0 2 <p>This was a tricky one. Both .getParameterMetadataData and .getParameterType can throw exceptions. On MS SQL Server, if there is a syntax error in the SQL, the former throws an Exception. On MySQL, the latter throws an exception (every time as far as I can tell, even tho' .getParameterMetaData succeeds). That introduces quite a performance penalty for MySQL (about 10% in my tests).</p> <p>However, if I treated MySQL as a special case and swallowed all exceptions and had a fallback to the original .setObject call, I could get everything to pass the test suite without a noticeable performance overhead (still measuring the latest MySQL performance as I write this).</p> <p>Although it will probably slow everyone down a little, having to make these calls, it should be more robust and may make the .setObject calls faster in some situations?</p> <p>Fixed in 0.3.0-SNAPSHOT. May need to be revisited for performance tweaks later. Uncomfortable with MySQL being a special case here... <img class="emoticon" src="http://dev.clojure.org/jira/images/icons/emoticons/sad.gif" height="20" width="20" align="absmiddle" alt="" border="0"/></p> <p>This fix breaks PostgreSQL and already has MySQL as an exception so I'm going to roll it back and we'll have to find another way to support Teradata I'm afraid!).</p> <p>PostgreSQL also complains about use of setObject with a null object and no parameter type when using protocolVersion 3. </p> <p>Seems to work ok with protocolVersion 2 though.</p> Global Rank Patch Code http://dev.clojure.org/jira/browse/JDBC-1 java.jdbc <p>Shantanu: Provide a mechanism to show the SQL being executed (configurable, so that it can be turned off)</p> <p>Sean: Good idea. Even better, a way to access statistics about the prepared statement after execution - timing etc?</p> <p>Shantanu: Yes, that would be an add-on value to show how are the queries performing.</p> JDBC-1

Provide option to return SQL generated / execution stats

Enhancement Minor Open Unresolved Sean Corfield Sean Corfield Sun, 8 May 2011 21:12:52 -0500 Sun, 8 May 2011 21:12:52 -0500 0 0 Global Rank http://dev.clojure.org/jira/browse/JDBC-56 java.jdbc <p>I use an agent to offload email processing and update a db on completion. If I use a c3p0 connection pool, I get an exception and a complaint that the connection isn't open (see stacktrace). The code works if I wrap the agent fn in a clojure.java.jdbc/with-connection and get a connection from the pool in the agent thread.</p> <p>The in jdbc.clj, the find-connection (db-find-connection in HEAD) checks for an existing connection and tries to use it without checking if it's open. This causes an exception with a c3p0 connection that was acquired in another thread.</p> <p>Here's the stacktrace I'm seeing (note the "You can't operate on a closed Connection!!!"):</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2013-05-16 10:32:22,208 [clojure-agent-send-off-pool-7] DEBUG com.mchange.v2.sql.SqlUtils - Converting Throwable to SQLException... java.lang.NullPointerException at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:186) at clojure.java.jdbc$prepare_statement.doInvoke(jdbc.clj:450) at clojure.lang.RestFn.invoke(RestFn.java:425) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.RestFn.applyTo(RestFn.java:132) at clojure.core$apply.invoke(core.clj:621) at clojure.java.jdbc$with_query_results_STAR_.invoke(jdbc.clj:646) at clj_record.core$find_by_sql$func__3362__auto____3378.invoke(core.clj:72) at clj_record.core$find_by_sql.invoke(core.clj:71) at clj_record.core$find_records.invoke(core.clj:85) at clj_record.core$find_record.invoke(core.clj:91) at neataudio.domain.interview_participant_relationship$find_record.invoke(interview_participant_relationship.clj:18) at neataudio.domain.interview_participant_relationship$update_email_status.invoke(interview_participant_relationship.clj:37) at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115$fn__5116.invoke(participant_controller.clj:104) at clojure.java.jdbc$transaction_STAR_$fn__3162.invoke(jdbc.clj:372) at clojure.java.jdbc$transaction_STAR_.invoke(jdbc.clj:371) at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115.invoke(participant_controller.clj:101) at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114.invoke(participant_controller.clj:101) at neataudio.controllers.participant_controller$notify$fn__5113.invoke(participant_controller.clj:101) at clojure.lang.AFn.applyToHelper(AFn.java:185) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:623) at clojure.core$binding_conveyor_fn$fn__4115.doInvoke(core.clj:1848) at clojure.lang.RestFn.applyTo(RestFn.java:146) at clojure.lang.Agent$Action.doRun(Agent.java:114) at clojure.lang.Agent$Action.run(Agent.java:163) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680) java.sql.SQLException: You can't operate on a closed Connection!!! at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106) at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:65) at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:222) at clojure.java.jdbc$prepare_statement.doInvoke(jdbc.clj:450) at clojure.lang.RestFn.invoke(RestFn.java:425) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.RestFn.applyTo(RestFn.java:132) at clojure.core$apply.invoke(core.clj:621) at clojure.java.jdbc$with_query_results_STAR_.invoke(jdbc.clj:646) at clj_record.core$find_by_sql$func__3362__auto____3378.invoke(core.clj:72) at clj_record.core$find_by_sql.invoke(core.clj:71) at clj_record.core$find_records.invoke(core.clj:85) at clj_record.core$find_record.invoke(core.clj:91) at neataudio.domain.interview_participant_relationship$find_record.invoke(interview_participant_relationship.clj:18) at neataudio.domain.interview_participant_relationship$update_email_status.invoke(interview_participant_relationship.clj:37) at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115$fn__5116.invoke(participant_controller.clj:104) at clojure.java.jdbc$transaction_STAR_$fn__3162.invoke(jdbc.clj:372) at clojure.java.jdbc$transaction_STAR_.invoke(jdbc.clj:371) at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115.invoke(participant_controller.clj:101) at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114.invoke(participant_controller.clj:101) at neataudio.controllers.participant_controller$notify$fn__5113.invoke(participant_controller.clj:101) at clojure.lang.AFn.applyToHelper(AFn.java:185) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:623) at clojure.core$binding_conveyor_fn$fn__4115.doInvoke(core.clj:1848) at clojure.lang.RestFn.applyTo(RestFn.java:146) at clojure.lang.Agent$Action.doRun(Agent.java:114) at clojure.lang.Agent$Action.run(Agent.java:163) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.NullPointerException at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:186) ... 28 more </pre> </div></div> OS X 10.8.3 <br/> Java 1.6.0_45 <br/> Clojure 1.5 <br/> c3p0 0.9.1.2 <br/> java.jdbc 0.2.1 (the problem also seems to be there in HEAD at the time of logging this) JDBC-56

Problem with find-connection when using agents with c3p0 connection pool

Defect Minor Open Unresolved Sean Corfield Ed O'Loughlin Thu, 16 May 2013 04:43:42 -0500 Tue, 21 May 2013 12:36:16 -0500 0 1 <p>Apologies: this is a crappy bug report. I'm under a bit of pressure at the moment so I can't make a test case.</p> <p>If you use the new API, you pass the db-spec into the functions and it will take care of getting the connection, using it, and closing it - and that will all happen on the same (agent) thread. I would not expect you to be able to get a connection in one thread and use it from another thread.</p> <p>When you get a chance, please post an example of both the working and non-working code, but my initial thinking is this isn't a bug, just an inherent limitation of working with connections across threads....?</p> Global Rank