[JDBC-56] Problem with find-connection when using agents with c3p0 connection pool

Thu, 16 May 2013 04:50:41 -0500

I use an agent to offload email processing and update a db on completion. If I use a c3p0 connection pool, I get an exception and a complaint that the connection isn't open (see stacktrace). The code works if I wrap the agent fn in a clojure.java.jdbc/with-connection and get a connection from the pool in the agent thread.

The in jdbc.clj, the find-connection (db-find-connection in HEAD) checks for an existing connection and tries to use it without checking if it's open. This causes an exception with a c3p0 connection that was acquired in another thread.

Here's the stacktrace I'm seeing (note the "You can't operate on a closed Connection!!!"):

2013-05-16 10:32:22,208 [clojure-agent-send-off-pool-7] DEBUG com.mchange.v2.sql.SqlUtils - Converting Throwable to SQLException...
java.lang.NullPointerException
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:186)
	at clojure.java.jdbc$prepare_statement.doInvoke(jdbc.clj:450)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at clojure.lang.AFn.applyToHelper(AFn.java:163)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.core$apply.invoke(core.clj:621)
	at clojure.java.jdbc$with_query_results_STAR_.invoke(jdbc.clj:646)
	at clj_record.core$find_by_sql$func__3362__auto____3378.invoke(core.clj:72)
	at clj_record.core$find_by_sql.invoke(core.clj:71)
	at clj_record.core$find_records.invoke(core.clj:85)
	at clj_record.core$find_record.invoke(core.clj:91)
	at neataudio.domain.interview_participant_relationship$find_record.invoke(interview_participant_relationship.clj:18)
	at neataudio.domain.interview_participant_relationship$update_email_status.invoke(interview_participant_relationship.clj:37)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115$fn__5116.invoke(participant_controller.clj:104)
	at clojure.java.jdbc$transaction_STAR_$fn__3162.invoke(jdbc.clj:372)
	at clojure.java.jdbc$transaction_STAR_.invoke(jdbc.clj:371)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113.invoke(participant_controller.clj:101)
	at clojure.lang.AFn.applyToHelper(AFn.java:185)
	at clojure.lang.AFn.applyTo(AFn.java:151)
	at clojure.core$apply.invoke(core.clj:623)
	at clojure.core$binding_conveyor_fn$fn__4115.doInvoke(core.clj:1848)
	at clojure.lang.RestFn.applyTo(RestFn.java:146)
	at clojure.lang.Agent$Action.doRun(Agent.java:114)
	at clojure.lang.Agent$Action.run(Agent.java:163)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:680)
java.sql.SQLException: You can't operate on a closed Connection!!!
	at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106)
	at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:65)
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:222)
	at clojure.java.jdbc$prepare_statement.doInvoke(jdbc.clj:450)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at clojure.lang.AFn.applyToHelper(AFn.java:163)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.core$apply.invoke(core.clj:621)
	at clojure.java.jdbc$with_query_results_STAR_.invoke(jdbc.clj:646)
	at clj_record.core$find_by_sql$func__3362__auto____3378.invoke(core.clj:72)
	at clj_record.core$find_by_sql.invoke(core.clj:71)
	at clj_record.core$find_records.invoke(core.clj:85)
	at clj_record.core$find_record.invoke(core.clj:91)
	at neataudio.domain.interview_participant_relationship$find_record.invoke(interview_participant_relationship.clj:18)
	at neataudio.domain.interview_participant_relationship$update_email_status.invoke(interview_participant_relationship.clj:37)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115$fn__5116.invoke(participant_controller.clj:104)
	at clojure.java.jdbc$transaction_STAR_$fn__3162.invoke(jdbc.clj:372)
	at clojure.java.jdbc$transaction_STAR_.invoke(jdbc.clj:371)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113.invoke(participant_controller.clj:101)
	at clojure.lang.AFn.applyToHelper(AFn.java:185)
	at clojure.lang.AFn.applyTo(AFn.java:151)
	at clojure.core$apply.invoke(core.clj:623)
	at clojure.core$binding_conveyor_fn$fn__4115.doInvoke(core.clj:1848)
	at clojure.lang.RestFn.applyTo(RestFn.java:146)
	at clojure.lang.Agent$Action.doRun(Agent.java:114)
	at clojure.lang.Agent$Action.run(Agent.java:163)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NullPointerException
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:186)
	... 28 more

[JDBC-55] More reflection warnings to investigate

Wed, 15 May 2013 10:34:01 -0500

Reflection warning, clojure/java/jdbc.clj:618:50 - reference to field executeQuery can't be resolved.
Reflection warning, clojure/java/jdbc.clj:618:33 - reference to field close can't be resolved.

[JDBC-46] A mechanism for reading specialized types from result sets could be useful

Wed, 15 May 2013 09:52:03 -0500

I had this idea when trying to adapt Korma to use with postgres tables with columns of array types. I had some success using Korma's transform functions to convert the java.sql.Array objects into a seq on the way out, but recent changes to Korma seem to indicate that the transform functions are not guaranteed to run before the connection is closed. This interferes with efforts to read the arrays, which I think can be their own result sets.

Presumably korma could be changed to support reading the arrays on the way out, but maybe being able to customize the read behavior within c.j.jdbc would be simpler.

The first idea I've had toward this end is to create a protocol with a single function that objects being read out of the resultset get passed through. It would default to the identity function, but users could e.g. extend it to java.sql.Array. I don't know if there are significant performance implications for an extra function call for each value coming out of the database.

I'll have a patch in this vein attached shortly.

[JDBC-53] create-table & drop-table doesn't honor naming strategy

Sun, 12 May 2013 20:56:38 -0500

create-table & drop-table doesn't take the current naming strategy into account.

Clojure JIRA

[JDBC-56] Problem with find-connection when using agents with c3p0 connection pool

[JDBC-55] More reflection warnings to investigate

[JDBC-46] A mechanism for reading specialized types from result sets could be useful

[JDBC-53] create-table & drop-table doesn't honor naming strategy