Clojure JIRA

[JDBC-46] A mechanism for reading specialized types from result sets could be useful

Tue, 21 May 2013 21:40:26 -0500

I had this idea when trying to adapt Korma to use with postgres tables with columns of array types. I had some success using Korma's transform functions to convert the java.sql.Array objects into a seq on the way out, but recent changes to Korma seem to indicate that the transform functions are not guaranteed to run before the connection is closed. This interferes with efforts to read the arrays, which I think can be their own result sets.

Presumably korma could be changed to support reading the arrays on the way out, but maybe being able to customize the read behavior within c.j.jdbc would be simpler.

The first idea I've had toward this end is to create a protocol with a single function that objects being read out of the resultset get passed through. It would default to the identity function, but users could e.g. extend it to java.sql.Array. I don't know if there are significant performance implications for an extra function call for each value coming out of the database.

I'll have a patch in this vein attached shortly.

[JDBC-56] Problem with find-connection when using agents with c3p0 connection pool

Tue, 21 May 2013 12:36:16 -0500

I use an agent to offload email processing and update a db on completion. If I use a c3p0 connection pool, I get an exception and a complaint that the connection isn't open (see stacktrace). The code works if I wrap the agent fn in a clojure.java.jdbc/with-connection and get a connection from the pool in the agent thread.

The in jdbc.clj, the find-connection (db-find-connection in HEAD) checks for an existing connection and tries to use it without checking if it's open. This causes an exception with a c3p0 connection that was acquired in another thread.

Here's the stacktrace I'm seeing (note the "You can't operate on a closed Connection!!!"):

2013-05-16 10:32:22,208 [clojure-agent-send-off-pool-7] DEBUG com.mchange.v2.sql.SqlUtils - Converting Throwable to SQLException...
java.lang.NullPointerException
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:186)
	at clojure.java.jdbc$prepare_statement.doInvoke(jdbc.clj:450)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at clojure.lang.AFn.applyToHelper(AFn.java:163)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.core$apply.invoke(core.clj:621)
	at clojure.java.jdbc$with_query_results_STAR_.invoke(jdbc.clj:646)
	at clj_record.core$find_by_sql$func__3362__auto____3378.invoke(core.clj:72)
	at clj_record.core$find_by_sql.invoke(core.clj:71)
	at clj_record.core$find_records.invoke(core.clj:85)
	at clj_record.core$find_record.invoke(core.clj:91)
	at neataudio.domain.interview_participant_relationship$find_record.invoke(interview_participant_relationship.clj:18)
	at neataudio.domain.interview_participant_relationship$update_email_status.invoke(interview_participant_relationship.clj:37)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115$fn__5116.invoke(participant_controller.clj:104)
	at clojure.java.jdbc$transaction_STAR_$fn__3162.invoke(jdbc.clj:372)
	at clojure.java.jdbc$transaction_STAR_.invoke(jdbc.clj:371)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113.invoke(participant_controller.clj:101)
	at clojure.lang.AFn.applyToHelper(AFn.java:185)
	at clojure.lang.AFn.applyTo(AFn.java:151)
	at clojure.core$apply.invoke(core.clj:623)
	at clojure.core$binding_conveyor_fn$fn__4115.doInvoke(core.clj:1848)
	at clojure.lang.RestFn.applyTo(RestFn.java:146)
	at clojure.lang.Agent$Action.doRun(Agent.java:114)
	at clojure.lang.Agent$Action.run(Agent.java:163)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:680)
java.sql.SQLException: You can't operate on a closed Connection!!!
	at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106)
	at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:65)
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:222)
	at clojure.java.jdbc$prepare_statement.doInvoke(jdbc.clj:450)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at clojure.lang.AFn.applyToHelper(AFn.java:163)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.core$apply.invoke(core.clj:621)
	at clojure.java.jdbc$with_query_results_STAR_.invoke(jdbc.clj:646)
	at clj_record.core$find_by_sql$func__3362__auto____3378.invoke(core.clj:72)
	at clj_record.core$find_by_sql.invoke(core.clj:71)
	at clj_record.core$find_records.invoke(core.clj:85)
	at clj_record.core$find_record.invoke(core.clj:91)
	at neataudio.domain.interview_participant_relationship$find_record.invoke(interview_participant_relationship.clj:18)
	at neataudio.domain.interview_participant_relationship$update_email_status.invoke(interview_participant_relationship.clj:37)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115$fn__5116.invoke(participant_controller.clj:104)
	at clojure.java.jdbc$transaction_STAR_$fn__3162.invoke(jdbc.clj:372)
	at clojure.java.jdbc$transaction_STAR_.invoke(jdbc.clj:371)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114$func__3362__auto____5115.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113$fn__5114.invoke(participant_controller.clj:101)
	at neataudio.controllers.participant_controller$notify$fn__5113.invoke(participant_controller.clj:101)
	at clojure.lang.AFn.applyToHelper(AFn.java:185)
	at clojure.lang.AFn.applyTo(AFn.java:151)
	at clojure.core$apply.invoke(core.clj:623)
	at clojure.core$binding_conveyor_fn$fn__4115.doInvoke(core.clj:1848)
	at clojure.lang.RestFn.applyTo(RestFn.java:146)
	at clojure.lang.Agent$Action.doRun(Agent.java:114)
	at clojure.lang.Agent$Action.run(Agent.java:163)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NullPointerException
	at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:186)
	... 28 more

[JDBC-55] More reflection warnings to investigate

Tue, 21 May 2013 10:08:02 -0500

Reflection warning, clojure/java/jdbc.clj:618:50 - reference to field executeQuery can't be resolved.
Reflection warning, clojure/java/jdbc.clj:618:33 - reference to field close can't be resolved.

[JDBC-57] Better support for Oracle

Mon, 20 May 2013 22:52:57 -0500

This is just a suggestion, and i am not sure if it is the right way to do this.

When fetching data from Oracle the oracle jdbc driver returns all the numeric type columns as BigDecimal and date/datetime columns as oracle.sql.TIMESTAMP. This causes a lot of headache as we have to remember to convert the data to the correct type.

Although, this can be handled at the application level by using a :row-fn, it would be really nice if this library handles the conversion itself.

[JDBC-53] create-table & drop-table doesn't honor naming strategy

Sun, 12 May 2013 20:56:38 -0500

create-table & drop-table doesn't take the current naming strategy into account.

[JDBC-13] Add unit tests for generated keys

Tue, 7 May 2013 21:28:53 -0500

Need to test that generated keys come back as expected for various databases - and that DBs which can't generate keys let you use transactions and select identity kind of SQL stuff.

[JDBC-54] db-transaction does not close connection?

Tue, 7 May 2013 21:27:36 -0500

See https://gist.github.com/aphyr/5493964

[JDBC-37] Provide support for alternate transaction strategies

Fri, 3 May 2013 12:50:19 -0500

The current design of java.jdbc prevents its use as a participant in a distributed XA transaction, where transactional control is delegated to a TransactionManager. It only works with local transactions and absorbs all nested transactions into the outermost one. It'd be nice to have a clean way to override this default behavior.

[JDBC-40] insert nulls using .setNull (e.g. for Teradata)

Mon, 22 Apr 2013 04:26:18 -0500

Hi, I am using this nice package with a large (government) Teradata machine. I found that I can not insert nil values using insert-rows: the Teradata JDBC driver tells me to use .setNull or the form of .setObject including an SQL type code.

The following replacement definition of set-parameters was the only change needed to get this to work.

(defn- set-parameters
  "Add the parameters to the given statement."
  [^PreparedStatement stmt params]
  (let [pmd (.getParameterMetaData stmt)]
    (dorun
     (map-indexed
      (fn [ix value]
        (let [typ (.getParameterType pmd (inc ix))]
          (if (nil? value)
            (.setNull stmt (inc ix) typ)
            (.setObject stmt (inc ix) value typ))))
      params))))

I am not sure whether the use of .getParameterMetaData is a generally appropriate because the docs http://docs.oracle.com/javase/7/docs/api/java/sql/ParameterMetaData.html say:

For some queries and driver implementations, the data that would be returned by a ParameterMetaData object may not be available until the PreparedStatement has been executed.

Anyway it does work for Teradata. Maybe it could be an option?

It just occurred to me that it is probably inefficient to retrieve the parameter metadata on every call to set-parameters. It could be done in insert-rows instead. On the other hand, I didn't actually notice any performance hit when inserting 40,000 rows.

[JDBC-51] Reflection warning removal

Fri, 19 Apr 2013 21:52:53 -0500

Before releasing 0.3.0, make a pass over the code and remove reflection where reasonable.

[JDBC-52] Allow execute! to do batch updates via :multi? optional argument

Fri, 19 Apr 2013 21:18:09 -0500

See https://groups.google.com/d/msg/clojure-java-jdbc/H3spAaWgkZI/9WJPXJsHiVsJ for background and justification.

execute! would take an optional :multi? argument, default false, and execute-helper would either call db-do-prepared directly (as it does now) or apply it, assuming (rest sql-params) is a sequence of vectors, which would become multiple params-groups.

I think this would also allow insert! to be simplified and implemented in terms of execute! which would be a nice symmetry.

[JDBC-43] Rewrite old API (with db) in terms of new API

Sun, 7 Apr 2013 14:23:17 -0500

Now that the new API exists, the old API should be rewritten in terms of the new API, passing db and binding it appropriately. Where no new equivalent exists, extend the new API (in particular, the form of insert that just provides rows without column names is not supported in the new API).

[JDBC-41] How to preserve order of rows?

Sun, 7 Apr 2013 03:38:00 -0500

I'd like to get the ordered record set within with-query-results
(or the way to restore the original order).
Is this possible with clojure/java.jdbc?

[JDBC-42] insert! always uses a transaction

Sat, 6 Apr 2013 18:53:54 -0500

execute! delete! and update! all accept an optional :transaction? argument that allows you to control whether the operation occurs in a transaction or not.

[JDBC-45] insert! (and insert) need to support insert of full rows with no column names

Sat, 6 Apr 2013 16:38:43 -0500

In order to fully deprecate insert-values (and to deprecate insert-rows at all), the new insert logic needs to support lack of column names.

[JDBC-49] jdbc insert-records can not handle spaces in columnames

Sat, 6 Apr 2013 16:14:41 -0500

If a database contains column-names that include spaces an sql/insert-record will fail. However, inserting via sql/do-commands or sql/insert-rows works as expected.

Attached you find a file showing the error.

[JDBC-50] insert-rows (incorrectly) patches incomplete records

Sat, 6 Apr 2013 15:38:46 -0500

When using sql/insert-rows with a dataset that contains incomplete rows the rows will be patched with values from the preceding rows. The behaviour to be expected is not documented.

I would expect insert-rows either only accepts complete rows. When accepting incomplete rows I would expect patching with nil-values instead of values from other rows.

Attached you find an example and the output when running in comments.

[JDBC-48] Support stored procedures with CallableStatement

Fri, 15 Mar 2013 22:51:30 -0500

JDBC's CallableStatement provides support for calling stored procedures. More specifically, it allows you to register OUT parameters which will become the statements (possibly many) ResultSet objects. A CallableStatement is a PreparedStatement, so I am hoping there wont be too much involved with regard to executing them. The main difference is being able to register and consume OUT parameters.

I'll be hacking on this, so patches are forthcoming. Any input is appreciated.

[JDBC-47] Transactions do not rollback on non-Exception Throwables

Tue, 26 Feb 2013 20:08:45 -0600

transaction* currently catches Exceptions for its rollback logic. This means Throwables subclassing java.lang.Error are not caught, and indeed the transaction seems to be immediately committed in this case (at least if the connection has autocommit set to true).

This happened to me in the case of an OutOfMemoryError. Another example is that the following code commits:

(transaction
  (insert-records "table" {:foo 12})
  (assert false))

Presumably this can be fixed by catching Throwable instead of Exception. I'll prepare a patch to that effect.

[JDBC-38] There is no way to execute a raw statement outside a transaction

Thu, 27 Dec 2012 07:03:44 -0600

To be able to enforce foreign key constraints in SQLite you need to execute the following statement:

PRAGMA foreign_keys = ON;

However, this statement can't be used from within a transaction. From the SQLite docs:

It is not possible to enable or disable foreign key constraints in the middle of a multi-statement transaction (when SQLite is not in autocommit mode). Attempting to do so does not return an error; it simply has no effect.

In clojure.java.jdbc both do-commands and do-prepared unconditionally run the supplied statements in a transaction. To me it seems like these two functions are the only ones that provide a way to execute an arbitrary SQL statement.

Example using [org.clojure/java.jdbc "0.2.3"] and [org.xerial/sqlite-jdbc "3.7.2"]:

(sql/with-connection {:classname   "org.sqlite.JDBC"
                      :subprotocol "sqlite"
                      :subname     "foo.db"}
  (sql/do-commands "PRAGMA foreign_keys = ON;")
  (sql/with-query-results res ["PRAGMA foreign_keys;"]
    (doall res)))
; => ({:foreign_keys 0})

When using this function instead of do-commands foreign keys work as expected:

(defn do-raw  [& commands]
  (with-open [^java.sql.Statement stmt (let [^java.sql.Connection con (sql/connection)] (.createStatement con))]
    (doseq [^String cmd commands]
      (.addBatch stmt cmd))
    (let [result (.executeBatch stmt)]
      (if (and (= 1 (count result)) (= -2 (first result)))
        (list (.getUpdateCount stmt))
        (seq result)))))

This function is simply do-commands with execute-batch inlined and the wrapping transaction form removed.

[JDBC-44] Add db-do-commands to match new API

Wed, 26 Dec 2012 16:10:17 -0600

New version of do-commands but with db explicitly passed in and transaction? argument - this will satisfy ~~JDBC-38~~.

[JDBC-20] Create new clean API that accepts connection etc (post 0.1.0)

Wed, 26 Dec 2012 15:41:14 -0600

Create streamlined API that relies on no bindings but instead accepts everything as an argument. Refactor existing API methods to operate in terms of this new API as appropriate.

[JDBC-35] Wrong timezone for java.sql.Date, java.sql.Time and java.sql.Timestamp objects returned by with-query-results

Wed, 12 Sep 2012 20:01:37 -0500

Our database stores date and time related information in UTC. But, the results from with-query-results creates objects in the local timezone. This is causing a lot of unexpected behavior when constructing Joda DateTime objects in clj-time. Currently, the with-query-results function is using the single argument getter functions in the ResultSet

getDate(int columnIndex)
getTime(int columnIndex)
getTimestamp(int columnIndex)

We can solve this problem if we can optionally pass the timezone information and call the two argument getter functions.

getDate(int columnIndex, Calendar cal)
getTime(int columnIndex, Calendar cal)
getTimestamp(int columnIndex, Calendar cal)

[JDBC-39] Boolean - BIT(1) - columns are returned as byte[] via some DB connections

Wed, 12 Sep 2012 19:52:05 -0500

The root cause appears to be that rs.getObject( i ) seems to take the field flags into effect over the basic java.sql.Types values.

In this case a BIT(1) column comes back from MySQL as type 16 (BOOLEAN), flags UNSIGNED. From Tungsten it comes back as type 16, flags BINARY BLOB UNSIGNED.

The resultset-seq construction code is pretty cavalier about using just getObject() instead of more specific types so this might be the start of a smarter approach to that.

[JDBC-18] Consider ["name = ? and age = ?" nil 42]

Sun, 12 Aug 2012 21:39:36 -0500

The query is "name = nil and age = 42" where the intent is probably "name is null and age = 42".

It's not clear whether this is really the library's responsibility (to rewrite the SQL) or whether this should be handled by another layer above so this ticket is a placeholder for discussion.

[JDBC-36] using exisiting java based connection pool in clojure

Sun, 12 Aug 2012 21:36:24 -0500

We have a connection pool in Java and would like to use the same for our complete project where code is mixture of Clojure/Java.

(def db (ConnectionManager/getConnection)) ; #<ConnectionHandle com.jolbox.bonecp.ConnectionHandle@53371566> connection type we get after excuting this line
sql/with-connection db
(sql/with-query-results res ["select * from DummyTable where rownum <=100"] (vec res)
(map #(println (class (val %)) " " (key %)) (first res))
)
)

[JDBC-34] get-connection's URI conventions incompatible with PostgreSQL

Tue, 31 Jul 2012 19:46:20 -0500

get-connection requires that provided URIs be structured like so:

dbtype://user:password@host:port/database

This is often sufficient, but many PostgreSQL URIs require the use of URI parameters to further configure connections. For example, postgresql.heroku.com provides JDBC URIs like this:

jdbc:postgresql://ec2-22-11-231-117.compute-1.amazonaws.com:5432/d1kuttup5cbafl6?user=pcgoxvmssqabye&password=PFZXtxaLFhIX-nCA0Vi4UbJ6lH&ssl=true

...which, when used outside of Heroku's network, require a further sslfactory=org.postgresql.ssl.NonValidatingFactory parameter.

The PostgreSQL JDBC driver supports a number of different URI parameters, and recommends putting credentials into parameters rather than using the user:password@ convention. Peeking over at the Oracle thin JDBC driver's docs, it appears that it expects credentials using its own idiosyncratic convention, user/password@.

<opinion>
This all leads me to think that get-connection should pass URIs along to DriverManager without modification, and leave URI format conventions up to the drivers involved. For now, my workaround is to do essentially that, using a map like this as input to with-connection et al.:

{:factory #(DriverManager/getConnection (:url %))
 :url "jdbc:postgresql://ec2-22-11-231-117.compute-1.amazonaws.com:5432/d1kuttup5cbafl6?user=pcgoxvmssqabye&password=PFZXtxaLFhIX-nCA0Vi4UbJ6lH&ssl=true"}

That certainly works, but I presume that such a workaround won't occur to many users, despite the docs/source.
</opinion>

I don't think I've used java.jdbc enough (or RDMBS' enough of late) to comfortably provide a patch (or feel particularly confident in the suggestion above). Hopefully the report is helpful in any case.

[JDBC-33] update-or-insert does not work with oracle

Sun, 10 Jun 2012 18:13:59 -0500

According to the oracle documentation

For a prepared statement batch, it is not possible to know the number of rows affected in the database by each individual statement in the batch. Therefore, all array elements have a value of -2. According to the JDBC 2.0 specification, a value of -2 indicates that the operation was successful but the number of rows affected is unknown.

As documented update-values returns (-2) which means that the (if (zero?)) check in update-or-insert-values will never return true.

Oracle does provide the number of rows updated via an the method getUpdateCount on instances of Statement.

The following change fixes this problem for oracle users, but I am not sure how it will affect other drivers.

(defn oracle-do-prepared
  "Executes an (optionally parameterized) SQL prepared statement on the                                           
  open database connection. Each param-group is a seq of values for all of                                        
  the parameters.                                                                                                 
  Return a seq of update counts (one count for each param-group)."
  [sql & param-groups]
  (with-open [^PreparedStatement stmt (prepare-statement (connection) sql)]
    (if (empty? param-groups)
      (transaction* (ƒ [] (vector (.executeUpdate stmt))))
      (do
        (doseq [param-group param-groups]
          (set-parameters stmt param-group)
          (.addBatch stmt))
        (transaction* (ƒ [] (let [result (.executeBatch stmt)
                                  count  (.getUpdateCount stmt)]
                              (vector count))))))))

[JDBC-32] with-connection does not accept standard jdbc URIs.

Sun, 10 Jun 2012 17:48:51 -0500

with-connect and other functions accepting a db-spec do not support standard jdbc uris, like this one:

jdbc:mysql://localhost:3306/<database>

In my environment passing that uri to yields an exception:

Exception in thread "main" java.lang.NullPointerException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at clojure.lang.RT.loadClassForName(RT.java:2050)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:175)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:166)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:168)
at clojure.java.jdbc$with_connection_STAR_.invoke(jdbc.clj:299)
at bungle_o.migration$create_exceptions.invoke(migration.clj:17)
at bungle_o.migration$_main.invoke(migration.clj:21)
at clojure.lang.Var.invoke(Var.java:411)
at user$eval191.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:6511)
at clojure.lang.Compiler.eval(Compiler.java:6501)
at clojure.lang.Compiler.eval(Compiler.java:6477)
at clojure.core$eval.invoke(core.clj:2797)
at clojure.main$eval_opt.invoke(main.clj:297)
at clojure.main$initialize.invoke(main.clj:316)
at clojure.main$null_opt.invoke(main.clj:349)
at clojure.main$main.doInvoke(main.clj:427)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)

This URI works fine, however:

mysql://localhost:3306/<database>

Is this intentional or a bug?

Here's the mysql documentation for reference: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-configuration-properties.html

[JDBC-31] distinct? throws clojure.lang.ArityException, when applied with no arguments

Sun, 10 Jun 2012 17:24:42 -0500

HSQLDB returns an empty ResultSet when using (.getGeneratedKeys stmt)
and no keys are generated. So this Exception is thrown for each record
without generated keys.

While this Exception is caught in do-prepared-return-keys, this can lead to a huge overhead caused by the JVM exception handling. I did a performance test.

Before Patch:

clojure.java.test-jdbc> (time (sql/with-connection hsqldb-db (count (apply sql/insert-records :dummy  (map #(hash-map :name (str %) :id %) (range 10000))))))
"Elapsed time: 3429.346743 msecs"
10000

After Patch:

 clojure.java.test-jdbc> (time (sql/with-connection hsqldb-db (count (apply sql/insert-records :dummy  (map #(hash-map :name (str %) :id %) (range 10000))))))
"Elapsed time: 1397.444753 msecs"

[JDBC-30] Consider to make do-prepared-return-keys public

Thu, 10 May 2012 14:12:16 -0500

Libraries like Korma, that build upon clojure.java.jdbc depend
on, or have use cases for do-prepared-return-keys. It's a useful
function and we would like to use it. As discussed over here:

https://github.com/ibdknox/Korma/pull/60#issuecomment-5630604

[JDBC-29] Performance improvement (Remove intermediate lazy sequence in resultset-seq)

Thu, 10 May 2012 12:09:00 -0500

This improves performance on large result sets by up to 30%.

[JDBC-26] Add sqlite3 support

Mon, 23 Apr 2012 03:35:50 -0500

sqlite3 has some problems:

1) PreparedStatements.addBatch does not do anything without some parameters, so executeBatch doesn't do anything.
2) Transaction support appears to require closing the ResultSet object from generatedKeys.

I've only tested the included patch with the default test dbs and sqlite3.

[JDBC-19] Consolidate jdbc / jdbc.internal namespaces (post 0.1.0)

Mon, 23 Apr 2012 03:22:42 -0500

Since clojure.java.jdbc.internal is not intended to be called externally, merge it into clojure.java.jdbc and make its members private. Then merge simplify functions where appropriate.

[JDBC-28] prepared-statement* behaves different on OpenJDK 1.6 and OS X Java 1.6.

Sat, 31 Mar 2012 01:30:25 -0500

I'm using the Clojure library Korma to generate SQL for Postgres, which depends on clojure.java.jdbc. Recently I added a custom 'upsert' query. Using it on normal java 1.6 is fine, but on openJDK it breaks due to the use of 'java.sql.Statement/RETURN_GENERATED_KEYS'. This makes the Postgres driver add a seemingly random "RETURNING *" at the end of the query.

On normal Java 1.6 java.sql.Statement/RETURN_GENERATED_KEYS doesn't exist and then a exception is caught and everything works as expected. Sounds like the world up side down, but I'm probably missing the rational behind this.

The line that causes the weird behavior is here: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/internal.clj#L315

The code that showcases the 'bug' is here:
https://gist.github.com/2e8a3d55d80707ce79e0

(Sorry for the monkey patching in the gist, if there is a better way I would love to hear it)

I guess a naive patch would be to remove the java.sql.Statement/RETURN_GENERATED_KEYS line, but there must be a reason for its existence I assume.

[JDBC-27] Use repeat instead of replicate

Wed, 29 Feb 2012 17:34:52 -0600

The replicate function is deprecated and the suggested alternative is to use repeat.

[JDBC-25] The classnames private var should be public

Sun, 12 Feb 2012 04:21:20 -0600

I've got a bunch of integration/system tests in Lobos which are run over multiple target database. As I do not want to force all target drivers to be made as dev-dependencies, I check if the driver is available before running those tests. As long as the classname key is in the db-spec it's all fine, but now java.jdbc support db-spec with that key, it would be useful to have access to the classnames var for me, so it would be great to have it made public. This isn't a blocker as up till now I don't really need it, but I'm planning to add some tests for string based db-spec.

[JDBC-22] Infer :classname from :subprotocol

Wed, 4 Jan 2012 23:53:10 -0600

Having to specify both :classname and :subprotocol seems like needless boilerplate. Only :subprotocol should be required.

[JDBC-24] Make where-params optional for update-values function

Wed, 4 Jan 2012 23:43:46 -0600

This prevents having to pass a wonky always-true clause (e.g. ["1=1"]) as the where-params when updating all the rows in a table.

[JDBC-23] Prepared statements' parameters don't get applied

Tue, 22 Nov 2011 01:01:05 -0600

Basically a goof line in the code.
If you pass a PreparedStatement to with-query-results, it chops off too many items from the param list, whereas if you pass in straight SQL, it's ok.

Deleted the offending line

[JDBC-21] Accept string URL as connection parameters

Wed, 2 Nov 2011 14:52:38 -0500

Should support connection parameters in this style:

(with-connection "postgres://user:password@host.com:port/db-name" [...])

[JDBC-15] resultset-seq uses structmap which has been deprecated

Sun, 16 Oct 2011 16:32:49 -0500

structmap has been deprecated. The open question is whether to replace its use with record or regular map (I'd expect the latter would be less surprising?).

[JDBC-3] Inconsistency between tuples and regular maps

Wed, 12 Oct 2011 16:29:40 -0500

Stuart Sierra: In general, c.c.sql is inconsistent about using tuples of column values versus maps of column=>value pairs. Both are useful, and I'd like to see versions of all the core functions that support both modes.

[JDBC-9] Solve duplicate column label issue

Tue, 11 Oct 2011 16:12:09 -0500

Per Peter Siewart (and others):

– While working with clojure.java.jdbc, I kept running into the "ResultSet must have unique column labels" exception, so I rewrote it to automatically rename duplicate columns. Now, I can run "SELECT * FROM tbl1 JOIN tbl2 ON tbl1.id = tbl2.id" without causing any errors.

This would definitely be a nice enhancement but so far no one with a signed CA has offered a patch

Figuring out the appropriate renaming strategy is key - a sensible default with an optional function to allow users to specify how to do this? Or perhaps only renaming if users specify such a function?

[JDBC-16] do-prepared doesn't throw on invalid SQL

Tue, 11 Oct 2011 14:12:42 -0500

(jdbc/do-prepared "syntactically invalid SQL")

returns nil, while

(jdbc/do-commands "syntactically invalid SQL")

throws as expected.

[JDBC-17] Type-hint public fns returning Connection

Mon, 10 Oct 2011 23:21:26 -0500

clojure.java.jdbc/connection and find-connection both return a java.sql.Connection but are not type-hinted.

There is a type hint on clojure.java.jdbc.internal/connection*, but it does not carry through to the public Var.

Tagging these functions eliminates 9 reflection warnings in a project I have.

[JDBC-14] delete-rows does not work with null WHERE

Sun, 4 Sep 2011 01:32:37 -0500

(delete-rows "foo" [])

generates the SQL

delete from foo where null

, which deletes 0 rows. Instead, I expect it to generate

delete from foo

[JDBC-6] Add hook to allow operations on PreparedStatement prior to query execution

Wed, 10 Aug 2011 02:23:49 -0500

It would be very useful to be able to pass in a function argument to several c.j.j functions that would be called on the PreparedStatement before executing the query, for example, calling setFetchSize(). I'm not sure what the syntax should look like for this...

[JDBC-4] Global connection mechanism / connection pooling

Wed, 10 Aug 2011 02:22:46 -0500

Nicolas Buduroi: What I would like to see added is a global connection mechanism like in ClojureQL and Lobos. We could even add a connection pool behind the scene.

[JDBC-12] exceptions when trying to insert into apache derby

Mon, 25 Jul 2011 16:52:00 -0500

java.sql.SQLException: Only executeBatch and clearBatch allowed in the middle of a batch.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.newSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.checkIfInMiddleOfBatch(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
at clojure.java.jdbc.internal$do_prepared_return_keys_STAR_$fn__84.invoke(internal.clj:272)
at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:185)
at clojure.java.jdbc.internal$do_prepared_return_keys_STAR_.doInvoke(internal.clj:272)
at clojure.lang.RestFn.applyTo(RestFn.java:140)
at clojure.core$apply.invoke(core.clj:542)
at clojure.java.jdbc$insert_values.doInvoke(jdbc.clj:259)
at clojure.lang.RestFn.invoke(RestFn.java:443)
at hiredman.triples$store_triple$fn_471$fn_472.invoke(triples.clj:39)
at clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:174)
at hiredman.triples$store_triple$fn__471.invoke(triples.clj:39)
at clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:156)
at hiredman.triples$store_triple.invoke(triples.clj:39)
at hiredman.clojurebot.factoids$fn__2689.invoke(factoids.clj:139)
at clojure.lang.MultiFn.invoke(MultiFn.java:167)
at hiredman.clojurebot.factoids$factoid_command_run.invoke(factoids.clj:173)
at conduit.core$fn_284$a_arr285$a_arr_286.invoke(core.clj:233)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$a_except$a_except__307.invoke(core.clj:350)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:42)
at clojurebot.conduit$a_indirect$this_fn__500.invoke(conduit.clj:9)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.irc$reply_fn$irc_reply_fn__350.invoke(irc.clj:22)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$select_fn.invoke(core.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.irc$irc_run$handle_msg__421.invoke(irc.clj:166)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$partial$fn__3678.doInvoke(core.clj:2009)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at conduit.core$comp_fn$a_comp__193.invoke(core.clj:45)
at conduit.core$a_run.invoke(core.clj:28)
at conduit.irc$irc_run$run__424.invoke(irc.clj:175)
at conduit.irc$irc_run.doInvoke(irc.clj:179)
at clojure.lang.RestFn.applyTo(RestFn.java:140)
at clojure.core$apply.invoke(core.clj:542)
at clojurebot.core$main$fn_4654.invoke(core.clj:214)
at clojure.lang.AFn.call(AFn.java:18)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.sql.SQLException: Only executeBatch and clearBatch allowed in the middle of a batch.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 185 more

replacing .executeUpdate with .executeBatch in do-prepared-return-keys* seems to fix the problem and be what was meant in the first place, the result is bound to 'counts' and .executeUpdate returns a single count while .executeBatch returns multiple

[JDBC-8] Review (and fix) reflection warnings

Sun, 17 Jul 2011 17:47:15 -0500

There are quite a few reflection warnings in the c.j.j code which should be reviewed and performance critical pieces type hinted.

[JDBC-11] Transaction not rolled back on Throwable exception

Sun, 17 Jul 2011 03:29:00 -0500

(transaction
(insert-values .....)
(throw (Throwable. "baaaaad")))

Reported by Sebastián Galkin on github.

Transactions rollback on Exception but not Throwable.

[JDBC-5] Adding stropping and qualified identifiers to java.jdbc

Sat, 9 Jul 2011 14:28:25 -0500

Following the following discussion:

https://groups.google.com/d/topic/clojure/j7fXdkFMrec/discussion

I've implemented a simple stropping feature that also provide a way to supply qualified identifiers and a function (as-identifier) that can be reused in user code. The implementation can be found there:

https://github.com/budu/java.jdbc/commit/0822d323cd4d267533a88241be68cb184ff9bf28

It's also attached to this issue.

Here's some example of how that feature could be used:

user> (create-table :foo [:bar :integer] [:baz :text])
"CREATE TABLE foo (bar integer, baz text)"
user> (create-table [:schema :foo] [:bar :integer] [:baz :text])
"CREATE TABLE schema.foo (bar integer, baz text)"
user> (with-stropping \" identity (create-table [:schema :foo] [:bar :integer] [:baz :text]))
"CREATE TABLE \"schema\".\"foo\" (\"bar\" integer, \"baz\" text)"
user> (with-stropping [[ ]] identity (create-table [:schema :foo] [:bar :integer] [:baz :text]))
"CREATE TABLE [schema].[foo] ([bar] integer, [baz] text)"

[JDBC-2] create-table should allow table-spec

Sat, 9 Jul 2011 14:19:18 -0500

create-table takes a name and a seq of column-specs but does not allow any table-spec to follow that so you cannot, for example, the engine type / charset / etc.

[JDBC-10] Attempts to return generated keys on batch inserts fails on PostgreSQL and MS SQL Server

Mon, 27 Jun 2011 22:12:10 -0500

From andreasmk2:

"JDBC driver for PostgreSQL and ms-sql does not support returning of generated keys from statements executed in batch. The SQL exception was thrown when 'do-prepared*' was called with 'return-keys' set to true. Currently only the function 'insert-values' calls a new method 'do-single-prepared-get-keys*' when single relation is inserted."

[JDBC-7] Clojure 1.2 compatibility

Wed, 8 Jun 2011 20:02:46 -0500

Per Aaron Bedra: "I just tried to use this on a 1.2 project that we have been working on for a while now and the dynamic defs are killing it. Can I change it to use the older ^{:dynamic true} syntax and release a new version? Is that the only thing that is keeping this off of 1.2? For now we need to make sure the new contrib libraries are 1.2 compatible, so people can start transitioning now."

[JDBC-1] Provide option to return SQL generated / execution stats

Sun, 8 May 2011 21:12:52 -0500

Shantanu: Provide a mechanism to show the SQL being executed (configurable, so that it can be turned off)

Sean: Good idea. Even better, a way to access statistics about the prepared statement after execution - timing etc?

Shantanu: Yes, that would be an add-on value to show how are the queries performing.