Re: JDBC batching MySQL

From: Borut Bolčina (borut.bolcin..mail.com)
Date: Wed Oct 27 2010 - 12:18:07 UTC

  • Next message: Andrus Adamchik: "Re: JDBC batching MySQL"

    Hi,

    I was tailing the mysql server's general query log which outputs all
    queries. In the case of datacontext.commit() with 100 objects there should
    be one INSERT statement, but instead there were 100.

    Because of this, there was no difference in performance.

    Otherwise, no exceptions, but the log says:

    [2010/10/27 14:16:14.478] INFO [access.QueryLogger:logQueryParameters]:
    [bind: 1->meta_dirty:NULL, 2->name:'Parkirišče Tivoli II', 3->type:'90',
    4->x:461470.17160042987, 5->y:101894.83749067372]
    [2010/10/27 14:16:14.480] INFO [access.QueryLogger:logGeneratedKey]:
    Generated PK: point_of_interest.id = 16743
    [2010/10/27 14:16:14.480] INFO [access.QueryLogger:logUpdateCount]: ===
    updated 1 row.

    That is "bind", and not "batch bind" as you just explained.

    -Borut

    2010/10/27 Andrus Adamchik <andru..bjectstyle.org>

    > Hi Borut,
    >
    > I may try running it in debugger this week or early next week (during
    > ApacheCon, when I have a bit of spare time).
    >
    > For now can you explain how do you know that batching didn't work? Was
    > there an exception? Was a a difference in performance, etc?
    >
    > IIRC the only difference in Cayenne logging output between batch and
    > no-batch mode is "bind" vs. "batch bind" String for parameters. Is this what
    > you are referring to?
    >
    > Andrus
    >
    > On Oct 27, 2010, at 12:00 PM, Borut Bolčina wrote:
    >
    > > Hello,
    > >
    > > in another thred Andrus suggested to have a look at current support for
    > > MySQL batching.
    > >
    > > According to:
    > >
    > > - http://bbish.net/htmls/mysql/news.html (Changes in MySQL Connector/J
    > > 5.1.3)
    > > -
    > http://www.jroller.com/mmatthews/entry/speeding_up_batch_inserts_for(bug>
    > > corrected in connectorJ 5.1.8)
    > >
    > > the batching is supported.
    > >
    > > I added
    > > rewriteBatchedStatements=true
    > >
    > > to conection string and in the modeller I set the
    > > Custom Adapter to org.apache.cayenne.dba.mysql.MySQLAdapter
    > >
    > > and in the code I initialized the adapter with
    > > DataDomain dataChannel = (DataDomain) dataContext.getChannel();
    > > DataNode dataNode = dataChannel.getNode("MapsNode");
    > > MySQLAdapter mySQLAdapter = (MySQLAdapter) dataNode.getAdapter();
    > > mySQLAdapter.setSupportsBatchUpdates(true);
    > >
    > > Then I tried different scenarios: datacontext.commit() on every 100
    > changed
    > > objects and commit after all (4000) objects were added to a datacontext.
    > >
    > > I was running my test with connectorJ 5.1.12 and sadly the batch insert
    > does
    > > not work. I was watching the MySQL server's general query log for
    > definite
    > > confirmation what the database is getting from the driver.
    > >
    > > Maybe there is something else to set-up? I really wish this would work,
    > as
    > > performance gains are enormous.
    > >
    > > Cheers,
    > > Borut
    >
    >



    This archive was generated by hypermail 2.0.0 : Wed Oct 27 2010 - 12:18:38 UTC