RE: Transactions , Primary Keys

From: Philip Copeland (pcopelan..voka.com)
Date: Sun Sep 23 2007 - 00:23:06 EDT

  • Next message: Giulio Cesare Solaroli: "Re: Optimize cascade deletes"

    Hi,

    I was planning to use a similar approach - but was thinking of using a
    temporary table - but you are right - if I can do this via a map it
    could work - as long as the number of objects in the import set is
    reasonable.

    Did you have an issue committing every "several thousand records". What
    would you do if it failed at that stage - did you figure out a way to
    resume again at a known point? Handling large imports is never easy.

    Phil

    -----Original Message-----
    From: Aristedes Maniatis [mailto:ar..aniatis.org]
    Sent: Sunday, 23 September 2007 11:54 AM
    To: use..ayenne.apache.org
    Subject: Re: Transactions , Primary Keys

    On 23/09/2007, at 8:26 AM, Philip Copeland wrote:

    > The main problem I'm facing is that objects don't get written to the
    > database (nor Primary Keys generated) unless we commit our
    > transaction.
    > What I'm wishing for is a mode where all changes are made to the
    > underlying database - but not committed until I have completed all the

    > work I need to do.

    Marcin and I have also spent a fair bit of time importing data into a
    Cayenne driven system. In our case we had an XML data source with
    (usually) about 150,000 objects which required all the keys and
    relationships to be remapped.

    I believe that if your solution requires extensive manipulation and
    searching of primary and foreign keys through Cayenne, you are missing a
    large part of the advantages of an ORM driven approach and using direct
    JDBC might be simpler. However, think about whether you are really
    approaching this in the best way.

    In our case we read in the source XML and create not only the Cayenne
    objects, but also a Map<Integer,PersistentEntity> which links those
    objects back to the PKs found in the original import data. That Map can
    then be used to look up the new object from the old PK/FK as required,
    without you needing to know anything about what new PK will be assigned
    upon commit.

    In our case we committed the context every several thousands records, or
    at certain stages in the process.

    Does this help?

    Ari Maniatis

    -------------------------->
    Aristedes Maniatis
    phone +61 2 9660 9700
    PGP fingerprint 08 57 20 4B 80 69 59 E2 A9 BF 2D 48 C2 20 0C C8



    This archive was generated by hypermail 2.0.0 : Sun Sep 23 2007 - 00:23:51 EDT