Hi,
I was planning to use a similar approach - but was thinking of using a
temporary table - but you are right - if I can do this via a map it
could work - as long as the number of objects in the import set is
reasonable.
Did you have an issue committing every "several thousand records". What
would you do if it failed at that stage - did you figure out a way to
resume again at a known point? Handling large imports is never easy.
Phil
-----Original Message-----
From: Aristedes Maniatis [mailto:ar..aniatis.org]
Sent: Sunday, 23 September 2007 11:54 AM
To: use..ayenne.apache.org
Subject: Re: Transactions , Primary Keys
On 23/09/2007, at 8:26 AM, Philip Copeland wrote:
> The main problem I'm facing is that objects don't get written to the
> database (nor Primary Keys generated) unless we commit our
> transaction.
> What I'm wishing for is a mode where all changes are made to the
> underlying database - but not committed until I have completed all the
> work I need to do.
Marcin and I have also spent a fair bit of time importing data into a
Cayenne driven system. In our case we had an XML data source with
(usually) about 150,000 objects which required all the keys and
relationships to be remapped.
I believe that if your solution requires extensive manipulation and
searching of primary and foreign keys through Cayenne, you are missing a
large part of the advantages of an ORM driven approach and using direct
JDBC might be simpler. However, think about whether you are really
approaching this in the best way.
In our case we read in the source XML and create not only the Cayenne
objects, but also a Map<Integer,PersistentEntity> which links those
objects back to the PKs found in the original import data. That Map can
then be used to look up the new object from the old PK/FK as required,
without you needing to know anything about what new PK will be assigned
upon commit.
In our case we committed the context every several thousands records, or
at certain stages in the process.
Does this help?
Ari Maniatis
-------------------------->
Aristedes Maniatis
phone +61 2 9660 9700
PGP fingerprint 08 57 20 4B 80 69 59 E2 A9 BF 2D 48 C2 20 0C C8
This archive was generated by hypermail 2.0.0 : Sun Sep 23 2007 - 00:23:51 EDT