Yes it is offline batch.
It usually consists of the following flow:
Read pre-staging table and load all objects into context.
Query staging table for object (if exists update, else insert)
Map objects into staging-table.
I will try the unregister. We have been simply dropping the context and
getting a new one. (Guess that isn't the best way to go about it? Would
this simply grab the same context?)
Anyway I will give it a try and also watch for the new sample project.
Thanks
Troy
----- Original Message -----
From: "Andrus Adamchik" <andru..bjectstyle.org>
To: <cayenne-use..bjectstyle.org>
Sent: Thursday, April 24, 2003 12:25 PM
Subject: Re: large data loads - best practices ?
> From the subject I assume that you are doing some kind of offline batch
> processing? Or is this a web application with lots of sessions that share
> some big areas of common read-inly data?
>
> The solution will depend on the answer :-)
>
> BTW, Arndt, who is using Cayenne for batch processing of huge volumes of
> data, is currently working on an example application that will be included
> in the future releases of Cayenne. He will be showing exactly that - how
> to keep memory usage at bay, no matter how many rows you have to read
> through.
>
> In short, ResultIterator and DataContext.unregisterObjects() are your
> friends:
>
> http://objectstyle.org/cayenne/userguide/perform/index.html#iterator
>
http://objectstyle.org/cayenne/api/cayenne/org/objectstyle/cayenne/access/Da
taContext.html#unregisterObject(org.objectstyle.cayenne.DataObject)
>
> Andrus
>
>
> > Having a bit of trouble with cayenne being very memory intensive...
> >
> > We are using it to load staging tables (lots of data) and have run into
> > a lot of trouble around memory.
> >
> > We are running on a pentium 4 with 1Gig of memory... and by allocating
> > 512 to the jvm it helps somewhat.
> >
> > I was just wondering if there is a best practice for doing this type of
> > process.
> >
> > I.e. committing 1 row at a time, clearing context each iteration maybe,
> > getting a whole new context each iteration?
> >
> > Any suggestions for performance tuning would be appreciated.
> >
> > Troy
>
>
>
This archive was generated by hypermail 2.0.0 : Thu Apr 24 2003 - 15:34:14 EDT