HI all,thanks for your answers!
Actually i'm using Cayenne 2.0. Resultiterator works well! The memory
doesn't grow, and the data are retrived in a resonable time (100.000 record,
60Mb in 50/60 seconds... this is the worst case).
So i think that this is a good solution that also allow me to maintain the
same logic i actually use for manage datacontext.
I was sure that Cayenne can solve my problem. Thanks for this great product!
;-)
Davide
On Wed, Mar 18, 2009 at 3:52 PM, Andrus Adamchik <andru..bjectstyle.org>wrote:
> Actually using a ResultIterator should be entirely safe in 3.0 in respect
> to memory management. If a user doesn't do something bad (like storing
> obtained objects in a list himself), and only one object is processed at a
> time, there won't be any memory issues.
>
> The servlet i'm implementig will be called by a different webapp
>>> (backoffice.. in a server 2 server way ) with a low traffic but i'm
>>> warried
>>> about the warning reported in the article:
>>>
>>> "In web applications, programmers must ensure that no open
>>> ResultIterators
>>> are kept between HTTP requests. Failure to do so may result in too many
>>> database connections being locked, thus quickly exhausting connection
>>> pool.
>>>
>>
> This one is still valid. Keep ResultIterator request-scoped, and close it
> at the end.
>
> In general, an application with Web GUI is NOT a good candidate for
>>> implementation using ResultIterators."
>>>
>>
> Well, there are always exceptions... The general warning here is based on
> the assumption that large results are usually taking lots of time to fetch
> and process. So users may not see the response fast enough. If the response
> time is reasonable in your case, you can use ResultIterator.
>
> Andrus
>
>
> On Mar 18, 2009, at 4:17 PM, Robert Zeigler wrote:
>
> Problem is that the data context will cache the objects (up to the max
>> object limit); this is alleviated somewhat in 3.0 with the use of weak
>> references
>> in the caching, but there's still potential for running out of memory.
>>
>> You can certainly use ResultIterator, with the caveat that you'll be
>> getting DataRows instead of DataObjects.
>> Alternatively, periodically discard the context that you're using and
>> replace it with a fresh context, something like:
>>
>> Query query = createMyQuery();//SQLTemplate, or SelectQuery, or
>> EJBQLQuery, or whatever.
>> while(notDone) {
>> ObjectContext context = ...;//create a new object context here.
>> //do things like set the query fetch limit and fetch offset.
>> List objs = context.performQuery(query);
>> for(Obj o : objs) {
>> //stream out the obj...
>> }
>> }
>>
>> Robert
>>
>> On Mar 18, 2009, at 3/186:14 AM , mr.abanjo wrote:
>>
>> Hi,i'm developing an application that must load a large ammount of data
>>> (100.000 records) and stream them out from a servlet.
>>>
>>> I created a "for" cycle that for every step load 200 record from the
>>> database (using datacontext.performquery method), and then flush the data
>>> in
>>> the serlvet output stream.
>>> In this way i suppose that the max ammount of memory allocated by cayenne
>>> is
>>> :
>>>
>>> object size * 200
>>>
>>> but seems that this is not true. When i try to get the data from the
>>> servlet, after few seconds, i receive an "out of memory" error. (my 500Mb
>>> allocated for the Heap are reached in a few time... :-(....)
>>>
>>> Which is the best practice in this case?
>>> I can increase the heap, but i want to find a workaround that allow me to
>>> have a small heap allocated during this operation.
>>>
>>> I found this article:
>>> http://cayenne.apache.org/doc20/iterating-through-data-rows.html
>>> Is this the right way?
>>>
>>> The servlet i'm implementig will be called by a different webapp
>>> (backoffice.. in a server 2 server way ) with a low traffic but i'm
>>> warried
>>> about the warning reported in the article:
>>>
>>> "In web applications, programmers must ensure that no open
>>> ResultIterators
>>> are kept between HTTP requests. Failure to do so may result in too many
>>> database connections being locked, thus quickly exhausting connection
>>> pool.
>>> In general, an application with Web GUI is NOT a good candidate for
>>> implementation using ResultIterators."
>>>
>>> Thanks
>>> Davide
>>>
>>
>>
>>
>
This archive was generated by hypermail 2.0.0 : Wed Mar 18 2009 - 11:55:24 EDT