Hi,
Yes, the paginated query would indeed be the only way for me to go forward.
The problem however is that I get the exception I posted earlier.
tx
Hans
Andrus Adamchik wrote:
> For paginated queries we contemplated a strategy of a list with
> constant size of fully resolved objects. I.e. when a page is swapped
> in, some other (LRU?) page is swapped out. We decided against it, as
> in a general case it is hard to consistently predict which page should
> be swapped out.
>
> However it should be rather easy to write such a list for a specific
> case with a known access order (e.g. a standard iteration order). In
> fact I would vote to even include such implementation in Cayenne going
> forward.
>
> More specifically, you can extend IncrementalFaultList [1], overriding
> 'resolveInterval' to swap out previously read pages, turning them back
> into ids. And the good part is that you can use your extension
> directly without any need to modify the rest of Cayenne.
>
> Andrus
>
>
> [1] http://cayenne.apache.org/doc/api/org/apache/cayenne/access/IncrementalFaultList.html
>
>
> On Nov 12, 2009, at 10:07 AM, Hans Pikkemaat wrote:
>
>> Hi,
>>
>> So this means that if I use a generic query that the query results
>> are always stored
>> completely in the object store (or the query cache if I configure it).
>>
>> Objects are returned in a list so as long I have a reference to this
>> list (because I'm
>> traversing it) these objects are not garbage collected.
>>
>> If I use the query cache the full query results are cached. This
>> means that I can only
>> tell it to remove the whole query.
>>
>> Effectively this means I'm unable to run a big query and process the
>> results as a stream.
>> So I cannot process the first results and then somehow make them
>> available for
>> garbage collection.
>>
>> The only option I have would be the iterated query but this is only
>> usefull for queries
>> one 1 table without any relations because it is not possible to use
>> prefetching nor is
>> it possible to manually construct relations between obects.
>>
>> My conclusion here is that cayenne is simply not suitable for doing
>> large batch wise
>> query processing because of the memory implications.
>>
>> tx
>>
>> HPI
>>
>> Andrus Adamchik wrote:
>>
>>> As mentioned in the docs, individual objects and query lists are
>>> cached independently. Of course query lists contain a subset of
>>> cached
>>> object store objects inside the lists. An object won't get gc'd if it
>>> is also stored in the query list.
>>>
>>> Now list cache expiration is controlled via query cache factory. By
>>> default this is an LRU map, so as long as the map has enough space to
>>> hold lists (its capacity == # of lists, not # of objects), the
>>> objects
>>> won't get gc'd.
>>>
>>> You can explicitly remove entries from the cache via QueryCache
>>> remove
>>> and removeGroup methods. Or you can use a different QueryCacheFactory
>>> that implements some custom expiration/cleanup mechanism.
>>>
>>> Andrus
>>>
>>> On Nov 11, 2009, at 3:43 PM, Hans Pikkemaat wrote:
>>>
>>>
>>>
>>>> Hi,
>>>>
>>>> I use the latest version of cayenne, 3.0b and am experimenting with
>>>> the object caching features.
>>>>
>>>> The documentation states that committed objects are purged from the
>>>> cache because it uses weak references.
>>>> (http://cayenne.apache.org/doc/individual-object-caching.html)
>>>>
>>>> If I however run a query using SQLTemplate which caches the objects
>>>> into the dataContext local cache (objectstore),
>>>> the objects don't seem to be purged at all. If I simply run the
>>>> query dump the contents using an iterator on the resulting
>>>> List then the nr of registered objects in the objectstore stays the
>>>> same (dataContext.getObjectStore().registeredObjectsCount()).
>>>> Even if I manually run System.gc() I don't see any changes (I know
>>>> this can be normal as gc() doesn't guarantee anything)
>>>>
>>>> What am I doing wrong? Under which circumstances will cayenne purge
>>>> the cache?
>>>>
>>>> tx
>>>>
>>>> Hans
>>>>
>>>>
>>>>
>>>
This archive was generated by hypermail 2.0.0 : Thu Nov 12 2009 - 03:44:34 EST