Re: Relationships that are really lazy

From: Craig Miskell (cmiskel..lbatross.co.nz)
Date: Tue Jan 28 2003 - 16:48:08 EST

  • Next message: Andrus Adamchik: "Re: Relationships that are really lazy"

    Just dreaming along in the same line... is this perhaps a specialized
    case of a more generic size-limited cache of objects. This is all just
    coming straight out of my head, unfiltered yet:

    This could probably be handled at the ObjectStore level, where each
    ObjectStore has a cache of current objects (full snapshots, the works).
    It has a limit to the size of this cache, and if new objects are fetched
    and the cache is full, older clean objects (those with no changes) are
    removed from the cache and replaced with the new ones (or rather, the
    older objects are returned to Hollow state). The cache strategy would
    be interesting, but could probably be just LRU, with the
    readProperty(Directly?) method being used to indicate "usage". The idea
    below could be implemented in terms of this, maybe. Perhaps a
    per-entity flag to indicate whether it should be cached like this? A
    cache/size per entity? It might be useful in low-memory situations, or
    in the below situation where you *know* that certain objects should not
    really be kept around once they've been used... set the cache size to
    the batch size and go for it?

    Just thinking out loud

    Craig

    On Wed, 2003-01-29 at 10:30, Andrus Adamchik wrote:
    > Another peformance feature I just thought of... Don't know when we can
    > implement it (we have a pretty big queue already), just hope that someday
    > I wake up and see it in Cayenne.
    >
    >
    > Background:
    >
    > One of the TopLink applications that I am working on right now has a case
    > when a to-many relationships resolves to potentially up to a 50 thousand
    > objects. All of these objects need to be iterated through to do some work
    > on them (normally just read their data). Performance (esp. memory use) of
    > this is extremely poor since TopLink reads all the objects in memory.
    >
    >
    > Current Solution:
    >
    > Instead of using relationships use queries. Cayenne (as well as TopLink)
    > allows to optimize this with ResultIterators as well as paged queries.
    >
    >
    > Object/Relational Solution:
    >
    > I think it would be cool to extend existing query solutions above to
    > relationships. I can see two cases, that can be specified in the mapping:
    >
    > - Incremental fill.
    > - Max objects.
    >
    > With incremental fill, this will work just like the paginated queries:
    >
    > http://objectstyle.org/cayenne/userguide/perform/index.html
    >
    > Reading all ObjectIds first, and resolving objects as needed. With "max
    > objects", we may add a parameter that is a number of max objects in the
    > array, "unfaulting" pages of objects once the number of faulted objects
    > exceeds this number.
    >
    >
    > I'll keep dreaming,
    >
    > Andrus
    >
    >
    >
    >



    This archive was generated by hypermail 2.0.0 : Tue Jan 28 2003 - 16:48:37 EST