Re: Relationships that are really lazy

From: Andrus Adamchik (andru..bjectstyle.org)
Date: Tue Jan 28 2003 - 17:01:31 EST

  • Next message: Andriy Shapochka: "Opinion on using DO class as entity id!?"

    I agree. We need a generic Cayenne-aware data structure that does this. A
    smarter version of current IncrementalFaultList. I guess LRU algorithm
    will be OK (at least I can't think of any other that would make better
    sense).

    This can be used all across Cayenne:

    - In DataContext ObjectStore
    - In relationship lists
    - In lists returned from queries

    Andrus

    > Just dreaming along in the same line... is this perhaps a specialized
    > case of a more generic size-limited cache of objects. This is all just
    > coming straight out of my head, unfiltered yet:
    >
    > This could probably be handled at the ObjectStore level, where each
    > ObjectStore has a cache of current objects (full snapshots, the works).
    > It has a limit to the size of this cache, and if new objects are fetched
    > and the cache is full, older clean objects (those with no changes) are
    > removed from the cache and replaced with the new ones (or rather, the
    > older objects are returned to Hollow state). The cache strategy would
    > be interesting, but could probably be just LRU, with the
    > readProperty(Directly?) method being used to indicate "usage". The idea
    > below could be implemented in terms of this, maybe. Perhaps a
    > per-entity flag to indicate whether it should be cached like this? A
    > cache/size per entity? It might be useful in low-memory situations, or
    > in the below situation where you *know* that certain objects should not
    > really be kept around once they've been used... set the cache size to
    > the batch size and go for it?
    >
    > Just thinking out loud
    >
    > Craig
    >
    > On Wed, 2003-01-29 at 10:30, Andrus Adamchik wrote:
    >> Another peformance feature I just thought of... Don't know when we can
    >> implement it (we have a pretty big queue already), just hope that
    >> someday I wake up and see it in Cayenne.
    >>
    >>
    >> Background:
    >>
    >> One of the TopLink applications that I am working on right now has a
    >> case when a to-many relationships resolves to potentially up to a 50
    >> thousand objects. All of these objects need to be iterated through to
    >> do some work on them (normally just read their data). Performance
    >> (esp. memory use) of this is extremely poor since TopLink reads all
    >> the objects in memory.
    >>
    >>
    >> Current Solution:
    >>
    >> Instead of using relationships use queries. Cayenne (as well as
    >> TopLink) allows to optimize this with ResultIterators as well as paged
    >> queries.
    >>
    >>
    >> Object/Relational Solution:
    >>
    >> I think it would be cool to extend existing query solutions above to
    >> relationships. I can see two cases, that can be specified in the
    >> mapping:
    >>
    >> - Incremental fill.
    >> - Max objects.
    >>
    >> With incremental fill, this will work just like the paginated queries:
    >>
    >> http://objectstyle.org/cayenne/userguide/perform/index.html
    >>
    >> Reading all ObjectIds first, and resolving objects as needed. With
    >> "max objects", we may add a parameter that is a number of max objects
    >> in the array, "unfaulting" pages of objects once the number of faulted
    >> objects exceeds this number.
    >>
    >>
    >> I'll keep dreaming,
    >>
    >> Andrus
    >>
    >>
    >>
    >>



    This archive was generated by hypermail 2.0.0 : Tue Jan 28 2003 - 17:01:31 EST