Re: Cayenne performance degrades over time

From: Andrus Adamchik (andru..bjectstyle.org)
Date: Wed Jun 22 2005 - 20:25:58 EDT

  • Next message: Joshua Pyle: "Re: Cayenne performance degrades over time"

    This is awesome indeed.

    One thing to be aware of... If an LRU map throws out an object and
    you still have a reference to this object somewhere in the code
    (directly or via a relationship), this may result in unpredictable
    behavior when you access this object later. One thing that I suggest
    is to write test cases to check for this condition, and another one -
    catch object removal events if possible and at least change object
    state to HOLLOW.

    Andrus

    On Jun 22, 2005, at 5:43 PM, Joshua Pyle wrote:

    > Awesome, this may be just the thing I will need in an application I
    > have to write soon.
    >
    > Thanks.
    >
    >
    > --
    > Joshua T. Pyle
    > Go has always existed.
    >
    >
    > On 6/22/05, Paul Cowan <pwc2..ahoo.com> wrote:
    >
    >> FYI for users interested in this topic...
    >>
    >> I just tested the 3 options Andrus listed below. I don't see much
    >> performance
    >> difference between number 1 and number 2. However, number 3
    >> yields much
    >> higher performance if you use an LRULinkedHashMap, and keep the
    >> maximum number
    >> of objects in the map small. The smaller the map, the faster the
    >> performance.
    >> I also get better performance calling commitChanges() every
    >> 10,000 changes
    >> instead of every 1,000. But don't see much difference between
    >> 10,000 and
    >> 100,000. All depends on the application of course.
    >>
    >> Source for my modified test program with the LRUHashMap is
    >> available here:
    >> http://www.buzzsurf.com/java/cayenne/PerfTest.java
    >>
    >> -Paul
    >>
    >>
    >>
    >> --- Andrus Adamchik <andru..bjectstyle.org> wrote:
    >>
    >>
    >>> In general (also answering Holger's message) - I am all for smarter
    >>> ObjectStore that covers a wider range of use scenarios. I am not
    >>> ready to
    >>> say which way is better or more generic - weak reference map or
    >>> an LRU
    >>> map. If you want to discuss this further, let's use cayenne-devel
    >>> list.
    >>>
    >>> For now there is a number of options:
    >>>
    >>> 1. Replace a DataContext every so often
    >>> 2. Use ObjectStore.startTrackingNewObjects()/unregisterNewObjects
    >>> () - we
    >>> don't advertise this API to often, but it works pretty well.
    >>> 3. Create an ObjectStore subclass along these lines:
    >>>
    >>> public class MyObjectStore extends ObjectStore {
    >>> public MyObjectStore(DataRowStore cache) {
    >>> super(cache);
    >>> super.objectMap = some weak-reference or LRU map...
    >>> }
    >>> }
    >>>
    >>>
    >>> DataDomain domain = Configuration.getSharedConfiguration
    >>> ().getDomain();
    >>> ObjectStore os = new MyObjectStore(domain.getSharedSnapshotCache());
    >>> DataContext context = new DataContext(domain, os);
    >>>
    >>> I haven't tried the last scenario, hope it won't break the faulting.
    >>>
    >>> Andrus
    >>>
    >>>
    >>>> Yes, when I create a new DataContext every 1000 iterations,
    >>>> performance
    >>>> does
    >>>> not suffer any longer.
    >>>>
    >>>> However, it kinda seems like an inconvienent or kludgy fix. I
    >>>> wonder if
    >>>> something could be done using weak references in the cache to
    >>>> solve this.
    >>>> I
    >>>> really liked using just one DataContext for this particular
    >>>> app. It does
    >>>> a
    >>>> lot of very short lived object creation.
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>> --- Andrus Adamchik <andru..bjectstyle.org> wrote:
    >>>>
    >>>>
    >>>>> Paul,
    >>>>>
    >>>>> Cache settings have effect on the shared cache in a multiuser
    >>>>> app. For a
    >>>>> single session DataContext (or rather its ObjectStore) has no
    >>>>> upper
    >>>>> limit
    >>>>> on the number of cached objects as it is designed to have a
    >>>>> shorter life
    >>>>> span.
    >>>>>
    >>>>> So a more realistic performance test would actually discard a
    >>>>> DataContext
    >>>>> and create a new one after a few operations (how many is "few"
    >>>>> depends
    >>>>> on
    >>>>> your expected usage patterns).
    >>>>>
    >>>>> FYI: Here is the stack structure (default is level 2):
    >>>>> http://objectstyle.org/cayenne/userguide/design/caching.html
    >>>>>
    >>>>> Andrus
    >>>>>
    >>>>>
    >>>>>
    >>>>>> Hi Andrus,
    >>>>>>
    >>>>>> I'm seeing behaviour in my app using Cayenne, where it runs
    >>>>>> slower and
    >>>>>> slower
    >>>>>> the longer the application runs without a restart. I suspect
    >>>>>> that it
    >>>>>>
    >>>>> will
    >>>>>
    >>>>>> eventually run out of memory. Upon restart, performance
    >>>>>> starts out
    >>>>>>
    >>>>> high
    >>>>>
    >>>>>> again.
    >>>>>>
    >>>>>> I wrote a very simple test application to rule out any of my
    >>>>>> custom
    >>>>>>
    >>>>> code.
    >>>>>
    >>>>>> It
    >>>>>> simply registers a new objects with the DataContext, sets a
    >>>>>> property,
    >>>>>>
    >>>>> and
    >>>>>
    >>>>>> calls commitChanges. The test app prints out the elapsed time
    >>>>>> for
    >>>>>>
    >>>>> every
    >>>>>
    >>>>>> 1000
    >>>>>> iterations. The results are here:
    >>>>>>
    >>>>>> http://www.buzzsurf.com/java/cayenne/results.txt
    >>>>>>
    >>>>>> If you want to run the simple test application yourself, it's
    >>>>>> here:
    >>>>>>
    >>>>>> http://www.buzzsurf.com/java/cayenne/CayennePerfTest.zip
    >>>>>>
    >>>>>> As you can see, performance degrades linearly over time. It
    >>>>>> seems
    >>>>>>
    >>>>> like
    >>>>>
    >>>>>> DataContext is caching a pointer to every object created,
    >>>>>> despite my
    >>>>>>
    >>>>> cache
    >>>>>
    >>>>>> settings?? I've been playing with the "Max Number of Objects"
    >>>>>> setting
    >>>>>>
    >>>>> in
    >>>>>
    >>>>>> the
    >>>>>> Modeler, but different settings seem to have no effect.
    >>>>>>
    >>>>>> -Paul
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>
    >>>>>
    >>>>>
    >>>>
    >>>>
    >>>>
    >>>
    >>>
    >>>
    >>
    >>
    >>
    >
    >



    This archive was generated by hypermail 2.0.0 : Wed Jun 22 2005 - 20:25:59 EDT