Re: Cayenne performance degrades over time

From: Joshua Pyle (joshua.t.pyl..mail.com)
Date: Wed Jun 22 2005 - 20:32:17 EDT

  • Next message: Andrus Adamchik: "Re: Cayenne performance degrades over time"

    I actually used this technique in a swing test I have and did run into
    at least one unusual issue.

    When I ran a query to return all of the object from the DB with no
    expression I got only 10 items even though there were over 10,000. If
    I increase the LRUHashMap max to 100 I got 100 items.

    -- 
    Joshua T. Pyle
    Go has always existed.
    

    On 6/22/05, Andrus Adamchik <andru..bjectstyle.org> wrote: > This is awesome indeed. > > One thing to be aware of... If an LRU map throws out an object and > you still have a reference to this object somewhere in the code > (directly or via a relationship), this may result in unpredictable > behavior when you access this object later. One thing that I suggest > is to write test cases to check for this condition, and another one - > catch object removal events if possible and at least change object > state to HOLLOW. > > Andrus > > > On Jun 22, 2005, at 5:43 PM, Joshua Pyle wrote: > > > Awesome, this may be just the thing I will need in an application I > > have to write soon. > > > > Thanks. > > > > > > -- > > Joshua T. Pyle > > Go has always existed. > > > > > > On 6/22/05, Paul Cowan <pwc2..ahoo.com> wrote: > > > >> FYI for users interested in this topic... > >> > >> I just tested the 3 options Andrus listed below. I don't see much > >> performance > >> difference between number 1 and number 2. However, number 3 > >> yields much > >> higher performance if you use an LRULinkedHashMap, and keep the > >> maximum number > >> of objects in the map small. The smaller the map, the faster the > >> performance. > >> I also get better performance calling commitChanges() every > >> 10,000 changes > >> instead of every 1,000. But don't see much difference between > >> 10,000 and > >> 100,000. All depends on the application of course. > >> > >> Source for my modified test program with the LRUHashMap is > >> available here: > >> http://www.buzzsurf.com/java/cayenne/PerfTest.java > >> > >> -Paul > >> > >> > >> > >> --- Andrus Adamchik <andru..bjectstyle.org> wrote: > >> > >> > >>> In general (also answering Holger's message) - I am all for smarter > >>> ObjectStore that covers a wider range of use scenarios. I am not > >>> ready to > >>> say which way is better or more generic - weak reference map or > >>> an LRU > >>> map. If you want to discuss this further, let's use cayenne-devel > >>> list. > >>> > >>> For now there is a number of options: > >>> > >>> 1. Replace a DataContext every so often > >>> 2. Use ObjectStore.startTrackingNewObjects()/unregisterNewObjects > >>> () - we > >>> don't advertise this API to often, but it works pretty well. > >>> 3. Create an ObjectStore subclass along these lines: > >>> > >>> public class MyObjectStore extends ObjectStore { > >>> public MyObjectStore(DataRowStore cache) { > >>> super(cache); > >>> super.objectMap = some weak-reference or LRU map... > >>> } > >>> } > >>> > >>> > >>> DataDomain domain = Configuration.getSharedConfiguration > >>> ().getDomain(); > >>> ObjectStore os = new MyObjectStore(domain.getSharedSnapshotCache()); > >>> DataContext context = new DataContext(domain, os); > >>> > >>> I haven't tried the last scenario, hope it won't break the faulting. > >>> > >>> Andrus > >>> > >>> > >>>> Yes, when I create a new DataContext every 1000 iterations, > >>>> performance > >>>> does > >>>> not suffer any longer. > >>>> > >>>> However, it kinda seems like an inconvienent or kludgy fix. I > >>>> wonder if > >>>> something could be done using weak references in the cache to > >>>> solve this. > >>>> I > >>>> really liked using just one DataContext for this particular > >>>> app. It does > >>>> a > >>>> lot of very short lived object creation. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> --- Andrus Adamchik <andru..bjectstyle.org> wrote: > >>>> > >>>> > >>>>> Paul, > >>>>> > >>>>> Cache settings have effect on the shared cache in a multiuser > >>>>> app. For a > >>>>> single session DataContext (or rather its ObjectStore) has no > >>>>> upper > >>>>> limit > >>>>> on the number of cached objects as it is designed to have a > >>>>> shorter life > >>>>> span. > >>>>> > >>>>> So a more realistic performance test would actually discard a > >>>>> DataContext > >>>>> and create a new one after a few operations (how many is "few" > >>>>> depends > >>>>> on > >>>>> your expected usage patterns). > >>>>> > >>>>> FYI: Here is the stack structure (default is level 2): > >>>>> http://objectstyle.org/cayenne/userguide/design/caching.html > >>>>> > >>>>> Andrus > >>>>> > >>>>> > >>>>> > >>>>>> Hi Andrus, > >>>>>> > >>>>>> I'm seeing behaviour in my app using Cayenne, where it runs > >>>>>> slower and > >>>>>> slower > >>>>>> the longer the application runs without a restart. I suspect > >>>>>> that it > >>>>>> > >>>>> will > >>>>> > >>>>>> eventually run out of memory. Upon restart, performance > >>>>>> starts out > >>>>>> > >>>>> high > >>>>> > >>>>>> again. > >>>>>> > >>>>>> I wrote a very simple test application to rule out any of my > >>>>>> custom > >>>>>> > >>>>> code. > >>>>> > >>>>>> It > >>>>>> simply registers a new objects with the DataContext, sets a > >>>>>> property, > >>>>>> > >>>>> and > >>>>> > >>>>>> calls commitChanges. The test app prints out the elapsed time > >>>>>> for > >>>>>> > >>>>> every > >>>>> > >>>>>> 1000 > >>>>>> iterations. The results are here: > >>>>>> > >>>>>> http://www.buzzsurf.com/java/cayenne/results.txt > >>>>>> > >>>>>> If you want to run the simple test application yourself, it's > >>>>>> here: > >>>>>> > >>>>>> http://www.buzzsurf.com/java/cayenne/CayennePerfTest.zip > >>>>>> > >>>>>> As you can see, performance degrades linearly over time. It > >>>>>> seems > >>>>>> > >>>>> like > >>>>> > >>>>>> DataContext is caching a pointer to every object created, > >>>>>> despite my > >>>>>> > >>>>> cache > >>>>> > >>>>>> settings?? I've been playing with the "Max Number of Objects" > >>>>>> setting > >>>>>> > >>>>> in > >>>>> > >>>>>> the > >>>>>> Modeler, but different settings seem to have no effect. > >>>>>> > >>>>>> -Paul > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>>> > >>>> > >>> > >>> > >>> > >> > >> > >> > > > > > >



    This archive was generated by hypermail 2.0.0 : Wed Jun 22 2005 - 20:32:19 EDT