Michael,
thanks for your input. I have started analyzing this in detail and the problem has become multidimensional.
Assuming a webstore pattern:
First, if the DataContext never goes out of scope, and the DataContext never releases references to DataObjects, then eventually *all* of the products will be loaded into the DataContext. I can't imagine that this would be the default behavior.
Second: webstore reality trumps delegation to DBMS of filtering & sort-ordering, since there are not only search-based fetches, but productList relationships to "Category" and "Vendor" objects. I was told by your team in a previous email that I (as designer) have no access to modifying or overriding relationship select-queries. Therefore if I use the Cayenne relationship (for a Vendor or Category) to get a list of Products, I would would be forced to apply a list-based filter or sort-ordering.
*If* the DataContext never releases DataObjects, then eventually (as a result of the relationship imposed list-based filtering/ordering), every Product will be faulted and loaded into memory.
Michael, this can't be correct because then it would mimic the behavior of a memory leak.
On the other hand, if it is correct, then my only option left is to completely disable all Cayenne relationships, and do custom select queries which replace the relationships while including the required filtering and sort ordering. This kind-of sounds screwy as well.
Therefore, I am now more confused about this than when I started. :)
My hope is that the DataContext now allows more control over garbage collecting objects.
Joe
On Apr 8, 2010, at 7:32 PM, Michael Gentry wrote:
> My Q4 comment may be based more upon my usage of Cayenne 2 than 3. I
> know there is some weak reference stuff in 3 that I haven't looked
> into yet and perhaps it will automatically clean things up now (which
> would be wonderful). I've always taken the approach of evicting
> things I no longer wanted so I could control my memory footprint
> better. Maybe Andrus/Andrey/etc will chime in with better info here
> if I am incorrect on this one.
>
> mrg
>
>
> On Thu, Apr 8, 2010 at 5:49 PM, Joe Baldwin <jfbaldwi..arthlink.net> wrote:
>> Hi Michael,
>>
>> Thank you for your methodical answers.
>>
>> RE Q3
>> I suspected that this might be one of my problems, and I am trying to redesign so that I only filter & order small result lists.
>>
>> RE Q4
>> Oops! OK, this one totally threw me. Given my understanding of the behavior and usage of the DataContext is somewhere between "wow" to "hmmm, I didn't know that", Perhaps I need to review a primer. :)
>>
>> But seriously:
>> Since I am deploying a web-app, my understanding is that in that type of scenario the container (via the Tomcat CayenneFilter) is instantiating and controlling the DataContext (and I am, presumably now accessing it via BaseContext.getThreadObjectContext();) However, beyond that piece of understanding, I am not sure what to do with it in order to manage objects.
>>
>> I read your docs once more and it says that the DataContext uses "weak references to store registered objects". I *thought* this meant that (assuming I use the default configuration), that the DataContext would allow the DataObjects to be garbage collected, if there is no reference (Java Reference) and if I did not implement query caching.
>>
>> I am confused by reading you Q4 comment and then the "Memory Management Strategy" documentation.
>>
>> (Note: I am attempting to learn more about Cayenne so that I can optimize (and re-factor) my code. I am still uncertain whether I should be implementing caching, not implementing caching, initiating garbage collection (after freeing objects), or just sitting back and letting Cayenne DataContext handle it.)
>>
>> Thanks,
>> Joe
>>
>>
>>
>>
>> On Apr 8, 2010, at 5:18 PM, Michael Gentry wrote:
>>
>>> Hi Joe,
>>>
>>> Assuming 2000 products fetched with a paginated query and a page size
>>> of 50, I believe the answers are ...
>>>
>>> Q1: 2000 hollow objects are created. Cayenne fetches the primary
>>> keys of the objects first in a paginated query.
>>>
>>> Q2: Whichever page (or pages) you access in the paginated list will
>>> fault those objects for that page (grouped by 50 at a time) into
>>> memory.
>>>
>>> Q3: If you try to filter or sort in-memory, then it will have to fetch
>>> all of the objects. Might be better to re-issue a query to the DB.
>>>
>>> Q4: If your list pointers go out-of-scope, but the DataContext is
>>> still in-scope, then the objects will still be in the DataContext
>>> until you evict them.
>>>
>>> mrg
>>>
>>>
>>> On Thu, Apr 8, 2010 at 4:24 PM, Joe Baldwin <jfbaldwi..arthlink.net> wrote:
>>>> Andrus,
>>>>
>>>> In this same context, I have a question about what behavior I should expect to see.
>>>>
>>>> Let us assume for this scenario:
>>>> 1. a web deployment with Tomcat, typical webstore pattern
>>>> 2. 2000-5000 products with an attribute "name"
>>>> 3. custom search engine built on top of cayenne
>>>> a. mostly standard configuration
>>>> b. set PageSize to 50
>>>> c. using (presumably) default caching
>>>> d. custom factory class method used to create a select query
>>>> 4. a user searches for "guitar" substring in the name field
>>>>
>>>> Questions:
>>>> 1. Based on my tests, it appears that "hollow" objects are created but no faults are fired. (true?)
>>>> 2. If I display any product data for a subset less than PageSize, then only <PageSize> number of objects are faulted (true ?)
>>>> 3. If I execute "filterObjects" or "orderList" on the resulting list, I assume the fault is fired for all objects in the list. (true ?)
>>>> 4. What normally happens to these objects if my list pointers go out of scope? (Are they GarbageCollected per Java rules?)
>>>>
>>>> (Note: I have not been able to get much visibility in understanding (3) & (4), but I think my tests show that 1 & 2 are correct.)
>>>> Joe
>>>>
>>>>
>>>>
>>>>
>>>> On Apr 8, 2010, at 12:42 PM, Andrus Adamchik wrote:
>>>>
>>>>> BTW, we started on some monitoring hooks inside Cayenne runtime in 3.0 per http://issues.apache.org/jira/browse/CAY-1173 , however then it was decided that it'll work much better with the 3.1 DI-based stack. So this is waiting for its time.
>>>>>
>>>>> Andrus
>>>>>
>>>>>
>>>>> On Apr 7, 2010, at 6:16 PM, Joe Baldwin wrote:
>>>>>
>>>>>> I am attempting to do more performance monitoring (with an eye towards optimizing my design & use of DataObject lists and lifetime).
>>>>>>
>>>>>> The first thing I would like to do is monitor how many DataObjects are created and which JSP session they belong.
>>>>>>
>>>>>> My idea was to simply create a constructor for the DataObject of interest and place monitoring hooks inside. I would assume that I could also create a finalize method to do similar things.
>>>>>>
>>>>>> Is this the best way to create custom monitoring using Cayenne DataObjects or is there a better way?
>>>>>>
>>>>>> (BTW, even after reading the docs, I am still somewhat mystified by how my configuration specifications control the life cycle and performance. So I am hoping these custom hooks will make my design decisions more visible, especially with multiple users.)
>>>>>>
>>>>>> Thanks,
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>>
This archive was generated by hypermail 2.0.0 : Thu Apr 08 2010 - 20:13:32 EDT