Hi Joe,
Your (original) understanding of weak references + data contexts in
cayenne 3 is correct.
Cayenne 2 did used strong references. Cayenne 3, however, uses weak
refs. So as long as your application doesn't have any references to
the object(s), it (they) should be properly GC'd.
As for the relationships issue... I wrote an app awhile ago that had
relationships where the "many" side of the relationship could easily
be in the thousands, and even ten thousands.
What I wound up doing was implementing a custom
"LowMemoryRelationshipList" that circumvented some of the internals of
cayenne to handle the relationship manually.
As for the relationship-imposed list/memory-based filtering... this
same app allowed you to navigate directly from one row to a separate
view for relatedd objects. For instance, if viewing a list of users,
there would be a column for "attendance" that would report on the
number of attendance records associated, and then allow you to click
the # of associated attendance records to go directly to a listing of
attendance records only for that user, and all of that was done by
direct sql querying. What's more, the same exact code handles all
entities and all relationships. One of the nice things about cayenne
is the very rich set of metadata it makes available to you about your
objects, and I was able to leverage that to build the system. So, I
think you don't necessarily /have/ to abandon cayenne-managed
relationships, although it will take a bit of creativity to keep as
much out of memory as possible. :)
Robert
On Apr 8, 2010, at 4/87:12 PM , Joe Baldwin wrote:
> Michael,
>
> thanks for your input. I have started analyzing this in detail and
> the problem has become multidimensional.
>
> Assuming a webstore pattern:
>
> First, if the DataContext never goes out of scope, and the
> DataContext never releases references to DataObjects, then
> eventually *all* of the products will be loaded into the
> DataContext. I can't imagine that this would be the default behavior.
>
> Second: webstore reality trumps delegation to DBMS of filtering &
> sort-ordering, since there are not only search-based fetches, but
> productList relationships to "Category" and "Vendor" objects. I
> was told by your team in a previous email that I (as designer) have
> no access to modifying or overriding relationship select-queries.
> Therefore if I use the Cayenne relationship (for a Vendor or
> Category) to get a list of Products, I would would be forced to
> apply a list-based filter or sort-ordering.
>
> *If* the DataContext never releases DataObjects, then eventually (as
> a result of the relationship imposed list-based filtering/ordering),
> every Product will be faulted and loaded into memory.
>
> Michael, this can't be correct because then it would mimic the
> behavior of a memory leak.
>
> On the other hand, if it is correct, then my only option left is to
> completely disable all Cayenne relationships, and do custom select
> queries which replace the relationships while including the required
> filtering and sort ordering. This kind-of sounds screwy as well.
>
> Therefore, I am now more confused about this than when I started. :)
>
> My hope is that the DataContext now allows more control over garbage
> collecting objects.
>
> Joe
>
>
>
> On Apr 8, 2010, at 7:32 PM, Michael Gentry wrote:
>
>> My Q4 comment may be based more upon my usage of Cayenne 2 than 3. I
>> know there is some weak reference stuff in 3 that I haven't looked
>> into yet and perhaps it will automatically clean things up now (which
>> would be wonderful). I've always taken the approach of evicting
>> things I no longer wanted so I could control my memory footprint
>> better. Maybe Andrus/Andrey/etc will chime in with better info here
>> if I am incorrect on this one.
>>
>> mrg
>>
>>
>> On Thu, Apr 8, 2010 at 5:49 PM, Joe Baldwin
>> <jfbaldwi..arthlink.net> wrote:
>>> Hi Michael,
>>>
>>> Thank you for your methodical answers.
>>>
>>> RE Q3
>>> I suspected that this might be one of my problems, and I am trying
>>> to redesign so that I only filter & order small result lists.
>>>
>>> RE Q4
>>> Oops! OK, this one totally threw me. Given my understanding of
>>> the behavior and usage of the DataContext is somewhere between
>>> "wow" to "hmmm, I didn't know that", Perhaps I need to review a
>>> primer. :)
>>>
>>> But seriously:
>>> Since I am deploying a web-app, my understanding is that in that
>>> type of scenario the container (via the Tomcat CayenneFilter) is
>>> instantiating and controlling the DataContext (and I am,
>>> presumably now accessing it via
>>> BaseContext.getThreadObjectContext();) However, beyond that piece
>>> of understanding, I am not sure what to do with it in order to
>>> manage objects.
>>>
>>> I read your docs once more and it says that the DataContext uses
>>> "weak references to store registered objects". I *thought* this
>>> meant that (assuming I use the default configuration), that the
>>> DataContext would allow the DataObjects to be garbage collected,
>>> if there is no reference (Java Reference) and if I did not
>>> implement query caching.
>>>
>>> I am confused by reading you Q4 comment and then the "Memory
>>> Management Strategy" documentation.
>>>
>>> (Note: I am attempting to learn more about Cayenne so that I can
>>> optimize (and re-factor) my code. I am still uncertain whether I
>>> should be implementing caching, not implementing caching,
>>> initiating garbage collection (after freeing objects), or just
>>> sitting back and letting Cayenne DataContext handle it.)
>>>
>>> Thanks,
>>> Joe
>>>
>>>
>>>
>>>
>>> On Apr 8, 2010, at 5:18 PM, Michael Gentry wrote:
>>>
>>>> Hi Joe,
>>>>
>>>> Assuming 2000 products fetched with a paginated query and a page
>>>> size
>>>> of 50, I believe the answers are ...
>>>>
>>>> Q1: 2000 hollow objects are created. Cayenne fetches the primary
>>>> keys of the objects first in a paginated query.
>>>>
>>>> Q2: Whichever page (or pages) you access in the paginated list will
>>>> fault those objects for that page (grouped by 50 at a time) into
>>>> memory.
>>>>
>>>> Q3: If you try to filter or sort in-memory, then it will have to
>>>> fetch
>>>> all of the objects. Might be better to re-issue a query to the DB.
>>>>
>>>> Q4: If your list pointers go out-of-scope, but the DataContext is
>>>> still in-scope, then the objects will still be in the DataContext
>>>> until you evict them.
>>>>
>>>> mrg
>>>>
>>>>
>>>> On Thu, Apr 8, 2010 at 4:24 PM, Joe Baldwin <jfbaldwi..arthlink.net
>>>> > wrote:
>>>>> Andrus,
>>>>>
>>>>> In this same context, I have a question about what behavior I
>>>>> should expect to see.
>>>>>
>>>>> Let us assume for this scenario:
>>>>> 1. a web deployment with Tomcat, typical webstore pattern
>>>>> 2. 2000-5000 products with an attribute "name"
>>>>> 3. custom search engine built on top of cayenne
>>>>> a. mostly standard configuration
>>>>> b. set PageSize to 50
>>>>> c. using (presumably) default caching
>>>>> d. custom factory class method used to create a select query
>>>>> 4. a user searches for "guitar" substring in the name field
>>>>>
>>>>> Questions:
>>>>> 1. Based on my tests, it appears that "hollow" objects are
>>>>> created but no faults are fired. (true?)
>>>>> 2. If I display any product data for a subset less than
>>>>> PageSize, then only <PageSize> number of objects are faulted
>>>>> (true ?)
>>>>> 3. If I execute "filterObjects" or "orderList" on the resulting
>>>>> list, I assume the fault is fired for all objects in the list.
>>>>> (true ?)
>>>>> 4. What normally happens to these objects if my list pointers go
>>>>> out of scope? (Are they GarbageCollected per Java rules?)
>>>>>
>>>>> (Note: I have not been able to get much visibility in
>>>>> understanding (3) & (4), but I think my tests show that 1 & 2
>>>>> are correct.)
>>>>> Joe
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Apr 8, 2010, at 12:42 PM, Andrus Adamchik wrote:
>>>>>
>>>>>> BTW, we started on some monitoring hooks inside Cayenne runtime
>>>>>> in 3.0 per http://issues.apache.org/jira/browse/CAY-1173 ,
>>>>>> however then it was decided that it'll work much better with
>>>>>> the 3.1 DI-based stack. So this is waiting for its time.
>>>>>>
>>>>>> Andrus
>>>>>>
>>>>>>
>>>>>> On Apr 7, 2010, at 6:16 PM, Joe Baldwin wrote:
>>>>>>
>>>>>>> I am attempting to do more performance monitoring (with an eye
>>>>>>> towards optimizing my design & use of DataObject lists and
>>>>>>> lifetime).
>>>>>>>
>>>>>>> The first thing I would like to do is monitor how many
>>>>>>> DataObjects are created and which JSP session they belong.
>>>>>>>
>>>>>>> My idea was to simply create a constructor for the DataObject
>>>>>>> of interest and place monitoring hooks inside. I would assume
>>>>>>> that I could also create a finalize method to do similar things.
>>>>>>>
>>>>>>> Is this the best way to create custom monitoring using Cayenne
>>>>>>> DataObjects or is there a better way?
>>>>>>>
>>>>>>> (BTW, even after reading the docs, I am still somewhat
>>>>>>> mystified by how my configuration specifications control the
>>>>>>> life cycle and performance. So I am hoping these custom hooks
>>>>>>> will make my design decisions more visible, especially with
>>>>>>> multiple users.)
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>
This archive was generated by hypermail 2.0.0 : Thu Apr 08 2010 - 20:35:30 EDT