Been thinking about the new prefetching model some more and found a
glaring performance hole - the most common N:1 prefetch case will
result in a cartesian product processing in memory. E.g. if one Artist
has 3 Paintings, and the Paintings are fetched with Artist prefetch,
the Artist DB data will be read repeatedly 3 times. The result will be
correct - 3 Paintings all pointing to a single Artist object, however
processing will be much slower.
Now will be making another pass over the code to restore the old
prefetch strategy for N:1 relationships. Hopefully the resulting code
will be tighter than it used to be.
Andrus
On Sep 6, 2009, at 9:43 PM, Andrus Adamchik wrote:
> Good to have a little time again to hack Cayenne internals.
>
> Just committed a pretty big change to the prefetching algorithm
> motivated by CAY-1250 bug report. So combining prefetching and
> inheritance now works 100%.
>
> One visible effect of this change is that all disjoint prefetch
> queries will now include the ID's of the source side of the prefetch
> relationship and a mandatory join to the source entity. In return
> for this small inefficiency (increased result set size... hopefully
> most ID's are small), we get a bunch of benefits, main one being the
> ability to process related fetched objects in a consistent manner
> regardless of the relationship semantics (1..1, 1..N, N..M). This
> strategy was used before for flattened relationships, now it is used
> for everything. On the other hand this change allowed to optimize
> some related cases, so all in all, there may be no performance
> penalty.
>
> It is still possible to go back and optimize it further to prevent
> the addition of the extra columns to the resultset in some cases
> (e.g. if both joined FK and PK are present in the result, only fetch
> one of them), I wish we could do that in some central location (like
> SelectTranslator) instead of writing endless if/else in the prefetch
> processing code.
>
> Now the prefetch code is easier to make sense of, with fewer if/
> else. And I am planning to refactor it further.
>
> Also I came very close to fixing the biggest remaining limitation of
> disjoint prefetching:
>
> https://issues.apache.org/jira/browse/CAY-1025
>
> Andrus
>
>
This archive was generated by hypermail 2.0.0 : Mon Sep 07 2009 - 08:54:22 EDT