Re: Batch faulting with Cayenne 3

From: Alexander Lamb (dev) ("Alexander)
Date: Thu Nov 15 2007 - 04:45:11 EST

  • Next message: Dave Merrin: "Reengineer Database Schema ant task"

    > The code below should work. Another way is to use multi-step
    > prefetching on a root of to-many relationship.

    Well, that's a good news so we shall probably implement my piece of
    code for most relationships. However, I am afraid I didn't understand
    your second sentence:-(
    >
    >
    >> Now, it would still be cool if we could have batch faulting for the
    >> odd places where we didn't set up the prefetching.
    >
    > While I used batch faulting in webobjects days and found it quite
    > useful, I could never understand how to make it work predictably
    > (i.e. fault the objects that I care about). I wouldn't object though
    > to somebody (or even myself) implementing it at the framework level
    > if whoever that is could explain me the algorithm used to select
    > which objects to fault. IIRC EOF builds internal "fault chains".
    > Wonder how much overhead this would incur in Cayenne.
    I don't think there is a particular order in which the faults are
    fired. Actually, it is not a problem since gradually in a few queries,
    all outstanding faults will be fired. My guess is that upon firing the
    first fault of a to-one registered to batch fault, you simply (ok, not
    that simple:-) look at all entities of same class in the DataContext
    and take the first X to be fired at random. X being the size of the
    batch. Of course, it some how means if there is a to-one which needs
    to be batch faulted it has to be flagged somewhere so you quickly know
    which objects to take into account.
    >
    > BTW relationship prefetching policies can be specified per JPA spec
    > (and hence will be implemented in Cayenne). However my understanding
    > is that JPA specifies a different kind of prefetch - which
    > attributes/relationships to resolve eagerly when an object is fetched.

    I am afraid I don't know anything about JPA, but it probably means in
    the future some kind of interface in the modeler to be able to specify
    those prefetches?

    Thanks!

    Alex

    >
    >
    > Andrus
    >
    >
    >
    > On Nov 14, 2007, at 9:33 AM, Alexander Lamb (dev) wrote:
    >
    >> Well, yes it is possible up to a point.
    >>
    >> Usually it is through the "to-many" relationship I get my objects.
    >> Some other times it might be through a custom query meaning I have
    >> to do it each time.
    >>
    >> However, as I said in the second email I sent about prefetching,
    >> the solution is maybe the following:
    >>
    >>..uppressWarnings("unchecked")
    >> public List<Role> getRoles () {
    >>
    >> if
    >> (org
    >> .apache
    >> .cayenne
    >> .Fault.class.isInstance(this.readPropertyDirectly("roles"))) {
    >> Expression exp = ExpressionFactory.matchExp("person", this);
    >> SelectQuery query = new SelectQuery(Role.class, exp);
    >> query.addPrefetch("profile");
    >> query.addPrefetch("person");
    >> List<Role> roles = getObjectContext().performQuery(query);
    >> ToManyList tml = new ToManyList(this,"roles");
    >> tml.addAll(roles);
    >> writePropertyDirectly("roles",tml);
    >> }
    >> return super.getRoles();
    >> }
    >>
    >> The advantage of this is that it does the prefetch but also sets
    >> correctly the "to-many" relationship, meaning it will not refetch
    >> everything if I do an addToRoles or removeFromRoles.
    >>
    >> If I want to refault the relationship, I do:
    >>
    >>
    >> if
    >> (org
    >> .apache
    >> .cayenne
    >> .access
    >> .ToManyList.class.isInstance(this.readPropertyDirectly("roles"))) {
    >> ((org.apache.cayenne.access.ToManyList)getRoles()).invalidate();
    >> }
    >>
    >> Is this the correct way of doing it?
    >>
    >> If so, could there be a way to add this in a generic way to the
    >> model?
    >>
    >> Now, it would still be cool if we could have batch faulting for the
    >> odd places where we didn't set up the prefetching.
    >>
    >> Alex
    >>
    >> Le 14 nov. 07 à 14:45, Andrus Adamchik a écrit :
    >>
    >>> Can you use prefetching instead? You got a list of users vis some
    >>> sort of query - just add prefetch to that query.
    >>>
    >>> Andrus
    >>>
    >>>
    >>> On Nov 14, 2007, at 8:11 AM, Alexander Lamb (dev) wrote:
    >>>
    >>>> Hello list,
    >>>>
    >>>> One thing is killing performance of our application: it is the
    >>>> resolving of individual to-one faults in lists.
    >>>>
    >>>> For example, we can have 200 roles each refering to a person.
    >>>>
    >>>> When we loop through the roles, for each role where we do a
    >>>> role.getPerson() there will be a return trip to the database.
    >>>>
    >>>> In the EOF days, there was a possibility to define a batch
    >>>> faulting strategy for the entity. In that we would say for
    >>>> example "batch fault 20 for person" and the first time a to-one
    >>>> fault to person from role would be found, it would look in the
    >>>> data context for up to 19 more to build a single SQL statement
    >>>> and fetch in one go the person objects and resolve up to 20 faults.
    >>>>
    >>>> Is this feature available somewhere in Cayenne 3m2 or planned in
    >>>> the near future?
    >>>>
    >>>> If not, is there some kind of callback or hook wich would allow
    >>>> us to do the same thing?
    >>>>
    >>>> Thanks,
    >>>>
    >>>> Alex
    >>>>
    >>>
    >>
    >



    This archive was generated by hypermail 2.0.0 : Thu Nov 15 2007 - 04:46:17 EST