Re: Caching query results

From: Francesco Fuzio (francescofuzi..iscali.it)
Date: Mon Sep 25 2006 - 10:56:53 EDT

  • Next message: Andrus Adamchik: "News publishing [Was: Cayenne web site]"

    Thank you for the answers: I'm definitely looking forward to trying the
    3.0 cool features you mentioned.

    As for 2.1 (since for us is important to keep data updated without
    relying on expiration timing) I was thinking about this approach (for a
    clustered environment)

    1) Enable Cayenne Replicated Shared Object Cache
    2) Disable Cayenne Query (i.e list ) Cache
    3) Use a Caching framework supporting automatic distributed
    refresh/invalidation policy (e.g Oscahe or Ehcache) to save query
    results as list of ObjectId's.
    4) In case of Query "Cache Hit" use the cached ObjectId's to retrieve
    the associated DataObjects via the DataContext [ public Persistent
    <http://incubator.apache.org/cayenne/1_2/api/cayenne/org/objectstyle/cayenne/Persistent.html>
    *localObject*(ObjectId
    <http://incubator.apache.org/cayenne/1_2/api/cayenne/org/objectstyle/cayenne/ObjectId.html> id,
    Persistent
    <http://incubator.apache.org/cayenne/1_2/api/cayenne/org/objectstyle/cayenne/Persistent.html> prototype)]

    What do you think, is this approach reasonable? Will it work?

    Thanks,

    Francesco.

    Andrus Adamchik wrote:
    > Until 3.0 invalidating query lists was totally up to the developer.
    >
    > In 3.0 we added three things - RefreshQuery (mentioned by Ari), "query
    > cache group" concept, and pluggable QueryCache. This makes clustering
    > and refresh policy management much simpler. I am running this latest
    > code (3.0 trunk essentially) in pre-production environment on a
    > cluster using OSQueryCache and it work extremely well.
    >
    > I am going to document it better at some point (maybe even write an
    > article). For now please post on the list if you have more questions
    > about that. Essentially I do something like this:
    >
    > dataDomain.setQueryCacheFactory(new OSQueryCacheFactory());
    >
    > and then configure "oscache.properties" with group refresh policies
    > (see OSQueryCache javadocs and OSCache framework documentation for
    > details).
    >
    > Once you have this set up, you can build in more bells and whistles if
    > you wish. For instance I implemented the following extensions on a
    > customer project:
    >
    > * a web console for interactive cache group invalidation on the cluster.
    > * hooks to invalidate certain groups when certain entities are
    > modified by the user via another new 3.0 feature - callbacks (I am
    > going to write an overview of callbacks soon).
    >
    > Andrus
    >
    >
    > On Sep 25, 2006, at 9:34 AM, Aristedes Maniatis wrote:
    >>
    >> On 25/09/2006, at 10:41 PM, Francesco Fuzio wrote:
    >>
    >>> <<It is important to understand that caching of *result lists* is
    >>> done independently from caching of *individual DataObjects and
    >>> DataRows*. Therefore the API is different as well. Also cached
    >>> results lists _are not synchronized across VMs (even the shared
    >>> cache)._>>_
    >>>
    >>> _So it seems that if my application is deployed in a cluster I have
    >>> to implement a custom _distributed_ invalidation mechanism.
    >>
    >> It depends on how important it is that the data is up to date. One
    >> solution might be to add functionality to Cayenne which invalidates
    >> objects in the cache after a certain period of time (for instance 1
    >> hour). It is now possible to invalidate objects one at time
    >> (http://cwiki.apache.org/confluence/display/CAYDOC/RefreshQuery)
    >>
    >> We are examining the same problem in our application (not a web
    >> cluster, but a client/server application). For now we switched off
    >> caching, but we need to think through the issues. One solution is to
    >> implement a distributing messaging system to send notifications
    >> between clients - I think Andrus was looking at XMPP at one time
    >> (http://www.xmpp.org/summary.html) for this. Certainly it would be
    >> handy if a messaging system was built into Cayenne for this. It would
    >> also be useful for things like pessimistic locking.
    >>
    >>
    >> Ari Maniatis
    >>
    >>
    >>
    >> -------------------------->
    >> ish
    >> http://www.ish.com.au
    >> Level 1, 30 Wilson Street Newtown 2042 Australia
    >> phone +61 2 9550 5001 fax +61 2 9550 4001
    >> GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
    >>
    >>
    >
    >
    > __________ NOD32 1.1767 (20060921) Information __________
    >
    > This message was checked by NOD32 antivirus system.
    > http://www.nod32.com
    >
    >
    >



    This archive was generated by hypermail 2.0.0 : Mon Sep 25 2006 - 10:57:33 EDT