Re: relationship query and cache refreshing in 3tier cayenne

From: Marcin Skladaniec (marci..sh.com.au)
Date: Wed May 14 2008 - 04:06:39 EDT

  • Next message: Andrus Adamchik: "Re: relationship query and cache refreshing in 3tier cayenne"

    Hello

    We have started to write the caching documentation, and we will post
    it to the list for verification this week. Before that I would like to
    clarify my previous email.
     From what I can see the LOCAL_CACHE refresh does propagate between
    clients. This is what I do in pseudo code:

    first initial check on client 1:
    performQuery(Painting).size() -> 10 //fetching the entire list of
    Paintings using SelectQuery
    getArtistWithName("Monet").getPaintings() -> 2 //fetching the list of
    paintings using RelationshipQuery

    now the same on client 2:
    performQuery(Painting).size() -> 10
    getArtistWithName("Monet").getPaintings() -> 2

    //create a new painting:
    getArtistWithName("Monet").addPainting(newPainting());
    commitChanges();

    //client 2 has all the data updated, as expected:
    performQuery(Painting).size() -> 11
    getArtistWithName("Monet").getPaintings() -> 3

    //back to client 1:
    performQuery(Painting).size() -> 11 [1]
    getArtistWithName("Monet").getPaintings() -> 2 [2]

    [1] - re-doing selectquery in different client seems to work! Andrus
    this opposes to what you have said about the RefreshQuery propagation!
    [2] - even so, the relationship is not ...

    All select queries are executed with paging and LOCAL_CACHE. Every
    entity uses a separate cache key.

    Best regards
    Marcin

    On 14/05/2008, at 8:46 AM, Andrus Adamchik wrote:

    > From what I see the problem is that RefreshQuery is not propagated
    > to remote clients. LOCAL_CACHE is local the tier where the query
    > originated. With some fixes in M4, this statement is true for both
    > nested contexts and ROP... Since by default there are no events
    > pushed from the server to the client and no peer-to-peer client
    > communication, client 1 uses its local cache and knows nothing about
    > the RefreshQuery executed by client 2.
    >
    > There are a few solutions to that:
    >
    > (1) set timemout on the query cache by using something like an
    > OSCache cache provider. This will give you a delayed refresh. This
    > may or may not work for your app, but this can be used with the
    > current Cayenne. For many apps that would be a reasonable
    > compromise, with cache ensuring that queries don't happen too often..
    >
    > (2) Setup server-to-client event push. I did that in the past with a
    > always-on XMPP connection in parallel with the main HTTPS data
    > channel (I know it worked for object updates... it needs to be
    > extended to cache groups invalidation). For a long time I wanted to
    > investigate Jetty Continuations (and more generally, queuing events
    > on the server for each live client, and letting client frequently
    > poll for events). This would allow to reuse the main ROP connection,
    > but this needs to be developed yet.
    >
    > Andrus
    >
    >
    > On May 7, 2008, at 7:25 PM, Marcin Skladaniec wrote:
    >> Hi
    >>
    >> In our application we are using LOCAL_CACHE and cache keys to
    >> refresh it, but only for a special context called 'shared' . We
    >> have overridden the CayenneContext commitChanges with following to
    >> ensure cache invalidation on every commit :
    >>
    >> if (!isSharedContext()) {
    >> try {
    >> List<Class> commitedClasses = new Vector<Class>();
    >> for (Object o : uncommittedObjects())
    >> if (!commitedClasses.contains(o.getClass()) && o instanceof
    >> PersistentObject)
    >> commitedClasses.add(o.getClass());
    >>
    >> super.commitChanges();
    >>
    >> for (Class<? extends PersistentObject> c : commitedClasses) {
    >> performGenericQuery(new RefreshQuery(new String[]
    >> { PersistentObject.defaultCacheKeyForEntity(c) }));
    >> }
    >>
    >> } catch (CayenneRuntimeException e) {
    >> runtimeExceptionThrown(e, false);
    >> }
    >> } else {
    >> logger.error("Attempt to save shared context", new
    >> IllegalStateException("Shared context is read-only"));
    >> }
    >>
    >> whenever a select query is executed we are setting the cache policy
    >> and keys
    >>
    >> if (query instanceof SelectQuery) {
    >> SelectQuery sq = ((SelectQuery) query);
    >> if (isSharedContext) {
    >> // if the query is on the shared context then use cache
    >> // sq.setCachePolicy(QueryMetadata.LOCAL_CACHE);
    >>
    >> // if the root class of the query is kind of PersistentObject
    >> then use the cache keys
    >> if (sq.getRoot() instanceof Class &&
    >> PersistentObject.class.isAssignableFrom((Class<? extends
    >> PersistentObject>) (sq.getRoot()))) {
    >> Class<? extends PersistentObject> c = (Class<? extends
    >> PersistentObject>) sq.getRoot();
    >> List<String> currentCacheGroups = new Vector<String>();
    >> if (sq.getCacheGroups() != null)
    >> currentCacheGroups = Arrays.asList(sq.getCacheGroups());
    >>
    >> String key = PersistentObject.defaultCacheKeyForEntity(c);
    >> if (!currentCacheGroups.contains(key)) {
    >> currentCacheGroups.add(key);
    >> sq.setCacheGroups(currentCacheGroups.toArray(new String[]
    >> {}));
    >> }
    >> }
    >>
    >> }
    >> }
    >>
    >>
    >> This works nicely, but there is one problem: the relationship query
    >> does not return refreshed values.
    >>
    >> An example. there are two views: One is a list of artists with
    >> painting counts, the counts are calculated using relationship
    >> (anArtist.getPaintings().size()). The second is a simple list of
    >> paintings.
    >> 1) client app 1: artists list shows:
    >> Monet - 5
    >> Malevich - 4
    >> vanGogh - 3
    >> painting list contains 12 lines
    >> client app 2 adds a new painting for vanGogh, it lists now
    >> Monet - 5
    >> Malevich - 4
    >> vanGogh - 4
    >> painting list contains 13 lines
    >> client app 1 lists the artists again:
    >> Monet - 5
    >> Malevich - 4
    >> vanGogh - 3 <- incorrect
    >> painting list contains 13 records, which is correct
    >>
    >> The odd thing is also that for a given client application both
    >> lists (artist and painting) are using the same context, so if the
    >> record is there, why it does not show up when accessed via
    >> relationship ?
    >>
    >> Is there something I'm doing wrong ? Is there a way to force the
    >> RelationshipQuery to refresh ?
    >> We are using Cayenne build from sources about a month ago (svn
    >> 642725).
    >> Marcin
    >>
    >>
    >>
    >





    This archive was generated by hypermail 2.0.0 : Wed May 14 2008 - 04:07:24 EDT