Re: Clearing query cache on commit

From: Gili (cowwo..bs.darktech.org)
Date: Wed Sep 14 2005 - 10:39:43 EDT

  • Next message: Cris Daniluk: "Re: Clearing query cache on commit"

            Ok, I have a patch in mind but first let me run my proposal by you (so
    you can poke holes in it).

    1) Given DataObject foo, define validateForInsert() in it...
    2) In the validation method(), retrieve all the queries whose results
    would become invalidated by the insert and invoke
    query.invalidateResults() on them

            The flexibility of this approach really appeals to me. The only
    downside I see is that if I insert 1000 items that would invalidate the
    same query, I'd have to construct it 1000 times and invoke
    invalidateResults() 1000 times on it. Ideally there should be a more
    efficient approach to doing so. So alternatively, we could introduce a
    method that takes in a *list* of inserted, modified and deleted objects
    of a given type/ObjEntity and based upon that it can invoke
    query.invalidateResults() once across the entire list. The downside of
    this approach is that it sounds like you'd need to add a listener to a
    ObjEntity and I don't think Cayenne currently supports that...?

    Gili

    Cris Daniluk wrote:
    > Please phrase your request in the form of a patch :)
    >
    > I think you'll find you are grossly underestimating the difficulty of
    > the problem though...
    >
    > On 9/14/05, Gili <cowwo..bs.darktech.org> wrote:
    >
    >> But ... why not simply add a flushCachedResults(query) method directly
    >>into Cayenne's API? It's implementation will likely consist of a single
    >>line of code and will save us a lot of work on the outside.
    >>
    >>Gili
    >>
    >>Cris Daniluk wrote:
    >>
    >>>Geee.... open your mouth one time and next thing you know, you've
    >>>volunteered for something :)
    >>>
    >>>The code is fairly integrated into our design, so it won't be
    >>>meanignful for me to post it. I will try to clean it up so that it is
    >>>standalone, and post a wiki entry.
    >>>
    >>>As for Gili's question, this method really does not allow you to "lazy
    >>>load" the objects. I'd be inclined to think that the performance gains
    >>>you might expect from lazy loading are fictitious. However, I have no
    >>>desire to hear you tell me why not, so I won't go down that road :)
    >>>I'll just say, if you want to lazy load, you should probably look at
    >>>writing your own manager to access your named queries. You can pass in
    >>>a DataContext and pull out the named query. The Context can then
    >>>decide whether or not to set refresh=true on a "one time only" basis,
    >>>based on the monitor that I will provide to the wiki when I get a
    >>>chance.
    >>>
    >>>Cris
    >>>
    >>>On 9/14/05, Gentry, Michael (Contractor) <michael_gentr..anniemae.com> wrote:
    >>>
    >>>
    >>>>*sniff* *sniff*
    >>>>
    >>>>Smells like a new wiki page? :-)
    >>>>
    >>>>/dev/mrg
    >>>>
    >>>>
    >>>>-----Original Message-----
    >>>>From: Gili [mailto:cowwo..bs.darktech.org]
    >>>>Sent: Wednesday, September 14, 2005 9:51 AM
    >>>>To: cayenne-use..bjectstyle.org
    >>>>Subject: Re: Clearing query cache on commit
    >>>>
    >>>>
    >>>>
    >>>> Yes... I'm also quite interested. It sounds exactly like what
    >>>>I'm
    >>>>looking for, minus having to invoking the query right away with
    >>>>refresh=true. For performance reasons, I'd want to fault the query
    >>>>results right away but not actually invoke it. Is this possible?
    >>>>
    >>>>Gili
    >>>>
    >>>>Eric Schneider wrote:
    >>>>
    >>>>
    >>>>>Cris,
    >>>>>
    >>>>>This sounds very cool. Would you mind sharing your implementation?
    >>>>>
    >>>>>Thanks,
    >>>>>Eric
    >>>>>
    >>>>>
    >>>>>On Sep 14, 2005, at 9:04 AM, Cris Daniluk wrote:
    >>>>>
    >>>>>
    >>>>>
    >>>>>>I just have a process that iterates cached queries and looks to see
    >>>>
    >>>>if
    >>>>
    >>>>
    >>>>>>the class type matches the class type of any just-committed objects.
    >>>>>>If so, I re-execute those queries with caching disabled
    >>>>>>(refresh=true).
    >>>>>>
    >>>>>>Actually, I do it in a slightly different way that avoids unnecessary
    >>>>>>refetching, etc, but that's what you get for free. As currently
    >>>>>>running, it never unnecessarily tosses a cached query, and never
    >>>>>>misses a query that needs tossing.
    >>>>>>
    >>>>>>On 9/13/05, Gili <cowwo..bs.darktech.org> wrote:
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>> I did some more research and it seems that depending on the
    >>>>
    >>>>>>>cache type
    >>>>>>>(local or shared) the cached results are either stored in
    >>>>>>>DataRowCache.snapshotLists (for shared) or
    >>>>
    >>>>ObjectStore.queryResultMap
    >>>>
    >>>>
    >>>>>>>(for local).
    >>>>>>>
    >>>>>>> As far as I can tell, currently it is impossible to flush
    >>>>>>>these caches
    >>>>>>>safely (DRC.clear() always flushes another unrelated map). I'm
    >>>>
    >>>>looking
    >>>>
    >>>>
    >>>>>>>for a single method that will flush both the local and shared query
    >>>>>>>result caches.
    >>>>>>>
    >>>>>>>Gili
    >>>>>>>
    >>>>>>>Gili wrote:
    >>>>>>>
    >>>>>>>
    >>>>>>>
    >>>>>>>>Hi,
    >>>>>>>>
    >>>>>>>> I need to clear the query cache on DataContext.commitChanges
    >>>>
    >>>>().
    >>>>
    >>>>
    >>>>>>>>The
    >>>>>>>>only way I can see this working is if I invoke DataRowStore.clear
    >>>>
    >>>>()
    >>>>
    >>>>
    >>>>>>>>but
    >>>>>>>>as Mike mentioned to me offline this sounds like an overly
    >>>>
    >>>>aggressive
    >>>>
    >>>>
    >>>>>>>>approach.
    >>>>>>>>
    >>>>>>>> Is there a finer-grained method for clearing query cache for
    >>>>
    >>>>all
    >>>>
    >>>>
    >>>>>>>>queries?
    >>>>>>>>
    >>>>>>>> My use-case is:
    >>>>>>>>
    >>>>>>>>1) Database is empty
    >>>>>>>>2) ContentType.getCanonicalInstance() returns an empty set, so I
    >>>>>>>>commit
    >>>>>>>>a new object to the database
    >>>>>>>>3) DataContext.commitChanges()
    >>>>>>>>4) I invoke ContentType.getCanonicalInstance() again. This time it
    >>>>
    >>>>I
    >>>>
    >>>>
    >>>>>>>>expect it to return the instance I commited to the DB in step 3,
    >>>>>>>>but it
    >>>>>>>>returns the cached result (empty set).
    >>>>>>>>5) DataContext.commitChanges()
    >>>>>>>>6) Database complains of unique constraint violation
    >>>>>>>>
    >>>>>>>> Currently I invoke
    >>>>>>>>"context.invalidateObjects(context.getObjectStore().getObjects ())"
    >>>>
    >>>>>>>>after
    >>>>>>>>each commitChanges(). I'd want to do something equivilent for the
    >>>>>>>>query
    >>>>>>>>cache.
    >>>>>>>>
    >>>>>>>>Thanks,
    >>>>>>>>Gili
    >>>>>>>>
    >>>>>>>
    >>>>>>>--
    >>>>>>>http://www.desktopbeautifier.com/
    >>>>>>>
    >>>>>>
    >>>>--
    >>>>http://www.desktopbeautifier.com/
    >>>>
    >>>
    >>>
    >>--
    >>http://www.desktopbeautifier.com/
    >>
    >
    >

    -- 
    http://www.desktopbeautifier.com/
    



    This archive was generated by hypermail 2.0.0 : Wed Sep 14 2005 - 10:39:44 EDT