Re: Clearing query cache on commit

From: Cris Daniluk (cris.danilu..mail.com)
Date: Wed Sep 14 2005 - 10:51:51 EDT

  • Next message: Gili: "Re: Clearing query cache on commit"

    This isn't really suitable as a patch. You can do this by extending
    CDO, and it should do what you want. But remember, validate* happens
    before the commit finishes, so you have no way of knowing if the
    commit will actually occur at this stage.

    Plus, I think the performance of this would be far worse than your
    complaints about my immediate reexecution of the cached query, though
    :)

    On 9/14/05, Gili <cowwo..bs.darktech.org> wrote:
    >
    > Ok, I have a patch in mind but first let me run my proposal by you (so
    > you can poke holes in it).
    >
    > 1) Given DataObject foo, define validateForInsert() in it...
    > 2) In the validation method(), retrieve all the queries whose results
    > would become invalidated by the insert and invoke
    > query.invalidateResults() on them
    >
    > The flexibility of this approach really appeals to me. The only
    > downside I see is that if I insert 1000 items that would invalidate the
    > same query, I'd have to construct it 1000 times and invoke
    > invalidateResults() 1000 times on it. Ideally there should be a more
    > efficient approach to doing so. So alternatively, we could introduce a
    > method that takes in a *list* of inserted, modified and deleted objects
    > of a given type/ObjEntity and based upon that it can invoke
    > query.invalidateResults() once across the entire list. The downside of
    > this approach is that it sounds like you'd need to add a listener to a
    > ObjEntity and I don't think Cayenne currently supports that...?
    >
    > Gili
    >
    > Cris Daniluk wrote:
    > > Please phrase your request in the form of a patch :)
    > >
    > > I think you'll find you are grossly underestimating the difficulty of
    > > the problem though...
    > >
    > > On 9/14/05, Gili <cowwo..bs.darktech.org> wrote:
    > >
    > >> But ... why not simply add a flushCachedResults(query) method directly
    > >>into Cayenne's API? It's implementation will likely consist of a single
    > >>line of code and will save us a lot of work on the outside.
    > >>
    > >>Gili
    > >>
    > >>Cris Daniluk wrote:
    > >>
    > >>>Geee.... open your mouth one time and next thing you know, you've
    > >>>volunteered for something :)
    > >>>
    > >>>The code is fairly integrated into our design, so it won't be
    > >>>meanignful for me to post it. I will try to clean it up so that it is
    > >>>standalone, and post a wiki entry.
    > >>>
    > >>>As for Gili's question, this method really does not allow you to "lazy
    > >>>load" the objects. I'd be inclined to think that the performance gains
    > >>>you might expect from lazy loading are fictitious. However, I have no
    > >>>desire to hear you tell me why not, so I won't go down that road :)
    > >>>I'll just say, if you want to lazy load, you should probably look at
    > >>>writing your own manager to access your named queries. You can pass in
    > >>>a DataContext and pull out the named query. The Context can then
    > >>>decide whether or not to set refresh=true on a "one time only" basis,
    > >>>based on the monitor that I will provide to the wiki when I get a
    > >>>chance.
    > >>>
    > >>>Cris
    > >>>
    > >>>On 9/14/05, Gentry, Michael (Contractor) <michael_gentr..anniemae.com> wrote:
    > >>>
    > >>>
    > >>>>*sniff* *sniff*
    > >>>>
    > >>>>Smells like a new wiki page? :-)
    > >>>>
    > >>>>/dev/mrg
    > >>>>
    > >>>>
    > >>>>-----Original Message-----
    > >>>>From: Gili [mailto:cowwo..bs.darktech.org]
    > >>>>Sent: Wednesday, September 14, 2005 9:51 AM
    > >>>>To: cayenne-use..bjectstyle.org
    > >>>>Subject: Re: Clearing query cache on commit
    > >>>>
    > >>>>
    > >>>>
    > >>>> Yes... I'm also quite interested. It sounds exactly like what
    > >>>>I'm
    > >>>>looking for, minus having to invoking the query right away with
    > >>>>refresh=true. For performance reasons, I'd want to fault the query
    > >>>>results right away but not actually invoke it. Is this possible?
    > >>>>
    > >>>>Gili
    > >>>>
    > >>>>Eric Schneider wrote:
    > >>>>
    > >>>>
    > >>>>>Cris,
    > >>>>>
    > >>>>>This sounds very cool. Would you mind sharing your implementation?
    > >>>>>
    > >>>>>Thanks,
    > >>>>>Eric
    > >>>>>
    > >>>>>
    > >>>>>On Sep 14, 2005, at 9:04 AM, Cris Daniluk wrote:
    > >>>>>
    > >>>>>
    > >>>>>
    > >>>>>>I just have a process that iterates cached queries and looks to see
    > >>>>
    > >>>>if
    > >>>>
    > >>>>
    > >>>>>>the class type matches the class type of any just-committed objects.
    > >>>>>>If so, I re-execute those queries with caching disabled
    > >>>>>>(refresh=true).
    > >>>>>>
    > >>>>>>Actually, I do it in a slightly different way that avoids unnecessary
    > >>>>>>refetching, etc, but that's what you get for free. As currently
    > >>>>>>running, it never unnecessarily tosses a cached query, and never
    > >>>>>>misses a query that needs tossing.
    > >>>>>>
    > >>>>>>On 9/13/05, Gili <cowwo..bs.darktech.org> wrote:
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>>> I did some more research and it seems that depending on the
    > >>>>
    > >>>>>>>cache type
    > >>>>>>>(local or shared) the cached results are either stored in
    > >>>>>>>DataRowCache.snapshotLists (for shared) or
    > >>>>
    > >>>>ObjectStore.queryResultMap
    > >>>>
    > >>>>
    > >>>>>>>(for local).
    > >>>>>>>
    > >>>>>>> As far as I can tell, currently it is impossible to flush
    > >>>>>>>these caches
    > >>>>>>>safely (DRC.clear() always flushes another unrelated map). I'm
    > >>>>
    > >>>>looking
    > >>>>
    > >>>>
    > >>>>>>>for a single method that will flush both the local and shared query
    > >>>>>>>result caches.
    > >>>>>>>
    > >>>>>>>Gili
    > >>>>>>>
    > >>>>>>>Gili wrote:
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>>Hi,
    > >>>>>>>>
    > >>>>>>>> I need to clear the query cache on DataContext.commitChanges
    > >>>>
    > >>>>().
    > >>>>
    > >>>>
    > >>>>>>>>The
    > >>>>>>>>only way I can see this working is if I invoke DataRowStore.clear
    > >>>>
    > >>>>()
    > >>>>
    > >>>>
    > >>>>>>>>but
    > >>>>>>>>as Mike mentioned to me offline this sounds like an overly
    > >>>>
    > >>>>aggressive
    > >>>>
    > >>>>
    > >>>>>>>>approach.
    > >>>>>>>>
    > >>>>>>>> Is there a finer-grained method for clearing query cache for
    > >>>>
    > >>>>all
    > >>>>
    > >>>>
    > >>>>>>>>queries?
    > >>>>>>>>
    > >>>>>>>> My use-case is:
    > >>>>>>>>
    > >>>>>>>>1) Database is empty
    > >>>>>>>>2) ContentType.getCanonicalInstance() returns an empty set, so I
    > >>>>>>>>commit
    > >>>>>>>>a new object to the database
    > >>>>>>>>3) DataContext.commitChanges()
    > >>>>>>>>4) I invoke ContentType.getCanonicalInstance() again. This time it
    > >>>>
    > >>>>I
    > >>>>
    > >>>>
    > >>>>>>>>expect it to return the instance I commited to the DB in step 3,
    > >>>>>>>>but it
    > >>>>>>>>returns the cached result (empty set).
    > >>>>>>>>5) DataContext.commitChanges()
    > >>>>>>>>6) Database complains of unique constraint violation
    > >>>>>>>>
    > >>>>>>>> Currently I invoke
    > >>>>>>>>"context.invalidateObjects(context.getObjectStore().getObjects ())"
    > >>>>
    > >>>>>>>>after
    > >>>>>>>>each commitChanges(). I'd want to do something equivilent for the
    > >>>>>>>>query
    > >>>>>>>>cache.
    > >>>>>>>>
    > >>>>>>>>Thanks,
    > >>>>>>>>Gili
    > >>>>>>>>
    > >>>>>>>
    > >>>>>>>--
    > >>>>>>>http://www.desktopbeautifier.com/
    > >>>>>>>
    > >>>>>>
    > >>>>--
    > >>>>http://www.desktopbeautifier.com/
    > >>>>
    > >>>
    > >>>
    > >>--
    > >>http://www.desktopbeautifier.com/
    > >>
    > >
    > >
    >
    > --
    > http://www.desktopbeautifier.com/
    >



    This archive was generated by hypermail 2.0.0 : Wed Sep 14 2005 - 10:52:09 EDT