Re: Clearing query cache on commit

From: Gili (cowwo..bs.darktech.org)
Date: Wed Sep 14 2005 - 13:14:15 EDT

  • Next message: Gili: "Unsupported ordering expression"

            What about the 2nd approach I mentioned at the bottom? A user-defined
    method is passed a list of commited DataObjects. This would solve the
    issues you mentioned, right?

    class ImageSpecification extends _ImageSpecification
    {
       public void wasCommited(List objects)
       {
         // Some queries are always invalidated
         getCanonicalInstanceQuery.invalidResults();

         // Some queries are conditionally invalidated
         for (int i=0; i<objects.size(); ++i)
         {
           ImageSpecification typedObject = (ImageSpecification) objects.get(i);
           Map parameters = new HashMap();
           parameters.put("width", new Integer(typedObject.getWidth());
           getByWidthQuery.queryWithParameters(parameters).invalidateResults();
         }
       }
    }

            In the above example, getByWidthQuery is a query with the qualifier
    "width = $width" -- it returns all ImageSpecification instances having a
    specific width. Note how I only invalidate the queries for the specific
    widths being commited.

    Gili

    Cris Daniluk wrote:
    > This isn't really suitable as a patch. You can do this by extending
    > CDO, and it should do what you want. But remember, validate* happens
    > before the commit finishes, so you have no way of knowing if the
    > commit will actually occur at this stage.
    >
    > Plus, I think the performance of this would be far worse than your
    > complaints about my immediate reexecution of the cached query, though
    > :)
    >
    > On 9/14/05, Gili <cowwo..bs.darktech.org> wrote:
    >
    >> Ok, I have a patch in mind but first let me run my proposal by you (so
    >>you can poke holes in it).
    >>
    >>1) Given DataObject foo, define validateForInsert() in it...
    >>2) In the validation method(), retrieve all the queries whose results
    >>would become invalidated by the insert and invoke
    >>query.invalidateResults() on them
    >>
    >> The flexibility of this approach really appeals to me. The only
    >>downside I see is that if I insert 1000 items that would invalidate the
    >>same query, I'd have to construct it 1000 times and invoke
    >>invalidateResults() 1000 times on it. Ideally there should be a more
    >>efficient approach to doing so. So alternatively, we could introduce a
    >>method that takes in a *list* of inserted, modified and deleted objects
    >>of a given type/ObjEntity and based upon that it can invoke
    >>query.invalidateResults() once across the entire list. The downside of
    >>this approach is that it sounds like you'd need to add a listener to a
    >>ObjEntity and I don't think Cayenne currently supports that...?
    >>
    >>Gili
    >>
    >>Cris Daniluk wrote:
    >>
    >>>Please phrase your request in the form of a patch :)
    >>>
    >>>I think you'll find you are grossly underestimating the difficulty of
    >>>the problem though...
    >>>
    >>>On 9/14/05, Gili <cowwo..bs.darktech.org> wrote:
    >>>
    >>>
    >>>> But ... why not simply add a flushCachedResults(query) method directly
    >>>>into Cayenne's API? It's implementation will likely consist of a single
    >>>>line of code and will save us a lot of work on the outside.
    >>>>
    >>>>Gili
    >>>>
    >>>>Cris Daniluk wrote:
    >>>>
    >>>>
    >>>>>Geee.... open your mouth one time and next thing you know, you've
    >>>>>volunteered for something :)
    >>>>>
    >>>>>The code is fairly integrated into our design, so it won't be
    >>>>>meanignful for me to post it. I will try to clean it up so that it is
    >>>>>standalone, and post a wiki entry.
    >>>>>
    >>>>>As for Gili's question, this method really does not allow you to "lazy
    >>>>>load" the objects. I'd be inclined to think that the performance gains
    >>>>>you might expect from lazy loading are fictitious. However, I have no
    >>>>>desire to hear you tell me why not, so I won't go down that road :)
    >>>>>I'll just say, if you want to lazy load, you should probably look at
    >>>>>writing your own manager to access your named queries. You can pass in
    >>>>>a DataContext and pull out the named query. The Context can then
    >>>>>decide whether or not to set refresh=true on a "one time only" basis,
    >>>>>based on the monitor that I will provide to the wiki when I get a
    >>>>>chance.
    >>>>>
    >>>>>Cris
    >>>>>
    >>>>>On 9/14/05, Gentry, Michael (Contractor) <michael_gentr..anniemae.com> wrote:
    >>>>>
    >>>>>
    >>>>>
    >>>>>>*sniff* *sniff*
    >>>>>>
    >>>>>>Smells like a new wiki page? :-)
    >>>>>>
    >>>>>>/dev/mrg
    >>>>>>
    >>>>>>
    >>>>>>-----Original Message-----
    >>>>>>From: Gili [mailto:cowwo..bs.darktech.org]
    >>>>>>Sent: Wednesday, September 14, 2005 9:51 AM
    >>>>>>To: cayenne-use..bjectstyle.org
    >>>>>>Subject: Re: Clearing query cache on commit
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>> Yes... I'm also quite interested. It sounds exactly like what
    >>>>>>I'm
    >>>>>>looking for, minus having to invoking the query right away with
    >>>>>>refresh=true. For performance reasons, I'd want to fault the query
    >>>>>>results right away but not actually invoke it. Is this possible?
    >>>>>>
    >>>>>>Gili
    >>>>>>
    >>>>>>Eric Schneider wrote:
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>Cris,
    >>>>>>>
    >>>>>>>This sounds very cool. Would you mind sharing your implementation?
    >>>>>>>
    >>>>>>>Thanks,
    >>>>>>>Eric
    >>>>>>>
    >>>>>>>
    >>>>>>>On Sep 14, 2005, at 9:04 AM, Cris Daniluk wrote:
    >>>>>>>
    >>>>>>>
    >>>>>>>
    >>>>>>>
    >>>>>>>>I just have a process that iterates cached queries and looks to see
    >>>>>>
    >>>>>>if
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>the class type matches the class type of any just-committed objects.
    >>>>>>>>If so, I re-execute those queries with caching disabled
    >>>>>>>>(refresh=true).
    >>>>>>>>
    >>>>>>>>Actually, I do it in a slightly different way that avoids unnecessary
    >>>>>>>>refetching, etc, but that's what you get for free. As currently
    >>>>>>>>running, it never unnecessarily tosses a cached query, and never
    >>>>>>>>misses a query that needs tossing.
    >>>>>>>>
    >>>>>>>>On 9/13/05, Gili <cowwo..bs.darktech.org> wrote:
    >>>>>>>>
    >>>>>>>>
    >>>>>>>>
    >>>>>>>>
    >>>>>>>>> I did some more research and it seems that depending on the
    >>>>>>
    >>>>>>>>>cache type
    >>>>>>>>>(local or shared) the cached results are either stored in
    >>>>>>>>>DataRowCache.snapshotLists (for shared) or
    >>>>>>
    >>>>>>ObjectStore.queryResultMap
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>(for local).
    >>>>>>>>>
    >>>>>>>>> As far as I can tell, currently it is impossible to flush
    >>>>>>>>>these caches
    >>>>>>>>>safely (DRC.clear() always flushes another unrelated map). I'm
    >>>>>>
    >>>>>>looking
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>for a single method that will flush both the local and shared query
    >>>>>>>>>result caches.
    >>>>>>>>>
    >>>>>>>>>Gili
    >>>>>>>>>
    >>>>>>>>>Gili wrote:
    >>>>>>>>>
    >>>>>>>>>
    >>>>>>>>>
    >>>>>>>>>
    >>>>>>>>>>Hi,
    >>>>>>>>>>
    >>>>>>>>>> I need to clear the query cache on DataContext.commitChanges
    >>>>>>
    >>>>>>().
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>>The
    >>>>>>>>>>only way I can see this working is if I invoke DataRowStore.clear
    >>>>>>
    >>>>>>()
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>>but
    >>>>>>>>>>as Mike mentioned to me offline this sounds like an overly
    >>>>>>
    >>>>>>aggressive
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>>approach.
    >>>>>>>>>>
    >>>>>>>>>> Is there a finer-grained method for clearing query cache for
    >>>>>>
    >>>>>>all
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>>queries?
    >>>>>>>>>>
    >>>>>>>>>> My use-case is:
    >>>>>>>>>>
    >>>>>>>>>>1) Database is empty
    >>>>>>>>>>2) ContentType.getCanonicalInstance() returns an empty set, so I
    >>>>>>>>>>commit
    >>>>>>>>>>a new object to the database
    >>>>>>>>>>3) DataContext.commitChanges()
    >>>>>>>>>>4) I invoke ContentType.getCanonicalInstance() again. This time it
    >>>>>>
    >>>>>>I
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>>>>>>expect it to return the instance I commited to the DB in step 3,
    >>>>>>>>>>but it
    >>>>>>>>>>returns the cached result (empty set).
    >>>>>>>>>>5) DataContext.commitChanges()
    >>>>>>>>>>6) Database complains of unique constraint violation
    >>>>>>>>>>
    >>>>>>>>>> Currently I invoke
    >>>>>>>>>>"context.invalidateObjects(context.getObjectStore().getObjects ())"
    >>>>>>
    >>>>>>>>>>after
    >>>>>>>>>>each commitChanges(). I'd want to do something equivilent for the
    >>>>>>>>>>query
    >>>>>>>>>>cache.
    >>>>>>>>>>
    >>>>>>>>>>Thanks,
    >>>>>>>>>>Gili
    >>>>>>>>>>
    >>>>>>>>>
    >>>>>>>>>--
    >>>>>>>>>http://www.desktopbeautifier.com/
    >>>>>>>>>
    >>>>>>>>
    >>>>>>--
    >>>>>>http://www.desktopbeautifier.com/
    >>>>>>
    >>>>>
    >>>>>
    >>>>--
    >>>>http://www.desktopbeautifier.com/
    >>>>
    >>>
    >>>
    >>--
    >>http://www.desktopbeautifier.com/
    >>
    >
    >

    -- 
    http://www.desktopbeautifier.com/
    



    This archive was generated by hypermail 2.0.0 : Wed Sep 14 2005 - 13:14:15 EDT