Re: sync cayenne cache in two web apps

From: Mike Kienenberger (mkienen..mail.com)
Date: Thu Jul 30 2009 - 17:45:54 EDT

  • Next message: Lawrence Gerstley: "Oracle trying to get key from sequence (DUAL)--how to stop?"

    Again, if you're really concerned about performance and impact, my
    guess is that it would be trivial to replace the JGroup protocol with
    a unix pipe instead of a network socket implementation, if you're on
    an OS that supports it.

    On Thu, Jul 30, 2009 at 5:43 PM, Mike Kienenberger<mkienen..mail.com> wrote:
    > I don't remember the specific implementation, and it might vary for
    > JavaGroups over what is used internally.  You really don't need to use
    > multicast if you only have two apps.   I'd go with the TCP setup, both
    > for reliability and to limit the impact on the rest of the network.
    > The same traffic probably goes over the wire, but the other machines
    > won't process those packets.
    >
    > Your best bet is to look at the EventBridge code and see what kind of
    > data it sends out.  It might vary between event types if it doesn't
    > simply fault those objects.
    >
    > On Thu, Jul 30, 2009 at 5:37 PM, Tobias
    > Schoessler<tobias.schoessle..mail.com> wrote:
    >> yes that's what I observe too. The messages sent when these updates occure,
    >> do they contain the change infromation or only the information which objects
    >> to invalidate? I got this asked when I asked for the multicast address, to
    >> estiamte traffic for this setup.
    >>
    >> On Thu, Jul 30, 2009 at 11:33 PM, Michael Gentry <mgentr..asslight.net>wrote:
    >>
    >>> Hi Tobias,
    >>>
    >>> I've not used the cache synchronization before, but I was under the
    >>> impression that the main overhead is when inserts/updates/deletes are
    >>> done, not when selects are done.  When you do an insert/update/delete,
    >>> that information must be broadcast, but selects do not.  I'm sure
    >>> someone will correct me if I am wrong on this.  :-)
    >>>
    >>> mrg
    >>>
    >>>
    >>> On Thu, Jul 30, 2009 at 5:23 PM, Tobias
    >>> Schoessler<tobias.schoessle..mail.com> wrote:
    >>> > So thank you for all the suggestions. The solution we finally ended up
    >>> with
    >>> > was the one Mike actually suggested intitially. We got our multicast ip,
    >>> > dropped the latest Jgroups.jar into both webapps lib directories,
    >>> selected
    >>> > Jgroups as the Syncronisation mechanism in the cayenne modeller, used the
    >>> > default jgroups udp.xml config file patched with our multicast ip address
    >>> > and 'snapp' the contexts were synchronized. Very satisfying - Cayenne
    >>> > rocks.! :)
    >>> >
    >>> > Before this I went down the route of trying to make cayenne use the
    >>> global
    >>> > JVM scope to store the shared cache. I moved the cayenne.jar up on the
    >>> > tomcat shared lib directory, out of the two web app lib folders. This did
    >>> > not work out well, I got stuck at the point where one web app worked fine
    >>> > the other one threw class cast exception on the mapping objects saying it
    >>> > cannot cast the types on itself. I assume this is due to the fact that
    >>> both
    >>> > webapps had their own copies of the mapping classes. I tried moving them
    >>> up
    >>> > into the shared tomcat lib aswell, but  then they could not see the web
    >>> app
    >>> > specific classes anymore. So anyway I am happy with our Jgroups solution
    >>> > now.
    >>> >
    >>> > The documentation reads lthis setup has some overhead. Does anybody have
    >>> > experience/numbers how much performance you loose when using jgroups
    >>> > syncronised caches compared to local cache?
    >>> >
    >>> > thanks again everyone.
    >>> >
    >>> > On Thu, Jul 30, 2009 at 10:47 AM, Tobias Schoessler <
    >>> > tobias.schoessle..mail.com> wrote:
    >>> >
    >>> >> Thanks everyone for the posts.
    >>> >>
    >>> >>..ike, I am still not convinced that using the Remote Notification
    >>> Feature
    >>> >> is really nessecary here. After all, there seems to be a JVM shared
    >>> between
    >>> >> webapps in Tomcat and the article posted seems to proof that there is a
    >>> >> possiblity to share information between the webapps on a JVM level. So I
    >>> >> think that using Remote Notification, which I understand to be designed
    >>> for
    >>> >> Cross JVM notification creates too much overhead.
    >>> >>
    >>> >> You mentioned the possibility of sharing the DataContext between the
    >>> >> webapps. I think I have to explore this possibility first, as this would
    >>> >> have less overhead compared to the notification based solution.
    >>> >> Currently I am using the
    >>> >>
    >>> org.objectstyle.cayenne.conf.ServletUtil.getSessionContext(request.getSession())
    >>> >> to obtain my DataContexts per request.
    >>> >> If I could change the scope the DataContexts are stored to cross web app
    >>> >> scope instead of session scope I could share the DataContexts between
    >>> the
    >>> >> two web apps. Assuming that I can setup the two webapps to share the
    >>> same
    >>> >> session Ids as described in the article.
    >>> >>
    >>> >> This might be a no go for me as the two contexts use different
    >>> >> authentication realms - I have to check this. But even then wouldn't it
    >>> be
    >>> >> possbile to configure the cayenne shared cache to use this cross web
    >>> context
    >>> >> scope for its shared cache. Then I could use
    >>> >> org.objectstyle.cayenne.conf.ServletUtil.getSessionContext(session) in
    >>> the
    >>> >> two web apps transparently and cayenne would refresh the DataContext
    >>> from
    >>> >> this shared cache in the background. Could somebody point me to where
    >>> this
    >>> >> shared cayenne cache is configured to have its scope? I assume it uses
    >>> JVM
    >>> >> static scope?
    >>> >>
    >>> >>..alcolm, thanks for suggesting this alternative. If I understand you
    >>> >> correctly you suggest to switch off the cayenne cache alltogether and
    >>> use
    >>> >> the jsptag based caching of the OScache project? The problem with this
    >>> is
    >>> >> that not all my responses are generated from jsptags. I have many ajax
    >>> >> requests generating json responses without bothering the jsp container.
    >>> >>
    >>> >> Tobias
    >>> >>
    >>> >>
    >>> >> On Thu, Jul 30, 2009 at 3:00 AM, Malcolm Edgar <malcolm.edga..mail.com
    >>> >wrote:
    >>> >>
    >>> >>> You can also use OSCache with Cayenne and have the cached queries
    >>> >>> expire frequently, i.e. after 30 seconds
    >>> >>>
    >>> >>> regards Malcolm Edgar
    >>> >>>
    >>> >>> On Thu, Jul 30, 2009 at 6:36 AM, Mike Kienenberger<mkienen..mail.com>
    >>> >>> wrote:
    >>> >>> > Before you make your own custom solution, you might want to read up
    >>> on
    >>> >>> > Javagroup.  It might not be a problem to use it in your environment.
    >>> >>> >
    >>> >>> > The main page starts off with this:
    >>> >>> >
    >>> >>> > http://www.jgroups.org/
    >>> >>> > ==================================
    >>> >>> > JGroups is a toolkit for reliable multicast communication.
    >>> >>> > (Note that this doesn't necessarily mean IP Multicast, JGroups can
    >>> >>> > also use transports such as TCP).
    >>> >>> >
    >>> >>> > [...]
    >>> >>> >
    >>> >>> > JGroups comes with a number of protocols (but anyone can write their
    >>> >>> > own), for example
    >>> >>> >    * Transport protocols: UDP (IP Multicast), TCP, JMS
    >>> >>> >
    >>> >>> > ==================================
    >>> >>> >
    >>> >>> > So even if the TCP version doesn't do what you need, you might find
    >>> it
    >>> >>> > easier to write your own Jgroup protocol than to write your own
    >>> >>> > cayenne event bridge.  It's more likely to be documented and there
    >>> >>> > will be more examples/end users to ask questions of.  There might
    >>> even
    >>> >>> > be a tomcat shared session protocol out there somewhere.
    >>> >>> >
    >>> >>> >
    >>> >>> > On Wed, Jul 29, 2009 at 4:16 PM, Tobias
    >>> >>> > Schoessler<tobias.schoessle..mail.com> wrote:
    >>> >>> >> well i am reading this from the documentation:
    >>> >>> >>
    >>> >>> >> "... At the minimum, JMS setup requires a JMS server running, and
    >>> >>> subjects
    >>> >>> >> for each of the DataDomains to be configured. JavaGroups is
    >>> >>> peer-to-peer
    >>> >>> >> library that is embedded into applications. Default configuration
    >>> >>> provided
    >>> >>> >> by CayenneModeler will work out of the box, provided that IP
    >>> multicast
    >>> >>> is
    >>> >>> >> enabled on the network."
    >>> >>> >>
    >>> >>> >> for the JMS solution the JMS server setup is a problem
    >>> >>> >> for the JavaGroups setup the "IP multicast is enabled on the
    >>> network."
    >>> >>>  is a
    >>> >>> >> problem
    >>> >>> >>
    >>> >>> >> so  for the custom tranport mechanism that you mentioned I stumbled
    >>> >>> upon
    >>> >>> >> this here
    >>> >>> >>
    >>> >>> >>
    >>> >>>
    >>> http://jee-bpel-soa.blogspot.com/2009/06/session-sharing-in-apache-tomcat.html
    >>> >>> >>
    >>> >>> >> which seems to describe cross context data sharing on tomcat web
    >>> >>> contexts
    >>> >>> >>
    >>> >>> >> but is there any code to look at to see how a custom transport
    >>> >>> mechanism can
    >>> >>> >> be setup?
    >>> >>> >>
    >>> >>> >> Tobias
    >>> >>> >>
    >>> >>> >>
    >>> >>> >> On Wed, Jul 29, 2009 at 8:28 PM, Mike Kienenberger <
    >>> mkienen..mail.com
    >>> >>> >wrote:
    >>> >>> >>
    >>> >>> >>> I've never set it up, but it's easily configurable.
    >>> >>> >>>
    >>> >>> >>> If you don't like the javagroups or JMS methodologies, you can
    >>> define
    >>> >>> >>> your own -- I don't know what tomcat app-data-sharing ability is
    >>> >>> >>> available -- it probably depends on the container, but I don't
    >>> >>> >>> remember reading about any in the past.
    >>> >>> >>>
    >>> >>> >>> However, the docs seem to indicate that using Javagroups is pretty
    >>> >>> >>> painless with no external configuration to deal with.
    >>> >>> >>>
    >>> >>> >>> I have a Cayenne 1.1.x application I wrote that used remote
    >>> >>> >>> notification internally to broadcast events between sessions, so I
    >>> >>> >>> know it's not difficult to set up and define your own event
    >>> >>> >>> broadcaster.  My guess is that doing it for javagroups is pretty
    >>> easy
    >>> >>> >>> since it sounds like a matter of just filling in the forms on the
    >>> >>> >>> modeler.
    >>> >>> >>>
    >>> >>> >>> On Wed, Jul 29, 2009 at 2:15 PM, Tobias
    >>> >>> >>> Schoessler<tobias.schoessle..mail.com> wrote:
    >>> >>> >>> > Thanks Mike,
    >>> >>> >>> >
    >>> >>> >>> > so the answer is yes, this can only be done using remote
    >>> >>> notification? is
    >>> >>> >>> > this correct?
    >>> >>> >>> >
    >>> >>> >>> > Isn't there a way to share the cache among two web application
    >>> >>> scopes
    >>> >>> >>> > without going through the hassle of setting up remote
    >>> notification?
    >>> >>> >>> >
    >>> >>> >>> > When the two webapps are running on the same physical machine,
    >>> >>> inside the
    >>> >>> >>> > same application server this seems overkill.
    >>> >>> >>> >
    >>> >>> >>> > Tobias
    >>> >>> >>> >
    >>> >>> >>> > On Wed, Jul 29, 2009 at 6:50 PM, Mike Kienenberger <
    >>> >>> mkienen..mail.com
    >>> >>> >>> >wrote:
    >>> >>> >>> >
    >>> >>> >>> >> Yes,
    >>> >>> >>> >>
    >>> >>> >>> >> Here's a Cayenne 2.0 document on it:
    >>> >>> >>> >>
    >>> >>> >>> >>
    >>> http://cayenne.apache.org/doc20/configuring-caching-behavior.html
    >>> >>> >>> >>
    >>> >>> >>> >> For 3.0:
    >>> >>> >>> >>
    >>> >>> >>> >> http://cayenne.apache.org/doc/configuring-caching-behavior.html
    >>> >>> >>> >>
    >>> >>> >>> >> On Wed, Jul 29, 2009 at 12:46 PM, Tobias
    >>> >>> >>> >> Schoessler<tobias.schoessle..mail.com> wrote:
    >>> >>> >>> >> > Hi,
    >>> >>> >>> >> >
    >>> >>> >>> >> > is it possible to sync the cayenne cache of two web
    >>> applications
    >>> >>> >>> running
    >>> >>> >>> >> in
    >>> >>> >>> >> > the same tomcat?
    >>> >>> >>> >> >
    >>> >>> >>> >> > I observe one web app showing outdated data when the other is
    >>> >>> >>> committing
    >>> >>> >>> >> > updates. Both apps are using the same mapping configuration.
    >>> >>> >>> >> >
    >>> >>> >>> >> > Do I need to use remote notification for this?
    >>> >>> >>> >> >
    >>> >>> >>> >> > thanks
    >>> >>> >>> >> >
    >>> >>> >>> >> > Tobias
    >>> >>> >>> >> >
    >>> >>> >>> >>
    >>> >>> >>> >
    >>> >>> >>>
    >>> >>> >>
    >>> >>> >
    >>> >>>
    >>> >>
    >>> >>
    >>> >
    >>>
    >>
    >



    This archive was generated by hypermail 2.0.0 : Thu Jul 30 2009 - 17:46:47 EDT