Re: Memory Management using Tomcat

From: Mike Kienenberger (mkienen..mail.com)
Date: Wed Sep 16 2009 - 15:37:17 EDT

  • Next message: Joe Baldwin: "Re: Memory Management using Tomcat"

    I think there's a default filter provided by Cayenne you can specify
    in your config file, but it really comes down to something as simple
    as this to make it per request:

        public void doFilter(ServletRequest servletRequest,
    ServletResponse servletResponse, FilterChain chain)
              throws IOException, ServletException
        {
            // set base context on thread

            chain.doFilter(servletRequest, servletResponse);

            // remove base context from thread
        }

    Attaching a simple one I wrote a long time ago that perserves the
    DataContext across a session (but insures it's in a clean state at the
    end of each request).

    On Wed, Sep 16, 2009 at 3:29 PM, Joe Baldwin <jfbaldwi..arthlink.net> wrote:
    > Mike,
    >
    > RE BaseContext.getThreadObjectContext()
    >
    > Of course, this could be my problem. I was using DataContext until 3.0 then
    > converted over to BaseContext.
    >
    >
    >> 2.)  BaseContext.getThreadObjectContext() just tells how you're
    >> getting a context.  It doesn't tell how it's managed.   Do you have a
    >> servlet filter that creates a new ObjectContext at the start of a
    >> request and then clears it out at the end?   Or does it do this per
    >> session?
    >
    > I do not know how it is being initialized in the webapp (in my experimental
    > non-web apps I explicitly intitialize it, but in the webapp it is already
    > initialized).
    >
    >
    >>  I've never looked at the method, but it might default to
    >> creating one permanent data context per thread if you don't do
    >> anything else (like set up a servlet filter).   That could be part of
    >> the problem.
    >
    >
    > That sounds plausible.  I could not find an example of how to initialize
    > this and manage it as you suggest in a web app.  I have not created a
    > servlet filter but have follow the Cayenne docs for configuration of the
    > web.xml.
    >
    > If this is insufficient then I agree, this could be the problem.
    >  Unfortunately, I have not found docs on how to accomplish what you are
    > recommending.
    >
    > Joe
    >
    >
    >
    >
    > On Sep 16, 2009, at 3:10 PM, Mike Kienenberger wrote:
    >
    >> 1.)  128 still seems small to me.   I don't think I run anything at
    >> less than 256.
    >> On the other hand, We have an app with 1000s of customers that uses
    >> 512Mb, I think.   So 1500 seems excessive.
    >>
    >> 2.)  BaseContext.getThreadObjectContext() just tells how you're
    >> getting a context.  It doesn't tell how it's managed.   Do you have a
    >> servlet filter that creates a new ObjectContext at the start of a
    >> request and then clears it out at the end?   Or does it do this per
    >> session?   I've never looked at the method, but it might default to
    >> creating one permanent data context per thread if you don't do
    >> anything else (like set up a servlet filter).   That could be part of
    >> the problem.
    >>
    >> On Wed, Sep 16, 2009 at 2:49 PM, Joe Baldwin <jfbaldwi..arthlink.net>
    >> wrote:
    >>>
    >>> Caveat: Apparently I am not as well. :)
    >>>
    >>> 1.) I looked at the 65M issue.  On my development box (OSX) I set it to
    >>> -Xms128m -Xmx128m (basically arbitrary).  So when I went to the remote
    >>> hosting company, I purchase a similar amount.
    >>>       a. However, we are doing *very* little work (lots of product
    >>> fetches
    >>> and only a few product inserts and updates) and it runs out of memory
    >>> *very*
    >>> fast which I *assume* means it is my code, but I don't know.
    >>>       b. I am doing research and here is a "web recommendation" (for all
    >>> that is worth)
    >>>               Whenever possible, Unidata recommends
    >>>       -Xmx1500m for 32-bit systems, and -Xmx2048m --Xmx4096m for 64-bit
    >>> systems.
    >>>       c. Since the host is 64-bit, I am wondering whether my assumptions
    >>> may be off for 64-bit systems.
    >>>
    >>> 2. DataContext: Sorry, but I am getting confused on this one.  I am using
    >>> BaseContext.getThreadObjectContext() based on recommendations (I
    >>> converted
    >>> all the old DataContext refs to BaseContext, but I don't really
    >>> understand
    >>> it from reading the docs) and am *not* releasing it at the end of
    >>> session.
    >>>  Not quite sure of how to do this properly.
    >>>
    >>> 3. Don't know how to set the cache to retain N number of objects. I
    >>> experimented with
    >>>       query.setPageSize(RowsPerPage);
    >>>       query.setCacheStrategy(QueryCacheStrategy.SHARED_CACHE);
    >>>       query.setCacheGroups("product", "ProductList");
    >>>
    >>> This seemed to help quite a bit but I still eventually ran out of memory.
    >>>  I
    >>> recently removed *all* the SHARED_CACHE and it ran out of memory very
    >>> fast.
    >>>
    >>> Thanks for your input
    >>> Joe
    >>>
    >>>
    >>>
    >>> On Sep 16, 2009, at 2:25 PM, Mike Kienenberger wrote:
    >>>
    >>>> Caveat: I'm not really an expert on Cayenne memory management.
    >>>>
    >>>> 1) Are you allocating enough heap memory to the app server to start
    >>>> with?   I don't know what the default is these days, but in the old
    >>>> days, an application by default only gets 64Mb of memory -- that's
    >>>> pretty small.
    >>>>
    >>>> 2) Are you using a new DataContext per request?  Or at least per
    >>>> session?
    >>>>
    >>>> 3) I seem to remember that the cache strategy is configurable.  Have
    >>>> you configured a cache that only retains N number of objects for a
    >>>> suitable value of N?
    >>>>
    >>>> On Wed, Sep 16, 2009 at 2:11 PM, Joe Baldwin <jfbaldwi..arthlink.net>
    >>>> wrote:
    >>>>>
    >>>>> Hi,
    >>>>>
    >>>>> I have asked this question a number of ways but I still have a very
    >>>>> serious
    >>>>> problem with Cayenne-specific memory management configuration
    >>>>> associated
    >>>>> with Tomcat.
    >>>>>
    >>>>> The problem with debugging is that given that I have very little
    >>>>> visibility
    >>>>> into the Cayenne memory management it is extremely difficult to debug
    >>>>> this
    >>>>> using conventional strategies.  Also I am not requesting anything from
    >>>>> the
    >>>>> system that is terribly exceptional so I am attempting to use default
    >>>>> settings as much as possible.
    >>>>>
    >>>>> My strategy is to ask the experts for a Cayenne configuration and
    >>>>> standard
    >>>>> memory management steps I should take to conform to the new Cayenne
    >>>>> memory
    >>>>> management design intentions.
    >>>>>
    >>>>> Problem:
    >>>>> 1. I have essentially a webstore, three tier design with Tomcat,
    >>>>> Cayenne
    >>>>> and
    >>>>> MySQL.
    >>>>> 2. When after only a few queries of products, tomcat freezes up and
    >>>>> reports
    >>>>> out of memory errors.
    >>>>>
    >>>>> I have attempted to configure the caching strategy ask best as I can
    >>>>> understand from the docks but this only gets me a few more hours of
    >>>>> usage
    >>>>> before the out of memory errors.  (I tried the SHARED_CACHE). The
    >>>>> NO_CACHE
    >>>>> strategy is worse.
    >>>>>
    >>>>> I would appreciate a set of steps (aka a primer) that should handle a
    >>>>> website with a lot of fetches of hundreds of data objects (i.e.
    >>>>> products)
    >>>>> and very few updates.
    >>>>>
    >>>>> Note: My gut feeling is that I am not properly managing the data object
    >>>>> array properly and it is leaking memory.
    >>>>>
    >>>>> I would appreciate any input, but I would first like to know what the
    >>>>> minimum require steps are for managing at data object result set
    >>>>> ArrayList
    >>>>> so as to properly cache and then properly free the memory after it is
    >>>>> no
    >>>>> longer needed.
    >>>>>
    >>>>> Context: Tomcat, MySQL, Cayenne 3.0M6
    >>>>>
    >>>>> Thanks,
    >>>>> Joe Baldwin
    >>>>>
    >>>>>
    >>>
    >>>
    >
    >





    This archive was generated by hypermail 2.0.0 : Wed Sep 16 2009 - 15:38:13 EDT