Queries use memory proportional to list length

From: Øyvind Harboe (oyvind.harbo..ylin.com)
Date: Thu Jun 26 2008 - 07:19:48 EDT

  • Next message: Øyvind Harboe: "Memory usage and select statements"

    Queries use memory proportional to the number of records returned
    by the query.

    I expected paged queries to use memory proportional to the page
    size, but paged queries require as much memory as normal queries.

    I thought the "fix" was to modify paged queries not to lock the
    entire list of objects into memory, hence I opened this issue:

    https://issues.apache.org/cayenne/browse/CAY-1075

    However, having implemented the attached LinearList which
    allows only sequential access to a query result with memory usage
    proportional to the page size, I think perhaps that it is better
    to let Cayenne allow easy implementation of application specific
    handling of this problem.

    Once memory consumption becomes a problem, there will
    be a large number of records involved. The application will
    have precise knowledge of the access pattern to the list
    and rather than having Cayenne guess what sort of strategy
    will work out best w.r.t. performance (speed + minimize
    memory consumption), why not let the application implement
    such query lists itself?

    Documenting the limitations of the attached LinearList is
    a pretty messy task, not something that I'd want to
    document & support the rest of my life.

    Cayenne can of course make it easier to implement such
    lists by many means, e.g.:

    - add a Wiki page
    - add a copy & paste & modify class
    - add a class that is intended for subclassing to handle
    application specific access patterns

    -- 
    Øyvind Harboe
    http://www.zylin.com/zy1000.html
    ARM7 ARM9 XScale Cortex
    JTAG debugger and flash programmer
    



    This archive was generated by hypermail 2.0.0 : Thu Jun 26 2008 - 07:20:23 EDT