Re: Big selects on PostGres : Configuring Statement.setFetchSize() of a selectquery

From: Andrus Adamchik (andru..bjectstyle.org)
Date: Fri May 15 2009 - 15:08:49 EDT

  • Next message: stefcl: "Re: Big selects on PostGres : Configuring Statement.setFetchSize() of a selectquery"

    I haven't experimented with that much (so no idea which drivers take
    advantage of that and which would simply ignore it), but that would
    indeed be a useful parameter for ResultIterator scenario. So maybe
    open a feature request in Jira?

    Andrus

    On May 15, 2009, at 5:45 PM, stefcl wrote:

    >
    > Thanks,
    >
    > Since the beginning I am convinced that resultiterators are the
    > right way,
    > unfortunately they are useless if the jdbc driver loads everything
    > in memory
    > before giving access to the resultset.
    >
    > I hope there's something to do about it because it's a showstopper
    > for me.
    >
    >
    > Robert Zeigler-6 wrote:
    >>
    >> Take a look at paginated queries (which isn't /quite/ what you want,
    >> but may help you):
    >>
    >> http://cayenne.apache.org/doc/paginated-queries.html
    >>
    >> As well as ResultIterator:
    >> http://cayenne.apache.org/doc/iterating-through-data-rows.html
    >>
    >> Robert
    >>
    >> On May 15, 2009, at 5/159:21 AM , stefcl wrote:
    >>
    >>>
    >>>
    >>> Thanks but my problem is not related to paging.
    >>>
    >>> I would like to be able to execute a select query which returns
    >>> approx
    >>> 100000 rows with a single selectQuery and process them one by one,
    >>> while
    >>> keeping only a few of them in memory (using the resultiterator).
    >>>
    >>> In jdbc, Statement.setFetchSize(1000) tells the jdbc driver to
    >>> retrieve
    >>> results from the database 1000 at a time and ask for the next 1000
    >>> as you're
    >>> iterating the resultset. Otherwise its default behavior is to
    >>> retrieve the
    >>> 100'000 rows in memory before you can start iterating the resultset.
    >>>
    >>> It's not the same thing as inserting a TOP or a LIMIT clause in the
    >>> query.
    >>>
    >>> Any help appreciated
    >>>
    >>>
    >>> Andrey Razumovsky wrote:
    >>>>
    >>>> Hi,
    >>>>
    >>>> Yes of course. You can use SelectQuery.setFetchLimit(int) and
    >>>> setFetchStart(int) methods. They do just that.
    >>>>
    >>>> Andrey
    >>>>
    >>>
    >>> --
    >>> View this message in context:
    >>> http://www.nabble.com/Big-selects-on-PostGres-%3A-Configuring-Statement.setFetchSize%28%29-of-a-selectquery-tp23555842p23560831.html
    >>> Sent from the Cayenne - User mailing list archive at Nabble.com.
    >>>
    >>
    >>
    >>
    >
    > --
    > View this message in context: http://www.nabble.com/Big-selects-on-PostGres-%3A-Configuring-Statement.setFetchSize%28%29-of-a-selectquery-tp23555842p23561312.html
    > Sent from the Cayenne - User mailing list archive at Nabble.com.
    >
    >



    This archive was generated by hypermail 2.0.0 : Fri May 15 2009 - 15:09:26 EDT