Re: Paging and SQL queries

From: Andrus Adamchik (andru..bjectstyle.org)
Date: Mon May 28 2007 - 16:25:29 EDT

  • Next message: Andrus Adamchik: "Modeling interfaces [ Was: Modelling improvements: inheritance + interfacing (Draft)]"

    On May 28, 2007, at 1:50 PM, Aristedes Maniatis wrote:

    >
    > On 28/05/2007, at 4:49 PM, Andrus Adamchik wrote:
    >
    >> No. "resolvesFirstPage" really a shorthand for "is it possible to
    >> reliably resolve objects on the first page using the existing
    >> algorithm".
    >
    > OK. I guess I would have named it isFirstPageResolvable() or
    > canResolveFirstPage(). But from your description it appears that it
    > isn't specific to the first page and that either it is resolvable
    > for any page, or not. So does it mean canResolveData()?

    Actually no. This is specific to the first page that most of the time
    can be resolved from the "select *" query result set, while the
    second and following pages are resolved using separate queries that
    select a group of id's.

    >> Just to confirm, is this a JDBC test or a test using DB client
    >> tools (on Derby this may be the same thing, still worth doing a
    >> JDBC run with your own code, as there can be unrelated bottlenecks
    >> in the tools). E.g. create two PreparedStatements, one with
    >> "select *", another with "select id", and only read "id" column in
    >> both resultsets. See how much faster the second case is.
    >
    > OK. I just tried it a different way and the results are the same.
    > In this case I used the Derby ij tool to make a JDBC connection to
    > a Derby server on localhost. That server has real data from one of
    > our clients with 120,000 students.
    >
    > select * from student; -> 53 seconds
    > select id from student; -> 4 seconds
    >
    > I performed several runs to try and exclude cache warming and the
    > variance was less than 10%. The output was redirected to a file on
    > disk. There are around 30 columns in the student table with a range
    > of data types. The output file on disk with all the fields was
    > 120Mb when finished.

    Could you also exclude "ij" from the picture? It can add the overhead
    of the terminal I/O, and the overhead of actually reading those extra
    29 columns - something Cayenne paginated query wouldn't do. What I
    think of as a *clean test* in this situation is a small Java program
    that opens a Connection, creates a PreparedStatement, and reads a
    ResultSet, discarding the data. In case of "select *" it would still
    only read the "id" column.

    Thanks
    Andrus

      
      



    This archive was generated by hypermail 2.0.0 : Mon May 28 2007 - 16:26:03 EDT