Re: Unit Test Slowness [Re: 3.0 M3 test results]

From: Tore Halset (halse..vv.ntnu.no)
Date: Tue Jan 29 2008 - 18:13:25 EST

  • Next message: Andrus Adamchik: "Re: VOTE: 3.0M3 Release"

    Hello.

    I have looked at this, but are not really able to reproduce the
    problem as I do not have access to Oracle. Even if I had, there is 6+
    years since I used Oracle.. Makes me feel old :/

    My first attempt is to reduce the work for DbLoader by specifying a
    the tables that should be tested. I have done this by adding a
    overridable method called includeTableName(String) that defaults to
    true. It is not very elegant, but might work.

    Could you please try the attached patch on oracle Andrus?

    Regards,
      - Tore.

    On Jan 28, 2008, at 23:00, Andrus Adamchik wrote:

    > I finally found time to run unit tests of the M3 tag on various DB's
    > (evaluating release artifacts is next on my list).... I'll post the
    > results in a little bit (so far all good), but I wanted to comment
    > on the merge test case slowness.
    >
    > Because of repeated DB metadata access via DbLoader I noticed a bit
    > of slowdown on MySQL. Then I started testing on Oracle, and this one
    > is a bummer. My unit tests are already running for about 50 minutes
    > with no end in sight... So maybe there is a way to cache the
    > DbLoader calls for the purpose of unit testing? Would be nice if we
    > could optimize that on trunk.

    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/
    cayenne/merge/DbMerger.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/
    cayenne/merge/DbMerger.java (revision 616534)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/
    cayenne/merge/DbMerger.java (working copy)
    ..-53,8 +53,12 @@
      public class DbMerger {

          private MergerFactory factory;
    -
    - public boolean includeTableName(String tableName){
    +
    + /**
    + * A method that return true if the given table name should be
    included. The default
    + * implemntation include all tables.
    + */
    + public boolean includeTableName(String tableName) {
              return true;
          }

    ..-82,7 +86,15 @@
              try {
                  conn = dataSource.getConnection();

    - DbLoader dbLoader = new DbLoader(conn, adapter, new
    LoaderDelegate());
    + final DbMerger merger = this;
    + DbLoader dbLoader = new DbLoader(conn, adapter, new
    LoaderDelegate()) {
    +
    + ..verride
    + public boolean includeTableName(String tableName) {
    + return merger.includeTableName(tableName);
    + }
    + };
    +
                  DataMap detectedDataMap = dbLoader.loadDataMapFromDB(
                          null,
                          null,
    ..-94,7 +106,7 @@
                  for (DbEntity dbEntity : dataMap.getDbEntities()) {
                      String tableName = dbEntity.getName();

    - if(!includeTableName(tableName)){
    + if (!includeTableName(tableName)) {
                          continue;
                      }

    ..-119,7 +131,7 @@
                  // TODO: support drop table. currently, too many tables
    are marked for drop
                  for (DbEntity e : dbEntityToDropByName.values()) {

    - if(!includeTableName(e.getName())){
    + if (!includeTableName(e.getName())) {
                          continue;
                      }

    ..-258,6 +270,11 @@

              // relationships to add
              for (DbRelationship rel : dbEntity.getRelationships()) {
    +
    + if (!includeTableName(rel.getTargetEntityName())) {
    + continue;
    + }
    +
                  if (findDbRelationship(detectedEntity, rel) == null) {
                      // TODO: very ugly. perhaps MergerToken should have
    a .isNoOp()?
                      AbstractToDbToken t = (AbstractToDbToken) factory
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/
    cayenne/access/DbLoader.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/
    cayenne/access/DbLoader.java (revision 616534)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/
    cayenne/access/DbLoader.java (working copy)
    ..-165,6 +165,14 @@
          public DbAdapter getAdapter() {
              return adapter;
          }
    +
    + /**
    + * A method that return true if the given table name should be
    included. The default
    + * implemntation include all tables.
    + */
    + public boolean includeTableName(String tableName) {
    + return true;
    + }

          /**
           * Retrieves catalogues for the database associated with this
    DbLoader.
    ..-284,6 +292,10 @@
                      if (name == null || name.startsWith("BIN$")) {
                          continue;
                      }
    +
    + if (!includeTableName(name)) {
    + continue;
    + }

                      Table info = new Table(cat, schema, name);
                      tables.add(info);
    ..-572,7 +584,11 @@
                          // start new entity
                          String fkEntityName =
    rs.getString("FKTABLE_NAME");
                          String fkName = rs.getString("FK_NAME");
    -
    +
    + if (!includeTableName(fkEntityName)) {
    + continue;
    + }
    +
                          fkEntity = map.getDbEntity(fkEntityName);

                          if (fkEntity == null) {



    This archive was generated by hypermail 2.0.0 : Tue Jan 29 2008 - 18:11:51 EST