Scaling paginated list
----------------------
Key: CAY-999
URL: https://issues.apache.org/cayenne/browse/CAY-999
Project: Cayenne
Issue Type: Improvement
Components: Cayenne Core Library
Affects Versions: 3.0
Reporter: Andrus Adamchik
Assignee: Andrus Adamchik
An idea for scaling IncrementalFaultList to store massive amount of objects, like hundreds of thousands.This pertains to the server-side IncrementalFaultList. The problems to solve are the speed of the initial list initialization and overall memory use.
1. Simplify ID representation:
Even unresolved lists can take significant amount of memory... Each unresolved object slot currently stores a DataRow with N number of entries, where N is the number of PK columns for the entity. I.e. most often than not - 1 entry. Here is a memory use calculation for various representations of an unresolved entry, based on a single int PK DbEntity.
a. DataRow - 120 bytes,
b. HashMap - 104 bytes,
c. Object[] - 32 bytes,
d java.lang.Integer - 16 bytes
[primitive int is even better, but it complicates the implementation, as we'd need a parallel int[] (long[], double[], etc.) , so all in all we may get no gain]
2. Swap out LRU pages
For the very large lists, it would make sense to un-fault resolved pages when more pages are resolved , so that the list size doesn't grow beyond a certain fixed amount, no matter how many pages are resolved. These parameters will have to be configurable per query, as some users would prefer to keep the entire thing...
-- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
This archive was generated by hypermail 2.0.0 : Wed Mar 05 2008 - 07:44:31 EST