I don't know if it is of any interest concerning this matter but the only thing that is probably slightly unusual is this.
I am using a few singletons to cache some DataObjects which I use frequently. I don't want to use a SelectQuery every time.
The singleton has a HashMap. Every 10 minutes a new DataContext is created. I do a SelectQuery and put all the found Objects into the HashMap. The old DataObjects are overwritten so should be destroyed by the GC.
When I need one of the objects I get it out of the HashMap and use DataContext.localObject() to create a new version of the DataObject within the given DataContext.
Here is the code of one of the singletons:
public class UnitPool implements Runnable, Serializable {
private static UnitPool instance = null;
private HashMap<String,Unit> units;
private UnitPool() {
units = new HashMap<String,Unit>();
loadMap();
}
private void loadMap() {
DataContext context = DataContext.createDataContext();
SelectQuery query = new SelectQuery(Unit.class);
List list = context.performQuery(query);
Iterator iter = list.iterator();
while ( iter.hasNext() ) {
Unit unit = (Unit)iter.next();
units.put(unit.getName(), unit);
}
}
public static UnitPool getInstance() {
if ( instance == null ) {
instance = new UnitPool();
Thread thread = new Thread(instance);
thread.start();
}
return instance;
}
public Unit getUnit(String unitName) {
return (Unit)units.get(unitName);
}
public Unit getUnit(DataContext context, String unitName) {
Unit unit = (Unit)units.get(unitName);
if ( unit == null ) {
return null;
}
List<Unit> list = new ArrayList<Unit>();
list.add(unit);
return (Unit)context.localObjects(list).get(0);
}
public void run() {
while ( true ) {
try {
Thread.sleep(10 * 60 * 1000);
loadMap();
} catch (InterruptedException e) {
}
}
}
}
When I want to use one of the units I can just use
UnitPool.getInstance().getUnit(context, "xxx");
JAVA: 1.5.0_06-b05
Cayenne: 1.2.1
Tomcat: 5.5.16
Axis: 1.3
Thanks
ayhan
> -----Ursprüngliche Nachricht-----
> Von: Andrus Adamchik [mailto:andru..bjectstyle.org]
> Gesendet: Mittwoch, 21. Februar 2007 15:20
> An: use..ayenne.apache.org
> Betreff: Re: AW: AW: possible bug / memory leak in DispatchQueue and
> EventManager?
>
> 30 is good. If you had a retained DataContext somewhere you'd get the
> same number as the number of stuck invocations.
>
> 1 million of Invocations is bad.
>
> I guess this needs to be logged as a possible bug to investigate,
> with as many details as possible on the JVM version, etc.
>
> Andrus
>
>
> On Feb 21, 2007, at 4:07 PM, Ayhan Kondoz wrote:
> > Yes seems live there are DataContext instances.
> >
> > http://img73.imageshack.us/img73/3068/memta6.jpg
> >
> > I started the GC a few times before I took this screenshot.
> > As you can see there are still 30 DataContext instances. No idea
> > why thought...
> >
> >
> >> -----Ursprüngliche Nachricht-----
> >> Von: Andrus Adamchik [mailto:andru..bjectstyle.org]
> >> Gesendet: Mittwoch, 21. Februar 2007 14:49
> >> An: use..ayenne.apache.org
> >> Betreff: Re: AW: possible bug / memory leak in DispatchQueue and
> >> EventManager?
> >>
> >> Do you see DataContext instances in the memory profile? I wonder how
> >> many of those do you have, as those are the only Cayenne objects that
> >> have hard reference to the listeners.
> >>
> >> Andrus
> >>
> >>
> >> On Feb 21, 2007, at 3:37 PM, Ayhan Kondoz wrote:
> >>
> >>> Hi,
> >>>
> >>> the JVM has 1 GB of memory and i already tried to run the GC
> >>> manually but it doesn't make a difference. I can start the GC from
> >>> the profiler tool but the number of instances does not change.
> >>> I guess that there are some other references to this objects so
> >>> that the weak ref. does not expire.
> >>>
> >>> And there are no EventManager exception in the log files.
> >>>
> >>>
> >>> ayhan
> >>>
> >>>> -----Ursprüngliche Nachricht-----
> >>>> Von: Andrus Adamchik [mailto:andru..bjectstyle.org]
> >>>> Gesendet: Mittwoch, 21. Februar 2007 14:17
> >>>> An: use..ayenne.apache.org
> >>>> Betreff: Re: possible bug / memory leak in DispatchQueue and
> >>>> EventManager?
> >>>>
> >>>> Seems like your assessment of the EventManager leaking is correct.
> >>>>
> >>>> Now the cause is not that clear. A shot in the dark - this is
> >>>> due to
> >>>> a combination of lots of spare memory in JVM (so weak references
> >>>> are
> >>>> not collected fast enough) and slow custom 'equals' and 'hashCode'
> >>>> methods in invocation.
> >>>>
> >>>> How much heap size do you have in your server? Also can you try to
> >>>> add a request filter that would do 'System.gc()' on every few
> >>>> thousands requests, and see if it makes any difference.
> >>>>
> >>>> Also - do you see any EventManager exceptions in the logs?
> >>>>
> >>>> Andrus
> >>>>
> >>>>
> >>>> On Feb 21, 2007, at 10:55 AM, Ayhan Kondoz wrote:
> >>>>> Hi,
> >>>>>
> >>>>>
> >>>>>
> >>>>> i think i found a possible bug / memory leak within the
> >>>>> DispatchQueue and EventManager.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> First a little bit about my setup:
> >>>>>
> >>>>>
> >>>>>
> >>>>> I have 3 servers. Each server runs an axis service. The service
> >>>>> uses cayenne 1.2.1 to connect to a database. It reads customer and
> >>>>> account information from the DB etc..
> >>>>>
> >>>>> The servers are using cayenne's shared caching with javagroups as
> >>>>> the messaging service so that changes made from one server are
> >>>>> dispatched to the other servers.
> >>>>>
> >>>>>
> >>>>>
> >>>>> The avarage connections per second is somewhere around 4-5.
> >>>>>
> >>>>>
> >>>>>
> >>>>> However I have a very strange problem with this setup so I startet
> >>>>> to search for the reason. The problem is that with nearly constant
> >>>>> and unchanging usage the load of each server increases over time.
> >>>>> To further test this I created a test server with a similar setup.
> >>>>> There I created a test program that creates totally constant
> >>>>> usage.
> >>>>> But even with the unchanging usage the load of the server is
> >>>>> increasing until the cpu load is so high that the requests can not
> >>>>> be processed anymore.
> >>>>>
> >>>>>
> >>>>>
> >>>>> I installed a java profiler to trying to pinpoint the location of
> >>>>> this error and this is what I found out.
> >>>>>
> >>>>>
> >>>>>
> >>>>> I let the server run for 24 hours and then stopped the program
> >>>>> which creates the test usage.
> >>>>>
> >>>>> But even while the server was idle there where still a lot of
> >>>>> instances in the java heap after the GC run.
> >>>>>
> >>>>>
> >>>>>
> >>>>> http://img206.imageshack.us/img206/9769/memorysy5.jpg
> >>>>>
> >>>>>
> >>>>>
> >>>>> Please note the HashMap, WeakReference and Invocation counts. I
> >>>>> pressume that the $ObjectProvider_*** is cayenne aswell but I am
> >>>>> not sure.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Now the following image shows the cpu profile with incoming
> >>>>> connections.
> >>>>>
> >>>>>
> >>>>>
> >>>>> http://img339.imageshack.us/img339/2441/cpuci0.jpg
> >>>>>
> >>>>>
> >>>>>
> >>>>> As you can see 58% of the cpu time is used within HashSet.add().
> >>>>>
> >>>>>
> >>>>>
> >>>>> So when I consider the two facts i think that there might be a
> >>>>> possible problem with the EventManager. The first table tells us
> >>>>> that there are over 1 million instances of HashSet's and cayenne
> >>>>> Invocations. So it seems like the set's within the DispatchQueue
> >>>>> are not recycled properly so that the object count rises over time
> >>>>> which result in extremly high processtime when trying to add new
> >>>>> HashSet's.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>> Ayhan
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Ayhan Kondoz
> >>>>>
> >>>>>
> >>>>>
> >>>>> Software-Entwicklung
> >>>>>
> >>>>>
> >>>>>
> >>>>> ------------------------------------------------------------------
> >>>>> --
> >>>>> --
> >>>>> ------------
> >>>>>
> >>>>> Telefon: +49 (0) 40 513 06 616
> >>>>>
> >>>>> Telefax: +49 (0) 40 513 06 998 616
> >>>>>
> >>>>> E-Mail: ayhan.kondo..reenet-ag.de
> >>>>> <mailto:ayhan.kondo..reenet-
> >>>>> ag.de>
> >>>>>
> >>>>> Website: http://www.freenet.de <http://www.freenet.de/> ; http://
> >>>>> www.freenet-ag.de <http://www.freenet-ag.de/>
> >>>>>
> >>>>> ------------------------------------------------------------------
> >>>>> --
> >>>>> --
> >>>>> ------------
> >>>>>
> >>>>> freenet.de AG
> >>>>>
> >>>>> Deelbögenkamp 4c
> >>>>>
> >>>>> 22297 Hamburg
> >>>>>
> >>>>> ------------------------------------------------------------------
> >>>>> --
> >>>>> --
> >>>>> ------------
> >>>>>
> >>>>> Vorsitzender des Aufsichtsrates: Prof. Dr. Helmut Thoma
> >>>>>
> >>>>> Vorstand: Eckhard Spoerr (Vors.),
> >>>>> Axel Krieger, Stephan Esch, Eric Berger
> >>>>>
> >>>>> Amtsgericht Hamburg HRB 74048
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Diese Information ist ausschließlich für die adressierte Person
> >>>>> oder Organisation bestimmt und könnte vertrauliches und/oder
> >>>>> privilegiertes Material enthalten. Personen oder Organisationen,
> >>>>> für die diese Information nicht bestimmt ist, ist es nicht
> >>>>> gestattet, diese zu lesen, erneut zu übertragen, zu verbreiten,
> >>>>> anderweitig zu verwenden oder sich durch sie veranlasst zu sehen,
> >>>>> Maßnahmen irgendeiner Art zu ergreifen. Sollten Sie diese
> >>>>> Nachricht
> >>>>> irrtümlich erhalten haben, bitten wir Sie, sich mit dem
> >>>>> Absender in
> >>>>> Verbindung zu setzen und das Material von Ihrem Computer zu
> >>>>> löschen.
> >>>>>
> >>>>>
> >>>>>
> >>>>> The information transmitted is intended only for the person or
> >>>>> entity to which it is addressed and may contain confidential
> >>>>> and/or
> >>>>> privileged material. Any review, retransmission, dissemination or
> >>>>> other use of, or taking of any action in reliance upon, this
> >>>>> information by persons or entities other than the intended
> >>>>> recipient is prohibited. If you received this in error, please
> >>>>> contact the sender and delete the material from any computer.
> >>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >
> >
This archive was generated by hypermail 2.0.0 : Wed Feb 21 2007 - 09:40:35 EST
