yeah, I am all for using multiversion cc as much as possible.
> Do we need to lock the DataRowStore for the duration of the commit?
> Would it be possible to only update the cache after the db-commit
> are finished and let the DataRowStore return old data while
> committing?
Good question. Maybe I am still thinking in terms of Cayenne 1.1
where the lock was absolutely needed.
On commit lock spans the following chain of operations: update query
preparation -> DB commit -> cache postprocessing. Maybe we can reduce
the scope to just the last step, resulting in some fuzziness in the
baseline snapshot versions.
Are there any brave souls with high volume update-heavy sites to try
this modification in production to see how much damage it causes?
Tore, do you want to try it in your app? I am asking cause it may
take me a while to do thorough stress testing, although just moving
the lock is trivial.
> What are the status of "I think we will be changing cache format
> from DataRows (that cache DB data) to some kind of object property-
> based "rows""?
> http://www.objectstyle.org/cayenne/lists/cayenne-devel/
> 2006/04/0248.html
haven't started to look at it yet, but this is something worth
investigating on the long run.
Andrus
On Dec 12, 2006, at 3:55 PM, Tore Halset wrote:
> Hello.
>
> I have moved the thread to cayenne-dev.
>
> This is interesting. It reminds me of the different ways to
> concurrency control in a database :) PostgreSQL MVCC being one of
> the best.
> http://www.postgresql.org/docs/current/static/mvcc-intro.html
>
> Do we need to lock the DataRowStore for the duration of the commit?
> Would it be possible to only update the cache after the db-commit
> are finished and let the DataRowStore return old data while
> committing?
>
> What are the status of "I think we will be changing cache format
> from DataRows (that cache DB data) to some kind of object property-
> based "rows""?
> http://www.objectstyle.org/cayenne/lists/cayenne-devel/
> 2006/04/0248.html
>
> - Tore.
>
> On Dec 12, 2006, at 13:21 , Andrus Adamchik wrote:
>
>> * On commit, DRS is locked for the duration of the query execution
>> (DataDomainFlushAction.flush())
>> * On select, DRS is locked AFTER the query for the duration of the
>> row-to-object conversion
>>
>> So yes, slow commits will block object queries.
>>
>> Hmm... I wonder if we can do cache update on select in a thread
>> separate from the calling thread (via EventManager?), and avoid
>> locking during object creation..? This should generally improve
>> throughput. Something to think about.
>>
>> Alternatively for slow commits you can try to use a special
>> DataContext that uses its own local instance of DataRowStore
>> (passed in ObjectStore constructor), thus avoiding a shared lock.
>>
>> Andrus
>>
>>
>> On Dec 12, 2006, at 1:48 PM, Tore Halset wrote:
>>> Hello.
>>>
>>> One of our applications blocked for some time today. It worked
>>> out fine after a couple of minutes. It is running cayenne-1.2. We
>>> will upgrade to 2.0.1 in the start of january. The attached
>>> stacktraces are extracted using JBoss jmx-console.
>>>
>>> Thread http-0.0.0.0-80-6 are performing a slow
>>> DataContext.commitChanges. It locks our MS SQL Server for some
>>> minutes. I guess we will be able to optimize this by deleting
>>> some old records or creating a index.
>>>
>>> Some other threads are performing simple queries, but are blocked
>>> by a lock in DataRowStore. Are all (other) queries blocked while
>>> doing a commitChanges? I have seen some discussion around locks
>>> in DataRowStore lately, but I do not know if this is related.
>>>
>>> - Tore.
>>>
>>> Thread: http-0.0.0.0-80-3 : priority:5, demon:true, threadId:38,
>>> threadState:BLOCKED,
>>> threadLockName:org.objectstyle.cayenne.access.DataRowStor..a2ae9
>>> org.objectstyle.cayenne.access.DataRowStore.getCachedSnapshot
>>> (DataRowStore.java:387)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.interceptOIDQue
>>> ry(DataDomainQueryAction.java:164)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.execute
>>> (DataDomainQueryAction.java:142)
>>> org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java:
>>> 766)
>>> org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery
>>> (ObjectContextQueryAction.java:253)
>>> org.objectstyle.cayenne.access.DataContextQueryAction.execute
>>> (DataContextQueryAction.java:90)
>>> org.objectstyle.cayenne.access.DataContext.onQuery
>>> (DataContext.java:1422)
>>> org.objectstyle.cayenne.access.DataContext.performQuery
>>> (DataContext.java:1411)
>>> org.objectstyle.cayenne.DataObjectUtils.objectForQuery
>>> (DataObjectUtils.java:290)
>>> org.objectstyle.cayenne.DataObjectUtils.objectForPK
>>> (DataObjectUtils.java:276)
>>> org.objectstyle.cayenne.DataObjectUtils.objectForPK
>>> (DataObjectUtils.java:185)
>>>
>>> Thread: http-0.0.0.0-443-2 : priority:5, demon:true, threadId:47,
>>> threadState:BLOCKED,
>>> threadLockName:org.objectstyle.cayenne.access.DataRowStor..a2ae9
>>> org.objectstyle.cayenne.access.ObjectResolver.synchronizedObjectsFro
>>> mDataRows(ObjectResolver.java:133)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.interceptObject
>>> Conversion(DataDomainQueryAction.java:373)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.execute
>>> (DataDomainQueryAction.java:151)
>>> org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java:
>>> 766)
>>> org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery
>>> (ObjectContextQueryAction.java:253)
>>> org.objectstyle.cayenne.access.DataContextQueryAction.execute
>>> (DataContextQueryAction.java:90)
>>> org.objectstyle.cayenne.access.DataContext.onQuery
>>> (DataContext.java:1422)
>>> org.objectstyle.cayenne.access.DataContext.performQuery
>>> (DataContext.java:1411)
>>> org.objectstyle.cayenne.access.ToManyList.resolvedObjectList
>>> (ToManyList.java:343)
>>> org.objectstyle.cayenne.access.ToManyList.size(ToManyList.java:296)
>>>
>>> Thread: EventDispatchThread-0 : priority:5, demon:true, threadId:
>>> 54, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..2cd2e
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: EventDispatchThread-1 : priority:5, demon:true, threadId:
>>> 55, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..2cd2e
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: EventDispatchThread-2 : priority:5, demon:true, threadId:
>>> 56, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..2cd2e
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: EventDispatchThread-3 : priority:5, demon:true, threadId:
>>> 57, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..2cd2e
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: EventDispatchThread-4 : priority:5, demon:true, threadId:
>>> 58, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..2cd2e
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: EventDispatchThread-0 : priority:5, demon:true, threadId:
>>> 59, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..9bfa
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: EventDispatchThread-1 : priority:5, demon:true, threadId:
>>> 60, threadState:WAITING, threadLockName:java.util.Collections
>>> $SynchronizedLis..9bfa
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:474)
>>> org.objectstyle.cayenne.event.EventManager$DispatchThread.run
>>> (EventManager.java:486)
>>>
>>> Thread: http-0.0.0.0-443-15 : priority:5, demon:true, threadId:
>>> 104, threadState:BLOCKED,
>>> threadLockName:org.objectstyle.cayenne.access.ToManyLis..77348f
>>> org.objectstyle.cayenne.access.ToManyList.resolvedObjectList
>>> (ToManyList.java:332)
>>> org.objectstyle.cayenne.access.ToManyList.size(ToManyList.java:296)
>>>
>>> Thread: http-0.0.0.0-80-6 : priority:5, demon:true, threadId:113,
>>> threadState:RUNNABLE, threadLockName:null
>>> java.net.SocketInputStream.socketRead0(Native Method)
>>> java.net.SocketInputStream.read(SocketInputStream.java:129)
>>> java.io.DataInputStream.readFully(DataInputStream.java:176)
>>> java.io.DataInputStream.readFully(DataInputStream.java:152)
>>> net.sourceforge.jtds.jdbc.SharedSocket.readPacket
>>> (SharedSocket.java:814)
>>> net.sourceforge.jtds.jdbc.SharedSocket.getNetPacket
>>> (SharedSocket.java:695)
>>> net.sourceforge.jtds.jdbc.ResponseStream.getPacket
>>> (ResponseStream.java:443)
>>> net.sourceforge.jtds.jdbc.ResponseStream.read(ResponseStream.java:
>>> 102)
>>> net.sourceforge.jtds.jdbc.ResponseStream.peek(ResponseStream.java:
>>> 87)
>>> net.sourceforge.jtds.jdbc.TdsCore.wait(TdsCore.java:3772)
>>> net.sourceforge.jtds.jdbc.TdsCore.executeSQL(TdsCore.java:1031)
>>> net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL
>>> (JtdsStatement.java:436)
>>> net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate
>>> (JtdsPreparedStatement.java:402)
>>> org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeUpda
>>> te(WrappedPreparedStatement.java:227)
>>> org.objectstyle.cayenne.access.jdbc.BatchAction.runAsIndividualQueri
>>> es(BatchAction.java:224)
>>> org.objectstyle.cayenne.access.jdbc.BatchAction.performAction
>>> (BatchAction.java:117)
>>> org.objectstyle.cayenne.dba.sqlserver.SQLServerBatchAction.performAc
>>> tion(SQLServerBatchAction.java:95)
>>> org.objectstyle.cayenne.access.DataNodeQueryAction.runQuery
>>> (DataNodeQueryAction.java:95)
>>> org.objectstyle.cayenne.access.DataNode.performQueries
>>> (DataNode.java:309)
>>> org.objectstyle.cayenne.access.DataDomainFlushAction.runQueries
>>> (DataDomainFlushAction.java:255)
>>> org.objectstyle.cayenne.access.DataDomainFlushAction.flush
>>> (DataDomainFlushAction.java:177)
>>> org.objectstyle.cayenne.access.DataDomain.onSyncFlush
>>> (DataDomain.java:830)
>>> org.objectstyle.cayenne.access.DataDomain$2.transform
>>> (DataDomain.java:801)
>>> org.objectstyle.cayenne.access.DataDomain.runInTransaction
>>> (DataDomain.java:856)
>>> org.objectstyle.cayenne.access.DataDomain.onSync(DataDomain.java:
>>> 798)
>>> org.objectstyle.cayenne.access.DataContext.flushToParent
>>> (DataContext.java:1261)
>>> org.objectstyle.cayenne.access.DataContext.commitChanges
>>> (DataContext.java:1165)
>>>
>>> Thread: http-0.0.0.0-443-18 : priority:5, demon:true, threadId:
>>> 187, threadState:BLOCKED,
>>> threadLockName:org.objectstyle.cayenne.access.DataRowStor..a2ae9
>>> org.objectstyle.cayenne.access.ObjectResolver.synchronizedObjectsFro
>>> mDataRows(ObjectResolver.java:133)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.interceptObject
>>> Conversion(DataDomainQueryAction.java:373)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.execute
>>> (DataDomainQueryAction.java:151)
>>> org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java:
>>> 766)
>>> org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery
>>> (ObjectContextQueryAction.java:253)
>>> org.objectstyle.cayenne.access.DataContextQueryAction.execute
>>> (DataContextQueryAction.java:90)
>>> org.objectstyle.cayenne.access.DataContext.onQuery
>>> (DataContext.java:1422)
>>> org.objectstyle.cayenne.access.DataContext.performQuery
>>> (DataContext.java:1411)
>>>
>>> Thread: http-0.0.0.0-443-19 : priority:5, demon:true, threadId:
>>> 188, threadState:BLOCKED,
>>> threadLockName:org.objectstyle.cayenne.access.DataRowStor..a2ae9
>>> org.objectstyle.cayenne.access.ObjectResolver.synchronizedObjectsFro
>>> mDataRows(ObjectResolver.java:133)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.interceptObject
>>> Conversion(DataDomainQueryAction.java:373)
>>> org.objectstyle.cayenne.access.DataDomainQueryAction.execute
>>> (DataDomainQueryAction.java:151)
>>> org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java:
>>> 766)
>>> org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery
>>> (ObjectContextQueryAction.java:253)
>>> org.objectstyle.cayenne.access.DataContextQueryAction.execute
>>> (DataContextQueryAction.java:90)
>>> org.objectstyle.cayenne.access.DataContext.onQuery
>>> (DataContext.java:1422)
>>> org.objectstyle.cayenne.access.DataContext.performQuery
>>> (DataContext.java:1411)
>>>
>>>
>>
>
>
This archive was generated by hypermail 2.0.0 : Tue Dec 12 2006 - 09:26:03 EST