mail archive users.openjpa.apache.org
(List home) (Recent threads) (2 other Apache OpenJPA lists)
Subscription Options
- RSS or Atom: Read-only subscription using a browser or aggregator. This is the recommended way if you don't need to send messages to the list. You can learn more about feed syndication and clients here.
- Conventional: All messages are delivered to your mail address, and you can reply. To subscribe, send an email to the list's subscribe address with "subscribe" in the subject line, or visit the list's homepage here.
- Low traffic list: less than 3 messages per day
- This list contains about 10,762 messages, beginning Jun 2007
- 0 messages added yesterday
Report the Spam
This button sends a spam report to the moderator. Please use it sparingly. For other removal requests, read this.
Are you sure? yes no
Audit(?) related memory leak/hog
Ad
Hi,
I have an OSGi app that uses OpenJPA 2.2.0 and makes use of the audit
functionality.
Some of the entities that get recorded contain BLOB fields (documents).
I'm getting occasional massive uses of memory (~900MB on a machine that
typically sits at ~150MB) closely followed by hangs and often a restart
by the karaf wrapper.
Looking at a memory dump I see that there are multiple copies of my BLOB
entity in memory (not just multiple records, but multiple copies of each
record) during a period of inactivity (where I wouldn't expect to see any).
Trying to track back responsibility for the objects gives me this tree:
this Document_JpaImpl
value ConcurrentHashMap$HashEntry
[0] ConcurrentHashMap$HashEntry[]
table ConcurrentHashMap$Segment
[7] ConcurrentHashMap$Segment[]
segments ConcurrentHashMap
_audits AuditManager$AuditCallback
[43] Object[]
elementData LifecycleEventManager$ListenerList
...
[43] Object[]
elementData LifecycleEventManager$ListenerList
...
[43] Object[]
elementData LifecycleEventManager$ListenerList
...
this$0
(I'm not sure how useful that is!)
There are a lot of other duplicate JPA entities in memory too, but they
are smaller and less of a problem.
Is this something I've done or a problem with OpenJPA?
Thanks
Jim
Jim -
No one else has reported memory leak issues when running with Audit
support... but that doesn't mean there aren't problems.
Can you post a heapdump to a FTP somewhere for analysis?
Thanks,
RickOn Mon, Aug 6, 2012 at 2:56 PM, Jim Talbut wrote:
> Hi,
>
> I have an OSGi app that uses OpenJPA 2.2.0 and makes use of the audit
> functionality.
> Some of the entities that get recorded contain BLOB fields (documents).
>
> I'm getting occasional massive uses of memory (~900MB on a machine that
> typically sits at ~150MB) closely followed by hangs and often a restart by
> the karaf wrapper.
>
> Looking at a memory dump I see that there are multiple copies of my BLOB
> entity in memory (not just multiple records, but multiple copies of each
> record) during a period of inactivity (where I wouldn't expect to see any).
>
> Trying to track back responsibility for the objects gives me this tree:
> this Document_JpaImpl
> value ConcurrentHashMap$HashEntry
> [0] ConcurrentHashMap$HashEntry[]
> table ConcurrentHashMap$Segment
> [7] ConcurrentHashMap$Segment[]
> segments ConcurrentHashMap
> _audits AuditManager$AuditCallback
> [43] Object[]
> elementData LifecycleEventManager$**ListenerList
> ...
> [43] Object[]
> elementData LifecycleEventManager$**ListenerList
> ...
> [43] Object[]
> elementData LifecycleEventManager$**ListenerList
> ...
> this$0
> (I'm not sure how useful that is!)
>
> There are a lot of other duplicate JPA entities in memory too, but they
> are smaller and less of a problem.
>
> Is this something I've done or a problem with OpenJPA?
>
> Thanks
>
> Jim
>
Rick,
I've put the file on dropbox and shared it with you - if anyone else
wants access just let me know (today if possible).
I've also put a few relevant Java files there to.
If you'd like me to put the files somewhere else just let me know where.
I removed the @Auditable attribute from the large entities and my memory
problems went away, so it does seem to be audit related.
If I'm reading the file correctly there are 108 instances of
Document_JpaImpl (there were never that many rows in the database).
The entities don't have an id value yet, but they do relate to a parent
assessmentResults entity (and there is a one-to-one relationship in
practice, though in theory the parent could have multiple children).
The documents #2-#52 all relate to assessmentResults #73, so I'm
thinking they are duplicates.
The byte arrays may be large enough to have been created in the tenured
generation - I don't know if that's upsetting things somehow (a full
garbage collection doesn't get rid of them).
I'm on holiday as of now and won't be responding to any emails after
today until 18 August, so please don't take silence as a lack of interest.
I am very keen to get to the bottom of this and will respond when I get
back, and will provide whatever information you need to identify the
problem.
Thanks.
JimOn 07/08/2012 17:46, Rick Curtis wrote:
> Jim -
>
> No one else has reported memory leak issues when running with Audit
> support... but that doesn't mean there aren't problems.
>
> Can you post a heapdump to a FTP somewhere for analysis?
>
> Thanks,
> Rick
>
> On Mon, Aug 6, 2012 at 2:56 PM, Jim Talbut wrote:
>
>> Hi,
>>
>> I have an OSGi app that uses OpenJPA 2.2.0 and makes use of the audit
>> functionality.
>> Some of the entities that get recorded contain BLOB fields (documents).
>>
>> I'm getting occasional massive uses of memory (~900MB on a machine that
>> typically sits at ~150MB) closely followed by hangs and often a restart by
>> the karaf wrapper.
>>
>> Looking at a memory dump I see that there are multiple copies of my BLOB
>> entity in memory (not just multiple records, but multiple copies of each
>> record) during a period of inactivity (where I wouldn't expect to see any).
>>
>> Trying to track back responsibility for the objects gives me this tree:
>> this Document_JpaImpl
>> value ConcurrentHashMap$HashEntry
>> [0] ConcurrentHashMap$HashEntry[]
>> table ConcurrentHashMap$Segment
>> [7] ConcurrentHashMap$Segment[]
>> segments ConcurrentHashMap
>> _audits AuditManager$AuditCallback
>> [43] Object[]
>> elementData LifecycleEventManager$**ListenerList
>> ...
>> [43] Object[]
>> elementData LifecycleEventManager$**ListenerList
>> ...
>> [43] Object[]
>> elementData LifecycleEventManager$**ListenerList
>> ...
>> this$0
>> (I'm not sure how useful that is!)
>>
>> There are a lot of other duplicate JPA entities in memory too, but they
>> are smaller and less of a problem.
>>
>> Is this something I've done or a problem with OpenJPA?
>>
>> Thanks
>>
>> Jim
>>
>
>
Jim - I'm not going to claim to know this audit code, but it looks like there is an OpenJPA bug. I think the problem is that in AuditManager<eclipse-javadoc:%E2%98%82=openjpa-kernel/src%5C/main%5C/java%3Corg.apache.openjpa.kernel%7BAuditManager.java%E2%98%83AuditManager>.afterBegin(...) we register a listener (Broker.addLifecycleListener), but we fail to deregister this listener. FYI -- I'm out on vacation from 8/16 - 9/3. -------------------------------------------------------------------------------------------------------------- Class Name | ShallowHeap | Retained Heap | Percentage -------------------------------------------------------------------------------------------------------------- org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl @ 0x74142b8c0 | 520 | 84,080,184 | 66.72% |- org.apache.openjpa.lib.conf.PluginValue @ 0x741455100 |Heap | Retained Heap | Percentage -------------------------------------------------------------------------------------------------------------- org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl @ 0x74142b8c0 | 520 | 84,080,184 | 66.72% |- org.apache.openjpa.lib.conf.PluginValue @ 0x741455100 | 72 | 83,620,760 | 66.36% | |- org.apache.openjpa.event.LifecycleEventManager @ 0x741bfba38|40 | 83,620,576 | 66.36% | | |- java.util.HashMap @ 0x741c0fab0 | 48 | 83,620,392 | 66.36% | | | '- java.util.HashMap$Entry[16] @ 0x741c22a28 | 80 | 83,620,344 | 66.36% -------------------------------------------------------------------------------------------------------------- ... and there are 10K entries in this map. Thanks, Rick ------- Thanks, RickOn Wed, Aug 8, 2012 at 2:36 AM, Jim Talbut wrote: > Rick, > > I've put the file on dropbox and shared it with you - if anyone else wants > access just let me know (today if possible). > I've also put a few relevant Java files there to. > If you'd like me to put the files somewhere else just let me know where. > > I removed the @Auditable attribute from the large entities and my memory > problems went away, so it does seem to be audit related. > > If I'm reading the file correctly there are 108 instances of > Document_JpaImpl (there were never that many rows in the database). > The entities don't have an id value yet, but they do relate to a parent > assessmentResults entity (and there is a one-to-one relationship in > practice, though in theory the parent could have multiple children). > The documents #2-#52 all relate to assessmentResults #73, so I'm thinking > they are duplicates. > > The byte arrays may be large enough to have been created in the tenured > generation - I don't know if that's upsetting things somehow (a full > garbage collection doesn't get rid of them). > > I'm on holiday as of now and won't be responding to any emails after today > until 18 August, so please don't take silence as a lack of interest. > I am very keen to get to the bottom of this and will respond when I get > back, and will provide whatever information you need to identify the > problem. > > Thanks. > > Jim > > > On 07/08/2012 17:46, Rick Curtis wrote: > >> Jim - >> >> No one else has reported memory leak issues when running with Audit >> support... but that doesn't mean there aren't problems. >> >> Can you post a heapdump to a FTP somewhere for analysis? >> >> Thanks, >> Rick >> >> On Mon, Aug 6, 2012 at 2:56 PM, Jim Talbut >> wrote: >> >> Hi, >>> >>> I have an OSGi app that uses OpenJPA 2.2.0 and makes use of the audit >>> functionality. >>> Some of the entities that get recorded contain BLOB fields (documents). >>> >>> I'm getting occasional massive uses of memory (~900MB on a machine that >>> typically sits at ~150MB) closely followed by hangs and often a restart >>> by >>> the karaf wrapper. >>> >>> Looking at a memory dump I see that there are multiple copies of my BLOB >>> entity in memory (not just multiple records, but multiple copies of each >>> record) during a period of inactivity (where I wouldn't expect to see >>> any). >>> >>> Trying to track back responsibility for the objects gives me this tree: >>> this Document_JpaImpl >>> value ConcurrentHashMap$HashEntry >>> [0] ConcurrentHashMap$HashEntry[] >>> table ConcurrentHashMap$Segment >>> [7] ConcurrentHashMap$Segment[] >>> segments ConcurrentHashMap >>> _audits AuditManager$AuditCallback >>> [43] Object[] >>> elementData LifecycleEventManager$****ListenerList >>> ... >>> [43] Object[] >>> elementData LifecycleEventManager$****ListenerList >>> ... >>> [43] Object[] >>> elementData LifecycleEventManager$****ListenerList >>> >>> ... >>> this$0 >>> (I'm not sure how useful that is!) >>> >>> There are a lot of other duplicate JPA entities in memory too, but they >>> are smaller and less of a problem. >>> >>> Is this something I've done or a problem with OpenJPA? >>> >>> Thanks >>> >>> Jim >>> >>> >> >> >
Thanks Rick. I've failed a Jira for this as https://issues.apache.org/jira/browse/OPENJPA... JimOn Wed, Aug 08, 2012 at 04:18:33PM -0500, Rick Curtis wrote: > Jim - > > I'm not going to claim to know this audit code, but it looks like there is > an OpenJPA bug. I think the problem is that in > AuditManager<eclipse-javadoc:%E2%98%82=openjpa-kernel/src%5C/main%5C/java%3Corg.apache.openjpa.kernel%7BAuditManager.java%E2%98%83AuditManager>.afterBegin(...) > we register a listener (Broker.addLifecycleListener), but we fail to > deregister this listener. FYI -- I'm out on vacation from 8/16 - 9/3. > > -------------------------------------------------------------------------------------------------------------- > > Class Name | Shallow > Heap | Retained Heap | Percentage > -------------------------------------------------------------------------------------------------------------- > org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl @ 0x74142b8c0 | > 520 | 84,080,184 | 66.72% > |- org.apache.openjpa.lib.conf.PluginValue @ 0x741455100 | > 72 | 83,620,760 | 66.36% > | |- org.apache.openjpa.event.LifecycleEventManager @ 0x741bfba38| > 40 | 83,620,576 | 66.36% > | | |- java.util.HashMap @ 0x741c0fab0 | > 48 | 83,620,392 | 66.36% > | | | '- java.util.HashMap$Entry[16] @ 0x741c22a28 | > 80 | 83,620,344 | 66.36% > -------------------------------------------------------------------------------------------------------------- > > ... and there are 10K entries in this map. > > Thanks, > Rick > > ------- > > Thanks, > Rick > > On Wed, Aug 8, 2012 at 2:36 AM, Jim Talbut wrote: > > > Rick, > > > > I've put the file on dropbox and shared it with you - if anyone else wants > > access just let me know (today if possible). > > I've also put a few relevant Java files there to. > > If you'd like me to put the files somewhere else just let me know where. > > > > I removed the @Auditable attribute from the large entities and my memory > > problems went away, so it does seem to be audit related. > > > > If I'm reading the file correctly there are 108 instances of > > Document_JpaImpl (there were never that many rows in the database). > > The entities don't have an id value yet, but they do relate to a parent > > assessmentResults entity (and there is a one-to-one relationship in > > practice, though in theory the parent could have multiple children). > > The documents #2-#52 all relate to assessmentResults #73, so I'm thinking > > they are duplicates. > > > > The byte arrays may be large enough to have been created in the tenured > > generation - I don't know if that's upsetting things somehow (a full > > garbage collection doesn't get rid of them). > > > > I'm on holiday as of now and won't be responding to any emails after today > > until 18 August, so please don't take silence as a lack of interest. > > I am very keen to get to the bottom of this and will respond when I get > > back, and will provide whatever information you need to identify the > > problem. > > > > Thanks. > > > > Jim > > > > > > On 07/08/2012 17:46, Rick Curtis wrote: > > > >> Jim - > >> > >> No one else has reported memory leak issues when running with Audit > >> support... but that doesn't mean there aren't problems. > >> > >> Can you post a heapdump to a FTP somewhere for analysis? > >> > >> Thanks, > >> Rick > >> > >> On Mon, Aug 6, 2012 at 2:56 PM, Jim Talbut > >> wrote: > >> > >> Hi, > >>> > >>> I have an OSGi app that uses OpenJPA 2.2.0 and makes use of the audit > >>> functionality. > >>> Some of the entities that get recorded contain BLOB fields (documents). > >>> > >>> I'm getting occasional massive uses of memory (~900MB on a machine that > >>> typically sits at ~150MB) closely followed by hangs and often a restart > >>> by > >>> the karaf wrapper. > >>> > >>> Looking at a memory dump I see that there are multiple copies of my BLOB > >>> entity in memory (not just multiple records, but multiple copies of each > >>> record) during a period of inactivity (where I wouldn't expect to see > >>> any). > >>> > >>> Trying to track back responsibility for the objects gives me this tree: > >>> this Document_JpaImpl > >>> value ConcurrentHashMap$HashEntry > >>> [0] ConcurrentHashMap$HashEntry[] > >>> table ConcurrentHashMap$Segment > >>> [7] ConcurrentHashMap$Segment[] > >>> segments ConcurrentHashMap > >>> _audits AuditManager$AuditCallback > >>> [43] Object[] > >>> elementData LifecycleEventManager$****ListenerList > >>> ... > >>> [43] Object[] > >>> elementData LifecycleEventManager$****ListenerList > >>> ... > >>> [43] Object[] > >>> elementData LifecycleEventManager$****ListenerList > >>> > >>> ... > >>> this$0 > >>> (I'm not sure how useful that is!) > >>> > >>> There are a lot of other duplicate JPA entities in memory too, but they > >>> are smaller and less of a problem. > >>> > >>> Is this something I've done or a problem with OpenJPA? > >>> > >>> Thanks > >>> > >>> Jim > >>> > >>> > >> > >> > > > > > -- > *Rick Curtis*
I had similar memory issues although I was not using the Audit feature. For me it turns out that the detach manager seems to be going pretty haywire and every 24 hours or so my host would run out of memory. I added this to my persistence.xml <property name="openjpa.DetachState" value="loaded(DetachedStateField=false,DetachedStateManager=false)"/> The issue seemed related to the detach manager and using memory dumps I could see that there were thousands of extra objects on the heap for no particular reason. Hope this helps. Bruce