Changes between Initial Version and Version 1 of Ticket #2000


Ignore:
Timestamp:
Apr 15, 2016, 9:27:50 AM (6 years ago)
Author:
Nicklas Nordborg
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #2000

    • Property Priority changed from major to critical
    • Property Component changed from web to core
  • Ticket #2000 – Description

    initial v1  
    2020
    2121In theory it should be possible to improve the second step a lot since in this stage the annotation importer is only working with a single item at a time. We do not need Hibernate to keep things in the first-level cache. If we can manage this it may be that the Hibernate commit step is also automatically solved. The change log step may be harder, since we are already using the stateless session here. However, it is maybe possible to replace this with our own batch SQL implementation as we have done for reporters and raw data already.
     22
     23It turned out that using the annotation importer to delete the existing annotations proved to be much worse. The initial parsing and updating of items used about the same time and amount of memory as when creating items. When committing BASE need to go through relations that may point to the deleted items and either delete them as well or nullify the reference (for example, any-to-any links and inherited/cloned annotations). This consumed more and more memory and reached a point where most of the time was spent doing GC. After 1.5 hours (60 minutes GC) I gave up and killed Tomcat. I'll see what happens if Tomcat get more memory...
     24