To prevent spam users, you can only post on this forum after registration, which is by invitation. If you want to post on the forum, please send me a mail (h DOT m DOT w DOT verbeek AT tue DOT nl) and I'll send you an invitation in return for an account.

How many rows (events) ProM 6 can deal with ?

Hi everyone, 

I think that all is in the title. I ask this question because I see that ProM knows some difficulties to run when I load a database with 10,000 events.

Thank you.

Meziane.

Answers

  • Hi Meziane,

    Felix Mannhardt once gave the following reply:

    I made some estimations in the documentation of XESLite:

    https://svn.win.tue.nl/trac/prom/browser/Documentation/Package%20XESLite.pdf

     

    In the most extreme memory saving variant (Sequential Access, w/o cache), there is a memory footprint of:

    - 1 object per trace with the following fields

      - 1x reference (8 byte)

      - 1x long (8 byte)

      - 2x int (8 byte)

      - 1x byte[] (8 byte)

     

    The byte[] is variable in size an should be smaller than 8 byte per event. It is not really predictable as it depends on the compression.

     

    All in all each trace adds an overhead of 32 bytes and each event something between 1-8 bytes (depending on the compression ratio).

     

    Assuming the compression does not work 100 million events would still be fitting in around 762 MB of RAM. Of course, each trace adds 32 bytes overhead. So if there are 10 events per trace, it would be 10.000.000 traces using another 300 MB.

     

    In XESLite all the attributes would be stored on the disk and remaining virtual memory of the operation system is used to cache those. So there is no real limit, but it might get slower.



    Joos Buijs

    Senior Data Scientist and process mining expert at APG (Dutch pension fund executor).
    Previously Assistant Professor in Process Mining at Eindhoven University of Technology
Sign In or Register to comment.