To prevent spam users, you can only post on this forum after registration, which is by invitation. If you want to post on the forum, please send me a mail (h DOT m DOT w DOT verbeek AT tue DOT nl) and I'll send you an invitation in return for an account.
Metrics for evaluation and ProDiGen
I am currently developing an algorithm of process mining. I met two problems.
In my algorithm, it consists of a number of loops. In each loop, I want to evaluate a candidate by four metrics, including fitness, precise, generalization, simplicity. Can anybody tell me how can I implement it?
I want to compare my algorithm with ProDiGen algorithm. But I can not find the package. Can anybody tell me where can I find it?
Thank you in advance.
Dear Si-Yuan Jing,For themetrics, you could have a look at the way it is done in the Evolutionary Tree Miner, see the sources on https://svn.win.tue.nl/trac/prom/browser/Packages/EvolutionaryTreeMiner/Trunk/src/org/processmining/plugins/etm/fitness/metrics. Note that these metrics assume the model to be a process tree, I'm not sure whether this fits your needs.As far as I know, the ProDiGen algorithm is not available in any version of ProM that we distribute. I've found a link to a ProDiGen website (http://tec.citius.usc.es/SoftLearn/ProDiGen.html) but that results in an error message. Perhaps you could contact the authors of the paper (see for example https://link.springer.com/chapter/10.1007/978-3-319-10172-9_8) directly?Kind regards,Eric.
Thanks for your help.
I also wonder that is there a package provides these metrics which are based on petri net? Right now, PNetReplayer is employed in my algorithm. I follow a formula which is used for calculation of fitness value. Is it right?
total_fitness = Sigma(trace_fitness * trace_num) / total_trace_num
Kind regards,Si-Yuan Jing
Dear Si-Yuan Jing,This depends on which fitness metric you want to have. Your formula computes the average trace fitness, which is not exactly the same as the log fitness as reported by the PNetReplayer. The difference between both is that for the average trace fitness the fitness values are accumulated, whereas for the log fitness the replays costs are accumulated.Kind regards,Eric.
In the above formula, the "trace_fitness" is gotten by SyncReplayResult.getInfo(PNRepResult.TRACEFITNESS).
I studied the code of org.processmining.plugins.astar.petrinet.AbstractPetrinetReplayer.java.
In line 382-386, the algorithm put the trace fitness into a SyncReplayResult object. I am not sure whether it is the log fitness or not. If not, where can I find the code for calculation of log fitness.
Additionally, I also wonder that where can I find a tool for calculation of precise, generalization and simplicity of a petri net model.
Dear Si-Yuan Jing,No, this is not the log fitness. Every SyncReplayResult corresponds to one alignment, and its fitness to a trace fitness. Unfortunately, the trace fitness are not sufficient to compute the log fitness. In short, both the trace fitness and the log fitness are fractions. To compute the log fitness one needs to divide the sum of the trace fitness nominators by the sum of the trace fitness denominators. For example, if we have trace fitnesses 2/4 and 3/12, this would result in 5/16, which is not the same as for the trace fitnesses 1/2 (=2/4) and 1/4 (=3/12), which would result in 2/6 (>5/16).As far as I know, the PNetReplayer does not compute the log fitness.For the other metrics, you would have to check the literature. There exist different metrics for precision and simplicity. ProM includes some tools to compute these metrics, I guess. Perhaps https://svn.win.tue.nl/trac/prom/wiki/ProM69/Plugins can be of help, which lists all plug-in sin ProM 6.9. You look look for "precision" and "generalization", for "simplicity" you could use the "Show Petri-net Metrics" plug-in.Kind regards,Eric.
Thanks so much! It helps me a lot.
- 1.5K All Categories
- 45 Announcements / News
- 215 Process Mining
- 6 - BPI Challenge 2020
- 9 - BPI Challenge 2019
- 24 - BPI Challenge 2018
- 27 - BPI Challenge 2017
- 8 - BPI Challenge 2016
- 66 Research
- 971 ProM 6
- 377 - Usage
- 285 - Development
- 8 RapidProM
- 1 - Usage
- 6 - Development
- 54 ProM5
- 19 - Usage
- 185 Event Logs
- 30 - ProMimport
- 75 - XESame