fitness and conformance of model generated by genetic miner

navid

Hi,
Can some one tell me how to check conformance of model generated by genetic miner and also the fitness of the model???
Thanks

aadrians

Hi Navid,

As far as I know, conformance checking require models in form of Petri net. If you have a Petri net model, you can use the "Conformance Checking" plugin in ProM 5 to check fitness based on missing/remaining tokens or "Replay a Log on Petri net for Conformance Analysis" plugin in ProM 6 to check fitness based on alignments. For the latter option, there is an online documentation available in https://svn.win.tue.nl/trac/prom/export/5647/Documentation/Replay a Log on Petri Net for Conformance Analysis plug-in.pdf

Regards,

Arya

navid

Thank you aadrians,

Well, for example in this document (http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4782560) they have evaluated the conformance of the model generated by Fuzzy miner, I want to know how???
About the Petrinet that you told me, I tried it, but it is really strange because when I evaluate the generated Petri net model produced by Alpha-algorithm, the model is totally correct, but when I use the same log besides the same produced Petrinet model on "replay a log on Petrinet for conformance analysis" plug-in, the result for fitness is zero and the traces are not replayed on the model.I have attached the .xes file of my log. Would you please take a look at it.
Thanks alot

navid

can someone help me about the my previous question???? I have attached  another version of my log which is problematic for me.
Thanks in advance

JBuijs

The petri net discovered by the Alpha algorithm is not sound and therefore the replayer cannot 'enforce proper completion', therefore, the most cost-efficient way of aligning your event log and model is to do nothing in the model and perform only move on logs on the log.

If you would mine a Petri net using the ILP miner (choose for the 'Petri net - empty after completion') option, then the replayer is able to produce alignments, although they are not reliable.

So, there is no 'real problem', everything works as it should, just not as you were hoping for.

navid

Thank you JBuijs

Well, with the ILP miner it worked with the highest fitness.Do you have any idea about my first question that how in that document they have calculated the fitness of Fuzzy logic, because I couldn't find any plug-in for conformance and fitness of Fuzzy and Genetic miner.I need to replay a log on the model produced by genetic miner.

Best Regards

JBuijs

Isn't this contained in the plug-ins? You might want to try ProM 5, the genetic miner there shows the fitness for sure. In ProM 6 the Fuzzy miner should also, otherwise check the ProM 5 version.

navid

Hi,
I have one more question about the conformance of Fuzzy model. Using the default configuration of Fuzzy miner , the conformance of the model is 0.69.  In order to improve the conformance value, I increased the cutoff value of edge filter. After increasing, the conformance value rose to 0.85 which seems to be a promising value.But , after evaluating the model, I found too many  wrong edges in the model. with this model most of the logs might fit the generated model but the model behavior is far from the real behavior of the system.My question is that how we should trust a model found by Fuzzy,Genetic or other plugins,since the conformance or fitness values of them seem not to be reliable.
Thanks

JBuijs

Hi Navid,

Conformance, or fitness is only one of four quality dimensions to describe the relationship between an event log and a process model.
The model can replay all recorded behavior, and much more, and is therefore fitting. It is however not precise, the second dimension you want to look at. Precision however is harder to calculate. Arya Adriansyah is working on measures for this right now.

For more information on the quality dimensions see the process mining book, and some related publications,  by Wil van der Aalst.

navid

Hi,
Considering the previous comments in this discussion, I still have one critical problem. In order to find the bottlenecks, I decided to use Prom 5.2 instead of 6.1. The problem is that  my mxml log cannot be replayed on the model produced by alpha miner in order to use the "performance analysis with petrinet " plug-in. Previously, JBuijs, told me the problem with Alpha miner is that the model is not sound,but with ILP miner,it worked. In Prom 5.2, to be able to implement performance analysis practically , I need to have a complete petrinet model which is possible to be replayed.My mxml file is attached here and also the xes format of that has been attached at some comments before. I appreciate any help about how I can apply performance analysis by replaying the log on prom 5.2 or any change that I need to do in my log.
Thank you in advance

navid

Hi All, at the previous comment I forgot to say that I couldn't find ILP miner in PROM 5 and moreover the model produced by ILP miner in Prom 6.1 was not interesting.It does not give any view about the process. I am looking forward any help.
NAVID

JBuijs

I know that Arya Adriansyah is busy implementing (and might just have finished) a more advanced performance replayer in ProM 6.
The disadvantage of the replayer of ProM 5 is that it only replays fitting traces until they can not be replayed in the process model any more, hence giving incomplete results.

As far as I know the ILP miner should be present in ProM 5!

navid

Hi
 In prom 6 the plugin is "mine for a petri net  using ILP".I couldn't find any plugin related to ILP in prom 5. Moreover, I am looking for bottlenecks,thus I need time-related values. But the number of fitting in "performance analysis with petrinet " is zero. I produced my log from XESame software then put the tick of " output to MXML insread of XES", I thought it might make problems. The log is almost similar to Repair Example. It is very strange that Repair is working but my log is not replayed.Should I make any change inside the log????
Thanks

ranbel

Hi Navid,

I hope the following doc will help u in determining the fitness between log and fuzzy model

http://alexandria.tue.nl/extra1/afstversl/wsk-i/xia2010.pdf

thanks