How to compare process models?

fordanic

I'm interested in being able to compare a number of different process models but I'm not sure how to do it. I came across the paper: Process Equivalence: Comparing Two Process Models Based on Observed Behavior, but as I understand this paper only presents a method for comparing two models using a common set of event logs. However, in my case I'm interested in comparing two (or or more) process models with separate sets of event logs.

Any suggestions?

JBuijs

Dear Fordanic,

To my knowledge to 'ready to go' plug-ins are available for this in ProM, although work is undergoing in comparative process mining. The main obstacle is however that the layout/visualization of the models needs to be synchronized in order to be able to quickly spot commonalities and differences.

fordanic

I'm assuming your first sentence should be "To my knowledge no 'ready to go' plug-ins are available for this in ProM", or?

I guess the ongoing work you're referring to is related to the process cubes and for example the paper "Comparative Process Mining in Education: An Approach Based on Process Cubes". Haven't read the paper yet (just the abstract) but does it suggest a quantitative way of comparing process models or is it merely talking about a qualitative/visual assessment of differences (as your reply suggests)?

fordanic

Hmh, wouldn't your Comparison Framework plugin be useful for this purpose?

JBuijs

Dear Fordanic,

You could indeed use this framework, and add comparators to it that somehow synchronise layout. At the moment, it is a rather basic framework with some very specific implementations that I needed.

If you want, you can extend it (you can rather easily add statistics on the log, model or model+log level). Please keep me up to date of your additions

JBuijs

Hi Fordanic,

I just saw this fresh blog post coming by, maybe something for you to investigate further?

http://bpmve.blogspot.com.au/2015/04/paper-process-visualization-techniques.html

JBuijs

Hi Fordanic,

You're in luck, through a question of a colleague I remembered this Master thesis that was defended in our group about a month ago on exactly your problem

See http://alexandria.tue.nl/extra1/afstversl/wsk-i/Mourik_2015.pdf

I hope the plug-ins work, although they are not available through the ProM package manager, you can check out the code from SVN and run it from Eclipse (there are guides on how to do that on https://svn.win.tue.nl/trac/prom/wiki/Contribute and other pages)

fordanic

Hmh, correct me if I'm wrong, but both the referenced master's thesis and the paper in the blogpost refer to techniques for visual assessment of differences between process models, i.e. the comparison is qualitative in nature. Are you aware of any attempts of deriving more quantitaive approaches?

For example, something similar to what is discussed in "Similarity of Business Process Models: Metrics and Evaluation", where label similarity, strucutural similarity and behavioral similarity are discusssed. Also, I'm thinking it should be possible to somehow define a metric based upon estimated trace fitness values between a set of derived process models and their corresponding event logs. If not being able to define a distance between process models but at least being able to rank a set of process models in terms of relative similarity.

JBuijs

Hi Fordanic,

You're right, my pointers were mainly visually/qualitative.

The paper you refer to would be my best bet indeed of more quantitative approaches for process model comparison. Your proposed solution would fall in the behavioral similarity class, and could already be discussed in the paper. There are some cave-ats such as how to simulate loops etc., but the general idea is nice.

aronlindberg

@JBuijs & @fordanic: Has there been any progress in this area? I'm creating multiple process models for the same dataset, and I would like to be able to compare which fits best. Is there a way to test this statistically, i.e. the way you would compare two regression models or SEM models using e.g. a Chi2-test?

Amirah_Al

Hi Fordanic, 
I need to compare different process models and prefer quantitative metrics. have you found a way or implemented a plugin for this purpose ? 

thanks 

abhijeetKale

Dear Amirah,
There is a plug in in ProM named Compare Causal Nets (GED) which gives a GED distance between two causal nets.