What is considered as best or optimal threshold for model fitness and precision

aryans14

Hello all fellow researchers and PM enthusiasts

I have question about fitness and precision dimension to evaluate quality of the process models. As we know that closer the fitness to 1, the more fitting the model is and the closer the precision to 1, the less generic and less false-positive behavior model contains.

Question is that is there any ideal fitness and precision value. Lets says models with x threshold on fitness and precision are good enough?

Again I know above values are complimentary where increase in one, results in decrease of other and being them closer to one is normally favored. But this is out of curuisity to know is there any threshold limit that above x threshold these values are considered good.

I appreciate if anyone has references to the threshold (if any).

Thank you all in anticipation

hverbeek

Hi,

I guess this is hard to say, as this depends on the actual metrics you are using for fitness and precision. Although for fitness the alignment-based fitness is now typically accepted as the default, such a default metric for precision does not exist yet.

Ideally, I guess the metrics should be such that values above 90% should be considered as good, but that's just my feeling

Kind regards, Eric.

aryans14

Thank you Prof. I am using alignment based token replay.

I think we cannot achieve both fitness and precision up to 90%. Either one has to be sacrificed.

Actually I am trying to achieve a balance between fitness and precision where both fitness and precision are somehow balanced instead of just focusing on one.

In this way fitness is around around 80 to 85 percent and precision around 54 percent compared to some other approaches where I am getting fitness as 90 percent but precision is around 20 percent.

So, I am sacrificing some fitness for improving precision in order to approach a balanced model.

Although looking at figures, my approach is performing better but I have no way to justify the threshold of these values.

hverbeek

A lot also depends on the discovery algorithm you are using. If the process model is not block-structured, but your discovery algorithm always discovers a block-structured model, then some mismatch will always occur. Having both perfect fitness and perfect precision is then impossible. If you have an idea about the process model you are trying to discover, then it may be worthwhile to use a discovery algorithm that can actually discover such a process model.