Log file columns not working when converting CSV to XES - CLI

NiceNick

Hi all,

I'm trying to convert a CSV file into a XES file using the CLI, using this script:

/*

Option builder to convert CSV into a XES.

*/

Options OPTIONS = new Options();

Option TRACE = OptionBuilder.hasArg().withArgName("traceColumn").create("trace");

Option EVENT = OptionBuilder.hasArg().withArgName("eventColumn").create("event");

Option COMPLETE = OptionBuilder.hasArg().withArgName("completionColumn").create("complete");

OPTIONS.addOption(TRACE);

OPTIONS.addOption(EVENT);

OPTIONS.addOption(COMPLETE);

/***********************Setting up the converter method***********************************/

Class[] cArg = new Class[2];

cArg[0] = File.class;

cArg[1] = CommandLine.class;

Method met = CSVConversionCLI.class.getDeclaredMethod("parseCSV",cArg); //Method is private within the class, so we need to use reflection to use it.

met.setAccessible(true);

/****************************************************************************************/

File param = new File("params.in");

Scanner sc = new Scanner(param);

com=sc.nextLine();

args= com.split(" ");

CommandLineParser parser = new PosixParser();

CommandLine commandLine = parser.parse(OPTIONS, args);

name=sc.nextLine();

File logFile = new File(name);

com=sc.nextLine();

args= com.split(" ");

folder=args[0];

name=args[1];
/*

Method is also a static method, so the first arg of invoke is null. The second arg is an Array that contains all input parameters

*/

XLog logXES = met.invoke(null,new Object[]{logFile,commandLine});

I can successfully convert a CSV to a XES file when using one column for the event and trace option, like this:

-event Info -trace CaseID

But when I try to use multiple columns for either trace and/or event (-event Info,CaseID -trace CaseID) it crashes giving me this:

Error while executing '[-f, script.txt]'
org.processmining.contexts.scripting.ScriptExecutor$ScriptExecutionException: Sourced file: inline evaluation of: ``import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.Command . . . '' : Typed variable declaration : Method Invocation met.invoke : at Line: 89 : in file: inline evaluation of: ``import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.Command . . . '' : met .invoke ( null , new Object [ ] {

Target exception: java.lang.reflect.InvocationTargetException

Exception in thread "main" org.processmining.contexts.scripting.ScriptExecutor$ScriptExecutionException: Sourced file: inline evaluation of: ``import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.Command . . . '' : Typed variable declaration : Method Invocation met.invoke : at Line: 89 : in file: inline evaluation of: ``import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.Command . . . '' : met .invoke ( null , new Object [ ] {

Target exception: java.lang.reflect.InvocationTargetException

    at org.processmining.contexts.scripting.ScriptExecutor.execute(ScriptExecutor.java:109)
    at org.processmining.contexts.cli.CLI.main(CLI.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.processmining.framework.boot.Boot.boot(Boot.java:331)
    at org.processmining.framework.boot.Boot.boot(Boot.java:298)
    at org.processmining.contexts.cli.CLI.main(CLI.java:161)
Caused by: Sourced file: inline evaluation of: ``import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.Command . . . '' : Typed variable declaration : Method Invocation met.invoke : at Line: 89 : in file: inline evaluation of: ``import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.Command . . . '' : met .invoke ( null , new Object [ ] {

Target exception: java.lang.reflect.InvocationTargetException

    at bsh.BSHMethodInvocation.eval(Unknown Source)
    at bsh.BSHPrimaryExpression.eval(Unknown Source)
    at bsh.BSHPrimaryExpression.eval(Unknown Source)
    at bsh.BSHVariableDeclarator.eval(Unknown Source)
    at bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
    at bsh.Interpreter.eval(Unknown Source)
    at bsh.Interpreter.eval(Unknown Source)
    at bsh.Interpreter.eval(Unknown Source)
    at org.processmining.contexts.scripting.ScriptExecutor.execute(ScriptExecutor.java:86)
    ... 8 more

Method I'm using:

Package: org.processmining.log.csvimport

private static XLog parseCSV(File inputFile, CommandLine commandLine) throws CSVConversionException, CSVConversionConfigException {
		CSVConversion conversion = new CSVConversion();
		CSVFile csvFile = new CSVFileReferenceUnivocityImpl(inputFile.toPath());
		CSVConfig importConfig = new CSVConfig(csvFile);
		CSVConversionConfig conversionConfig = new CSVConversionConfig(csvFile, importConfig);
		conversionConfig.autoDetect();

		if (commandLine.hasOption(TRACE.getOpt())) {
			conversionConfig.setCaseColumns(ImmutableList.of(commandLine.getOptionValue(TRACE.getOpt())));
		}
		
		if (commandLine.hasOption(EVENT.getOpt())) {
			conversionConfig.setEventNameColumns(ImmutableList.of(commandLine.getOptionValue(EVENT.getOpt())));
		}
		
		if (commandLine.hasOption(START.getOpt())) {
			conversionConfig.setStartTimeColumn(commandLine.getOptionValue(START.getOpt()));
		}
		
		if (commandLine.hasOption(COMPLETE.getOpt())) {
			conversionConfig.setCompletionTimeColumn(commandLine.getOptionValue(COMPLETE.getOpt()));
		}
		
		ProgressListener cmdLineProgressListener = new ProgressListenerPrintStreamImpl(System.out);
		ConversionResult<XLog> result = conversion.doConvertCSVToXES(cmdLineProgressListener, csvFile, importConfig,
				conversionConfig);
		return result.getResult();
	}

Can some give an insight on how to correctly use multiple columns to generate a XES file?

Thanks a lot,

NiceNick.

hverbeek

Hi NiceNick,

In your failing example, EVENT.getOpt() evaluates to the string "Info,CaseID", and ImmutableList.of("Info,CaseID") result in a list containing a single string "Info,CaseID". I cannot see where you split this string into the strings "Info" and "CaseID". Does this happen somewhere?

Kind regards,

Eric.

NiceNick

Hello hverbeek,

Previously, I was using a space to separate the string (for example: -event Info CaseID -trace CaseID) and the script ran without any problem.
But, after analyzing the results, the script was only using the fist column for the event (the Info column). Which means that no matter how many columns I put on event, only Info was used to generate the XES in the event option. The "," was my last failed attempt.

When all columns were separated by space, it was split here:

com=sc.nextLine();

args= com.split(" ");

CommandLineParser parser = new PosixParser();

CommandLine commandLine = parser.parse(OPTIONS, args);

Do you have any idea how to use two or more columns from a CSV to produce a XES?

Best regards.

hverbeek

Hi NiceNick,

You can use the comma, but then you need to spit the argument on the comma before passing it to setEventNameColumns:

		if (commandLine.hasOption(EVENT.getOpt())) {
			conversionConfig.setEventNameColumns(ImmutableList.of(commandLine.getOptionValue(EVENT.getOpt()).split(",")));
		}
	

Kind regards,

Eric.

NiceNick

Hello hverbeek,

Thanks for your answer
But can I do the same without changing the parseCSV method (as this is a method from a plugin)? It's not a requirement to have the event names separated by a "," (as there can be one event column or multiple).
If you can think of another way to do this it will be appreciated!

Thanks in advance. 

hverbeek

Hi NiceNick,

No, I guess you cannot do this with the current input CSV and the current parseCSV method. Can you change the input CSV by adding an extra column where the value is concatenation of the Info and CaseID columns? Other than that, I do not know. The current parseCSV method assumes that a single column is selected for the event name, not a lot to do about that.

Kind regards,

Eric.

NiceNick

Hi hverbeek,

Thanks for your response!
"Can you change the input CSV by adding an extra column where the value is concatenation of the Info and CaseID columns?" I guess will try this work around as well as manually change the parseCSV method as you suggested earlier.

Best regards!