The F1 meter tool allows users to measure the F1 value of their slicers by validating their slices against the gold standards of the Bencher suite. For this purpose, we validate each benchmark, using the Erlang Dependence Graph, extracting the common nodes between the gold standard and the slice being validated. The gold standard EDGs are represented in a particular format (.dot files), where each line contains the properties of a node or an edge. Each node has several attributes. One of these attributes is slice=true, which represents whether a node is part of the slice or not. All the user's slices will be compared with these .dot files.

All these .dot files together with the gold standard programs can be found in the "F1_meter/benchmarks/gold_standard" folder. In folder "F1_meter/benchmarks/Original EDGs" the user can find the .dot files, without any slice=true label, that represent the EDG of the original programs.

Download

You can download F1meter here.

Instructions to use the F1 meter tool:

1. Before executing the F1 meter tool it is necessary to map the result obtained with the user's slicer into the .dot files given in the "F1_meter/benchmarks/Original EDGs" folder and copy them to the "F1_meter/benchmarks/userSlices" folder. (This step is necessary for the correct performance of the tool, you cannot reorder the nodes or delete any node either).
2. Run the F1_meter.jar file and press on the Start button.
3. The result of the comparison will be shown in the table where the user can compare it against the best previous result.
4. You can double click on each row to see the missing and surplus nodes of your solution.
5. As a side effect, a last.csv file will be saved in the /Results folder with the statistical results of each benchmark.

To keep a record of the best results, every time the global F1 value is improved a file called best.csv will be stored in the "Results/bests" folder.