sinemgetir / state-elimination-mt Goto Github PK

I have a question concerning the correctness check for extension 1:

A solution to the first extension task (see Section 3.2) is correct if the produced model represents a uniform FSA which is equivalent to the one provided as input. This can be confirmed by checking equality of the resulting model to the one produced by the reference implementation using JFLAP. This correctness test may be automated using the model comparison tool EMF Compare [BP08], which should report no changes between a produced output model and the corresponding reference output model.

Technically, the issue here is that the JFLAP implementation seems to use its own data structure to represent the FSA, which lives outside the EMF world (automata.fsa.FiniteStateAutomaton). I assume it's not the idea that every solution provides its own JFLAP-to-EMF-transformation just to enable the correctness check, right? Is there an available transformation we can use?

It is an interesting solution for the state elimination case study. However, the SHARE demo is not provided and I was not able to run the NMF solution with Visual Studio, due to existence of some reference error in the files.

In addition, evaluation result is not provided for the solution and therefore I could not verify the correctness of the regular expressions that are generated as final results. However, it is assumed that the correctness is provided for all the models.

Finally the paper is well written and well structured.

The evaluation sheet provides more details for the evaluation of this solution.
NMF-ttc-evaluation-sheet-state-elimination.xlsx

Since timeouts usually happen after a user-specified time duration, it seems like a good idea if all solutions used the same duration as the reference solution. However, neither the paper nor the test framework specifies a duration. Any suggestions on which duration to use?

Unfortunately, I was not able to run the solution. I contacted the authors via e-mail on Wednesday June, 07 asking them for instructions for running the solution, but did not receive any working instructions by now. A SHARE image is not provided. So I cannot verify the provided performance data (correctness is not mentioned in the paper).

The solution is based on a Yage, a new graph-based model transformation language. Compared to other graph-based transformation language, a core feature seems to be the representation of attributes and edges using HashMaps, which might at some point enable certain performance benefits. However, the average performance results in the case do not support this interpretation yet.

The way I understand the solution, it is based on a number of rules for dealing with specific cases of eliminating a state, and a control flow for orchestrating the rules, leading to a natural and generally suitable solution to the case.

An aspect of the solution description I found hard to understand is where the eight different cases for "rerouting edges" come from, which lead to a total of 20 rules. Judging from Listing 3, the case distinction with manifold patterns in the style of "pkKpkK" may be unneccesarily complicated.

Comments about the paper:

The paper sells Yage as a "fresh start", focusing on "simple data structures and algorithms". However, from the presentation it's not so evident what this simplicity entails. The idea to specify rules as graph patterns where elements have different actions (visualized as colors) is just like in other graph-based transformation languages. While I do not expect a related work discussion, it might be better to be more specific when claiming novelty.

I didn't understand the mentioned issue with the naming of edges. Representing a name in an object-structure is simple, so when aiming a deliberately simple design, I wonder how one winds up with a complicated solution.

The depiction of the rule in Figure 2 has some formatting issues, like illegible labels and labels which overlap with arrowheads.

Hi,

the following imports in classes of the EvaluationFramework project cannot be resolved:

import automata.State;
import automata.fsa.FSATransition;
import automata.fsa.FiniteStateAutomaton;

I could not find the imported classes in the a text-search on the repository as well. Is there something missing?

The definition of the transformation rules are quite clear. However, the implementation of the transformation for this approach is not accessible. The SHARE demo and SCM link were missed and therefore it was not convenient to provide comprehensive review and evaluation (suitability,....).

In addition, evaluation result is not provided for the solution and therefore I could not verify the correctness of the regular expressions that are generated as final results. However, it is assumed that the correctness is provided for all the models.

Finally I would suggest discussion section for the implementation of transformation in the paper. These enable to have better insight about the solution.

The evaluation sheet provides more details for the evaluation of this solution.
SDMLib-ttc-evaluation-sheet-state-elimination.xlsx

The paper features clear and self-contained specifications for the basis case and extension 1. However, the specification of extension 2 is not self-contained, for example: "outgoing transitions are recalculated" - how precisely are they calcuated?

The paper then just refers to the source code of the reference implementation, leaving it to the contestants to "reverse-engineer" the specification from its implementation (which I find unnecessarily complicated).

I wonder if a specification of the extension can be made available.