sqlab / peaas Goto Github PK

The outputs are with the same answer. Try to verify if it is the limitation of the input generation.

你們在產生 output 或進行 fault localization 時，有 security 的問題要注意。因為程式可以寫
system("rm -rf *");
這樣你們的資料就會被殺光。你們可能要先過濾程式是否有 system, unlink 等 function call.
後續，我再教你們如何處理這類型的問題。

Current implementation ignores user authentication. That is, the service is in the public without user authentication. In the future, the judge server should pass authentication information to IOgen-FL server.

http://judge14.cpe.acm-icpc.tw/website/main.php?input=1&output=0&fault=0&pid=10419&cid=5&timelimit=4

你們在產生 output 或進行 fault localization 時，有 security 的問題要注意。因為程式可以寫
system("rm -rf *");
這樣你們的資料就會被殺光。你們可能要先過濾程式是否有 system, unlink 等 function call.
後續，我再教你們如何處理這類型的問題。

1. using eval of python to evaluate the single constraint, instead of lexical analysis and evaluate the expression in our own by
   new_val = random(); relace all $xx variables with the new_val in the A_str and call eval(A_str)
2. test if it works in former implementation
3. test for more complicated constraints: $1%2==0 (even number)

main.php?pid=xxx&cid=xxx

You had better put temp files into subdirectories, for example
mkdir -p $cid/$pid/.....

Otherwise, too many files in a flat folder will trash your eyes and overhead your system.

The generated inputs should be in different difficulty levels, for example, in easy, medium, or hard level. For larger input entries, the generated input should be a downloadable file.

For example:
http://judge14.cpe.acm-icpc.tw/website/main.php?pid=10400&input=1&output=0&faultrank=0

This problem is only with input generation, but no output generation and fault ranking.

你們在產生 output 或進行 fault localization 時，有 security 的問題要注意。因為程式可以寫
system("rm -rf *");
這樣你們的資料就會被殺光。你們可能要先過濾程式是否有 system, unlink 等 function call.
後續，我再教你們如何處理這類型的問題。

if someone know the file name(already run FL and the result hasn't cleared yet) and modify web address can directly jump to results

直接改web address的c/cpp那邊的名字會跳到結果

1. more efficient input generation (random selection between input value range, check the input constraint. 過程可能會一直找不到符合的輸入值)。
2. 依照難度的輸入，更多筆數的輸入資料(根據 size, < 5k < 10k ), download link for large input and output files.
3. fault localization with diff options (用顏色區別不同的輸出)
4. input specification -aware fault localization (了解輸入格式的除錯功能)

檢定的測資很不嚴謹
when k >=3 and k < n-2 output is wrong.

EX : n =6, k =3 output is 20
but most AC program output 19.

There are several input blocks. The generator only produces one block(at least 3-5 blocks). Also, the number of inputs in a block is 1000000. The generated number is too small (should at least > 20).

We should consult to diff -u for unified diff format, parse the format, and display in user friendly way.

許成功。蓋。

我剛剛限制機器無法連到外面機器，但仍可用 ssh, web, 連進來。若有使用的問題, 再通知我。

用完之後 GCOV 會失敗
老師之前看到的結果是之前沒刪除的舊結果

Pass or failed runs must be based on each test case , instead of the whole input file.
Therefore, the faulty ranking must be input format-aware.

另外我想到，或許不能用現有的 output file 來算 fault ranking。必須也了解 input 格式，只挑錯誤的單筆測資，進行 fault ranking 計算。
或許除了富凱之外，政德也來參與 input-aware fault ranking 的修改。

舉例來說，倘若一個 input file 裡面有 10 筆測資，造成程式錯誤的有1筆，但9筆都正確，但在我們的 ranking 計算，就會被打散。因為現在計算的單元是一個 input file ，而非裡面每筆資料。當初富凱也有提出這樣的問題，我為了趕快有結果呈現，所以簡化了方法。
現在看來，還是要知道 input 格式，分開每筆測資的執行結果。

The input data should be restricted by size (for example 10k), with download links.

the max value is 2^32. the generator should produce the max value.