A tiny tool to
Normalize ARM instructions
Remove duplicate word from a vocabulary file.
Remove rows that contain empty fields from a CSV file.
Two files to be expected.
If the input file is test.csv The output files would be in the same folder with names
- test.csv-non-empty, and
- test.csv-empty
./goTool -removeEmpty=true -input=test.csv -indices="1,3,9" hasHeader=true
- Input file is test.csv
- Indexes of fields to be checked are 1, 3 and 9.
./goTool -removeEmpty=true -input=test.csv -indices="0"
- Input file is test.csv
- Index of a field to be checked is 0.
./goTool -removeEmpty=true -input=test.csv
- Input file is test.csv
- All fields are to be checked.
Remove rows that contain number of tokens out of limitation.
Two files to be expected.
If the input file is test.csv The output files would be in the same folder with names
- test.csv-non-qualified, and
- test.csv-qualified
./goTool -filter=true -input=test.csv -indices="1,3,9" -hasHeader=true -min=1 -max=512
- Input file is test.csv
- Indexes of fields to be checked are 1, 3 and 9.
- Minimum length is 1
- Maximum length is 512
./goTool -filter=true -input=test.csv -indices="1,3,9" -hasHeader=true -min=1
- Input file is test.csv
- Indexes of fields to be checked are 1, 3 and 9.
- Minimum length is 1
./goTool -filter=true -input=test.csv -indices="1,3,9" -hasHeader=true
- Input file is test.csv
- Indexes of fields to be checked are 1, 3 and 9.
- Minimum length is 1
Merge X86 and ARM CSV files and create fn2fn CSV.
./goTool -merge=true -f1=f1.csv -f2=f2.csv
Randomly select a percentage of lines of a given file to generate a new file
./goTool -shrink=true -input=fn2fn.csv -p=0.6 -hasHeader -s
input
: input file is "fn2fn.csv"p
: shrink percentage is 0.6h
: keep headers. If true, first line would be appeared in the generated file(s); otherwise, all lines are randomized into the generated file(s)s
: keep both files. If true, keep both fn2fn.csv.0.6 and fn2fn.csv.0.4
Rotate previous dataset folder structure to new dataset folder structure
{ARCHITECTURE}-{COMPILER}-{LIBRARY}/{VERSION}/-{OPTIMIZATION}/
{LIBRARY}-{VERSION}/{ARCHITECTURE}-{COMPILER}-{OPTIMIZATION}{OBFUSCATION}/
metrics of output from ml
log file of the format:
Epoch 1/33
798/798 [==============================] - 6341s 8s/step - loss: 2.7144 - accuracy: 0.4629 - val_loss: 2.0714 - val_accuracy: 0.4864
Epoch 2/33
798/798 [==============================] - 6511s 8s/step - loss: 2.1010 - accuracy: 0.5019 - val_loss: 1.8767 - val_accuracy: 0.4864
A CSV file with the columns:
epoch,total time,time per step,loss,accuracy,val_lost,val_accuracy
File name will be the name of the input file appends with .csv
./goTool -logMatrix=true -input=FILE_NAME
-input=
accepts a log file name.
e.g.:
/goTool -logMatrix=true -input=seq2seq.txt
- Input file is
seq2seq.txt
- Output file is
seq2seq.txt.csv