Automated way of clearing down column data about miller HOT 7 CLOSED

@karudonaldson more generally perhaps:

$ cat example.csv
one,two,three,four,seven,eight,thirty
pan,pan,1,0.347,0.727,0.252,10
eks,pan,2,0.759,0.522,0.396,20
wye,wye,3,0.205,0.338,0.069,30
eks,wye,4,0.381,0.134,0.051,40
wye,pan,5,0.573,0.864,0.495,50
zee,pan,6,0.527,0.493,0.260,60
eks,zee,7,0.612,0.188,0.115,70
zee,wye,8,0.599,0.976,0.584,80
hat,wye,9,0.031,0.750,0.024,90
pan,wye,10,0.503,0.953,0.479,100

mlr --csv --from example.csv put '
  begin {
    @names = ["two", "seven", "thirty"]
  }
  for (name in @names)  {
   $[name] = ""
  }
'

one,two,three,four,seven,eight,thirty
pan,,1,0.347,,0.252,
eks,,2,0.759,,0.396,
wye,,3,0.205,,0.069,
eks,,4,0.381,,0.051,
wye,,5,0.573,,0.495,
zee,,6,0.527,,0.260,
eks,,7,0.612,,0.115,
zee,,8,0.599,,0.584,
hat,,9,0.031,,0.024,
pan,,10,0.503,,0.479,

from miller.

Thank you both for your help! I managed to get a preliminary solution tested this morning :)

from miller.

If you have

a,b,c
1,2,4
3,5,6
7,8,9

you can use put and run mlrgo --csv put '$b=""' input.csv to get

a,b,c
1,,4
3,,6
7,,9

Have you read the manual? It is very well written.
Have you done any tests?

from miller.

Hi @karudonaldson I'm not sure I understand what your goal is.
Looking for a tool that monitors the creation of new files in a folder, and as soon as there is a new one apply a Miller command?

One tool for monitoring a folder is, for example, inotifywait.

from miller.

Thanks for the response @aborruso.

Ideally looking for functionality to:

• clear predefined column data without removing the column
• above process must retain headers and column order
• require a process to monitor for newly created CSV files in a folder
• if a new file is detected, execute column clearing process

It appears miller can do the column clearing process, but not sure how?

from miller.

Miller is used to process structured text files. You can write a bash script and have it do what you want.

If you have an example of input and the processing you would like to do, I can give you an example.

from miller.

ok so imagine example.csv looks like this

I need to strip column data in column "two" without affecting the header or column order.
The result being:

There is roughly 100+ columns, where around 30 need to be cleared.

Does that make more sense?

from miller.