thijsbenschop / pii-scan Goto Github PK

Scan.R searches all Stata (.dta), SAS (.sas7bdat), SPSS (.sav), and comma-separated values (.csv) files found in the specified directory for variables that may contain personally identifiable information (PII) using strings that commonly appear as part of variable names or labels that contain PII. (Note: Scan.R does not search labels in .csv files.) Results are displayed to the screen and saved to a comma-separated values file in the current working directory containing the variables and data flagged as potential PII.

To execute the script, type Rscript scan.R --path=[Target] into the command line, replacing [Target] with the folder containing the data to scan. An excel spreadsheet titled "PII_output" will be saved in the current working directory for your review.

• --quiet: Silent operation; do not display possible PII to the screen
• --no-output: Do not output search results to CSV file
• --output-file=output_filename.csv: Write csv of possible PII to file "output_filename.csv" instead of "PII_output.csv"
• --strict: Use strict matching when comparing strings. For example, match "lat" but not "latin"
• --nolabels: Do not scan variable labels for PII search terms

• R must be installed on the local system. Installers can be downloaded at http://cran.us.r-project.org.
• The dplyr, purrr, optparse, rio and tools packages are required. To install missing packages, run install.packages("PACKAGE_NAME") where PACKAGE_NAME is the name of the package to install. Scan.R will warn if any of these packages are missing.

scan.R will exit with a value of 10 (ten) when possible PII is found or a value of 0 (zero) when no PII is identified.

The script was written to audit data files for personally identifiable information. It provides a solution to quickly searching a large number of files in a particular directory or searching files that contain a large number of variables. However, it does not fully replace manual detection of PII.

scan.R searches variables and labels for the following strings: address, bday, beneficiary, birth, birthday, block, census, child, city, community, compound, coord, country, daughter, degree, district, dob, email, father, fax, first_name, fname, gender, gps, house, husband, last_name, lat, lname, loc, location, lon, minute, mother, municipality, name, network, panchayat, parish, phone, precinct, school, second, sex, social, spouse, son, street, subcountry, territory, url, village, wife, zip

Running this script on a Windows machine may result in the following error:

Error in df_parse_dta_file(spec, encoding) :
  Failed to parse [pathname]: Unable to allocate memory.
Calls: import ... standardize_attributes -> read_dta -> df_parse_dta_file -> .Call
Execution halted

If you receive this error, remove the current version of the R package "Haven" and install the 1.1.0 version as a local zip file. The older version can be downloaded at: https://mran.microsoft.com/snapshot/2017-11-01/bin/windows/contrib/3.5/.

Please use the issue tracker for all support requests.

See license file.

Special thanks to IPA for their How to Search Datasets for Personally Identifiable Information document which inspired this project.