Giter Club home page Giter Club logo

ctgan's Introduction

ctgan

Codecov test coverage R build status

The ctgan package provides an R interface to CTGAN, a GAN-based data synthesizer. The package enables one to create synthetic samples of confidential or proprietary datasets for sharing. For more details and use cases, see the papers in the References section.

Installation

You can install the development version from GitHub with:

# install.packages("remotes")
remotes::install_github("kasaai/ctgan")

Example

A quick example:

library(ctgan)

# Install dependencies before first usage
# install_ctgan()

synthesizer <- ctgan()
synthesizer %>% 
  fit(iris, epochs = 20)
#> Epoch 1, Loss G: 1.1087, Loss D: -0.0124
#> Epoch 2, Loss G: 1.1455, Loss D: 0.0002
#> Epoch 3, Loss G: 1.1319, Loss D: -0.0197
#> Epoch 4, Loss G: 1.1054, Loss D: 0.0126
#> Epoch 5, Loss G: 1.0720, Loss D: -0.0240
#> Epoch 6, Loss G: 1.0652, Loss D: -0.0669
#> Epoch 7, Loss G: 1.0788, Loss D: -0.0513
#> Epoch 8, Loss G: 1.0766, Loss D: -0.0393
#> Epoch 9, Loss G: 1.0464, Loss D: -0.0116
#> Epoch 10, Loss G: 1.0297, Loss D: 0.0042
#> Epoch 11, Loss G: 1.0110, Loss D: -0.0482
#> Epoch 12, Loss G: 0.9828, Loss D: -0.0413
#> Epoch 13, Loss G: 0.9493, Loss D: -0.0031
#> Epoch 14, Loss G: 1.0067, Loss D: -0.0687
#> Epoch 15, Loss G: 0.9857, Loss D: -0.0215
#> Epoch 16, Loss G: 0.9618, Loss D: 0.0044
#> Epoch 17, Loss G: 0.9280, Loss D: 0.0061
#> Epoch 18, Loss G: 0.8579, Loss D: -0.0139
#> Epoch 19, Loss G: 0.8993, Loss D: 0.0400
#> Epoch 20, Loss G: 0.8277, Loss D: 0.0320
synthesizer %>% 
  ctgan_sample() %>% 
  # Dataset-specific post-processing
  dplyr::mutate_if(is.numeric, ~ pmax(.x, 0.1))
#> # A tibble: 100 x 5
#>    Sepal.Length Sepal.Width Petal.Length Petal.Width Species   
#>           <dbl>       <dbl>        <dbl>       <dbl> <chr>     
#>  1         4.67        3.49        3.97         1.94 virginica 
#>  2         7.75        3.14        3.87         1.96 setosa    
#>  3         5.84        3.68        4.81         2.46 versicolor
#>  4         5.03        2.90        2.74         2.38 virginica 
#>  5         5.01        4.39        0.423        1.83 versicolor
#>  6         5.74        3.33        3.20         2.58 virginica 
#>  7         4.13        3.15        2.88         3.24 setosa    
#>  8         6.83        2.81        3.25         3.60 setosa    
#>  9         5.21        3.92        6.06         2.15 setosa    
#> 10         4.23        3.98        2.81         2.52 virginica 
#> # … with 90 more rows

This generated dataset can then be shared, but one can also serialize and share the trained synthesizer:

model_dir <- tempdir()
synthesizer %>% 
  ctgan_save(model_dir)

ctgan_load(model_dir)
#> A CTGAN Model

References

If you use ctgan, please consider citing the original work,

  • Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. Modeling Tabular data using Conditional GAN. NeurIPS, 2019.

and the work implementing the R package,

  • Kevin Kuo. Generative Synthesis of Insurance Datasets. arXiv:1912.02423, 2019.
@inproceedings{xu2019modeling,
  title={Modeling Tabular data using Conditional GAN},
  author={Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
  booktitle={Advances in Neural Information Processing Systems},
  year={2019}
}

@misc{kuo2019generative,
    title={Generative Synthesis of Insurance Datasets},
    author={Kevin Kuo},
    year={2019},
    eprint={1912.02423},
    archivePrefix={arXiv},
    primaryClass={stat.AP}
}

ctgan's People

Contributors

kevinykuo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

ctgan's Issues

Installation Error

Same exact error as above with the 'init()' got an unexpected keyword argument 'gen_dim'

Error in py_get_attr_impl(x, name, silent) : AttributeError: module 'ctgan' has no attribute 'CTGANSynthesizer'

Newly installed the ctgan package and ctgan using install_ctgan(). Now trying to run the example from the Github page (synthesizer <- ctgan()) and getting the error:

Error in py_get_attr_impl(x, name, silent) :
AttributeError: module 'ctgan' has no attribute 'CTGANSynthesizer'

Edit: FWIW it looks like v0.6.0 of ctgan changed the name of CTGANSynthesizer to just CTGAN (https://pypi.org/project/ctgan/)

Error after install: Error in py_get_attr_impl(x, name, silent) : AttributeError: 'module' object has no attribute 'ctgan_model'

I went through the install process, then ran install_ctgan_deps() and it all seemed to install with no issues. However when I then tried

library(ctgan)
synthesizer <- ctgan()

I would get the following error:

Error in py_get_attr_impl(x, name, silent) : 
  AttributeError: 'module' object has no attribute 'ctgan_model'

Here's my output from install_ctgan_deps()

> install_ctgan_deps()
Using virtual environment '~/.virtualenvs/r-reticulate' ...
Collecting torch
  Downloading https://files.pythonhosted.org/packages/7e/94/0ed9f7899aa0f5e7ff753a3a2b6944c146eef3f4cd51c59ab07c4575992b/torch-1.3.1-cp37-none-macosx_10_7_x86_64.whl (71.1MB)
Collecting torchvision
  Downloading https://files.pythonhosted.org/packages/fa/71/0e76ba50c8c9aeb8349d827d02278c1b5eb4da9cdc17ca26b5bd47ec034a/torchvision-0.4.2-cp37-cp37m-macosx_10_7_x86_64.whl (641kB)
Collecting sklearn
  Downloading https://files.pythonhosted.org/packages/1e/7a/dbb3be0ce9bd5c8b7e3d87328e79063f8b263b2b1bfa4774cb1147bfcd3f/sklearn-0.0.tar.gz
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/60/9a/a6b3168f2194fb468dcc4cf54c8344d1f514935006c3347ede198e968cb0/numpy-1.17.4-cp37-cp37m-macosx_10_9_x86_64.whl (15.1MB)
Collecting pandas
  Downloading https://files.pythonhosted.org/packages/16/b5/bab3477466a4d9e705d40829ac65683155e7977acbc07f05b06fabded1be/pandas-0.25.3-cp37-cp37m-macosx_10_9_x86_64.whl (10.2MB)
Collecting pillow>=4.1.1 (from torchvision)
  Downloading https://files.pythonhosted.org/packages/85/28/2c72ba965b52884a0bd71e419761fc162763dc2e5d9bec2f3b1949f7272a/Pillow-6.2.1-cp37-cp37m-macosx_10_6_intel.whl (3.9MB)
Collecting six (from torchvision)
  Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting scikit-learn (from sklearn)
  Downloading https://files.pythonhosted.org/packages/b1/db/102a43a72afd2b0938ca039af6047e2a2bf5e5d5eb1bbe38ce616e2d750d/scikit_learn-0.22-cp37-cp37m-macosx_10_6_intel.whl (10.9MB)
Collecting pytz>=2017.2 (from pandas)
  Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
Collecting python-dateutil>=2.6.1 (from pandas)
  Downloading https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227kB)
Collecting joblib>=0.11 (from scikit-learn->sklearn)
  Downloading https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl (294kB)
Collecting scipy>=0.17.0 (from scikit-learn->sklearn)
  Downloading https://files.pythonhosted.org/packages/43/df/3f85a3a762e5dd8d593614288c1f86a5dfbb8b20071192e850d1a9934e7d/scipy-1.3.3-cp37-cp37m-macosx_10_6_intel.whl (27.7MB)
Building wheels for collected packages: sklearn
  Building wheel for sklearn (setup.py): started
  Building wheel for sklearn (setup.py): finished with status 'done'
  Stored in directory: /Users/jal/Library/Caches/pip/wheels/76/03/bb/589d421d27431bcd2c6da284d5f2286c8e3b2ea3cf1594c074
Successfully built sklearn
Installing collected packages: numpy, torch, pillow, six, torchvision, joblib, scipy, scikit-learn, sklearn, pytz, python-dateutil, pandas
Successfully installed joblib-0.14.1 numpy-1.17.4 pandas-0.25.3 pillow-6.2.1 python-dateutil-2.8.1 pytz-2019.3 scikit-learn-0.22 scipy-1.3.3 six-1.13.0 sklearn-0.0 torch-1.3.1 torchvision-0.4.2
You are using pip version 19.0.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Installation complete.


Restarting R session...

and here's my sessionInfo():

> sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS  10.15.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ctgan_0.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3          lattice_0.20-38     zeallot_0.1.0       crayon_1.3.4        dplyr_0.8.99.9000   assertthat_0.2.1    grid_3.6.0         
 [8] R6_2.4.1            jsonlite_1.6        magrittr_1.5        pillar_1.4.2.9001   rlang_0.4.2         rstudioapi_0.10     Matrix_1.2-17      
[15] reticulate_1.13     generics_0.0.2      vctrs_0.2.0.9007    lobstr_1.1.1        tools_3.6.0         glue_1.3.1          purrr_0.3.3        
[22] compiler_3.6.0      pkgconfig_2.0.3     tidyselect_0.2.5    tibble_2.99.99.9010
> 

Installation error?

The installation seemed successful but I don't seem to be able to load the library successfully? And thus can't use the functions?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.