netflix / surus Goto Github PK

License: Apache License 2.0

R 20.44% C++ 9.47% PigLatin 3.27% Java 66.82%

surus's Introduction

Surus

A collection of tools for analysis in Pig and Hive.

Description

Over the next year we plan to release a handful of our internal user defined functions (UDFs) that have broad adoption across Netflix. The use cases for these functions are varied in nature (e.g. scoring predictive models, outlier detection, pattern matching, etc.) and together extend the analytical capabilities of big data.

Functions

ScorePMML - A tool for scoring predictive models in the cloud.
Robust Anomaly Detection (RAD) - An implementation of the Robust PCA.

Building Surus

Surus is a standard Maven project. After cloning the git repository you can simply run the following command from the project root directory:

mvn clean package

On the first build, Maven will download all the dependencies from the internet and cache them in the local repository (~/.m2/repository), which can take a considerable amount of time. Subsequent builds will be faster.

Using Surus

After building Surus you will need to move it to your Hive/Pig instance and register the JAR in your environment. For those unfamiliar with this process see the Apache Pig UDF, and Hive Plugin, documentation.

You can also install the anomaly detection R package trivially with this code

library(devtools)
install_github(repo = "Surus", username = "Netflix", subdir = "resources/R/RAD")

surus's People

Contributors

Stargazers

Watchers

Forkers

joeledwards372 rajeshkp applied-duality yangspeaking kirinjulia1992 ysh329 dmoliveira damienclaveau tspannatpivotal mindis mmolaro andrie swathimystery sheltowt eleph konggas yeshwanthbabu securapawn pronix ashtonizmev codeaudit punsiitg tengqu fysoft2006 kainoa21 modulexcite kushalvenkatesh mathkann rahuls5 cloudxtreme admercs mehmet-yildirim harishmaiya kinzer1 hardiku nicolaisguevara mdimercurio rtvt123 zhengyizz jdcc alexandrescu blueriverquantlabs sumitraina harryprince mrshanth nagyistge fsgp ersel faducoder tomnis chen0031 us1415 ruochenvincent cliffinlin rrrongon andy9775 next-josealbertoarcos jai2033shankar kelvinni 0xbadbac0n fdaqlik anuhyav bcui6611 dheerajinampudi 52nlp lokiwager artwr navarescoolblue mylons pulkitsmc nhovhannisya kianqunki seawolf888 aftnix poundhu fakenetflix thravu tobievii tamimatths gachugh shubhsoni muhamed-farooq ullasdiwakar josealberto-arcos-sanchez djangid katoomba-demo tungk evilsid melmccann reynoldsm88 eleyeth a8156268 bornfreesoul obinnaobeleagu adam-netflix ertugrulakbas doytsujin test-mass-forker-org-1 dratak-invrs neilzhu89

surus's Issues

Make Surus available in maven central

Hi, we would like to reuse your library in our organisation. Are you planning to publish it on maven central in near future?

Implementation doubt

Hello, and thank you for this wonderful R package.

There's something about this implementation I don't understand. If the goal in Zhou's paper is to minimize this:

Then why is it that in the R package implementation the convergence condition is as follows?

while (iter < maxIter & diff > tol){
  ...
  diff = std::abs(obj_prev - obj);
  ...
}

What I do not understand is the use of the absolute value when calculating the difference. Why do you allow the objective metric to grow? Shouldn't that absolute value be removed?

Diffrent result between R and Java version

I have a time series (56 observations) like this:

 data ={    3.197097, 3.029077, 3.005744, 2.969745, 2.988609, 2.97782, 2.933626,
               3.185347, 3.241275, 3.117891, 3.071268, 3.118897, 3.152572, 3.232348,
               3.424237, 3.323964, 3.302709, 3.341312, 3.341527, 3.375134, 3.543823,
               3.879864, 3.420371, 3.294217, 3.49587, 3.521571, 3.599039, 3.925218,
               3.99248, 3.689928, 3.749015, 3.583267, 3.704804, 3.742834, 3.599793,
               3.699821, 3.630572, 3.684399, 3.725435, 3.743818, 3.744296, 3.667758,
               3.899343, 3.724631, 3.551779, 3.557395, 3.748661, 3.569791, 3.520395,
               3.529122, 3.604996, 3.623308, 3.586358, 3.793575, 3.837355, 3.753702}

When I run with R:

library(RAD)
data = c(3.197097, 3.029077, 3.005744, 2.969745, 2.988609, 2.97782, 2.933626, 3.185347, 3.241275, 3.117891, 3.071268, 3.118897, 3.152572, 3.232348, 3.424237, 3.323964, 3.302709, 3.341312, 3.341527, 3.375134, 3.543823, 3.879864, 3.420371, 3.294217, 3.49587, 3.521571, 3.599039, 3.925218, 3.99248, 3.689928, 3.749015, 3.583267, 3.704804, 3.742834, 3.599793, 3.699821, 3.630572, 3.684399, 3.725435, 3.743818, 3.744296, 3.667758, 3.899343, 3.724631, 3.551779, 3.557395, 3.748661, 3.569791, 3.520395, 3.529122, 3.604996, 3.623308, 3.586358, 3.793575, 3.837355, 3.753702)

a=AnomalyDetection.rpca(data, frequency = 7)

S_matrix=a$S_transform

View(data.frame(S_matrix))

It returns a vector, with the length is 55 (less 1 than the number of the data):
(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.130887399206267, 0, 0, 0, 0, 0.00318375301259443, 0, -0.0885939624397428, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.00646893256411638, 0, 0)

It says that we got 4 anomaly points.

When It comes to Java:

I just adjust the input in RAD_test file.

double[] ts = new double[] { 3.197097, 3.029077, 3.005744, 2.969745, 2.988609, 2.97782, 2.933626, 3.185347, 3.241275, 3.117891, 3.071268, 3.118897, 3.152572, 3.232348, 3.424237, 3.323964, 3.302709, 3.341312, 3.341527, 3.375134, 3.543823, 3.879864, 3.420371, 3.294217, 3.49587, 3.521571, 3.599039, 3.925218, 3.99248, 3.689928, 3.749015, 3.583267, 3.704804, 3.742834, 3.599793, 3.699821, 3.630572, 3.684399, 3.725435, 3.743818, 3.744296, 3.667758, 3.899343, 3.724631, 3.551779, 3.557395, 3.748661, 3.569791, 3.520395, 3.529122, 3.604996, 3.623308, 3.586358, 3.793575, 3.837355, 3.753702};

I got the S_matrix from observed as below:

(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.129439154052943, 0, -0.0123981691519606, 0, 0, 0, 0.168267483707591, 0.156119414262503, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.104831613422113, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
In this case, we had 5 anomaly points.

as you can see, the Java version returns a vector much different from R version.
Beside the length (55 of R, to compare with 56 of Java, It doesnt matter), the values are a big deal.
With each version, I get a brand new result (4 versus 5 anomaly points). It makes me so confused.

I hope you can help me out.

Thank you so much.

Netflixoss build conversion ?s

Do you want to keep this maven and handle your own publishing or do you want to convert to gradle and the autogenerated cloudbees jobs and bintray publishing?

Hive Example

I see in the resources there is a pig example, does anyone have a quick and dirty hive example I could learn from? I have been a hive person, but not a pig guy, thus the pig example is tough for me to make the transition with.

Thanks!

Error while installing the R Package

I am using R 33 & RTools 3.4. I have set Rtools PATH variable correctly.

I have tried the below 2 options but couldnt succedd in the installation. Could you please support.
Option 1:

Install RAD from CRAN

install.packages("RAD")
Installing package into ‘C:/Users/admin/Documents/R/win-library/3.3’(as ‘lib’ is unspecified)
also installing the dependency ‘mvtnorm’
Package which is only available in source form, and may need compilation of C/C++/Fortran: ‘mvtnorm’
Do you want to attempt to install these from sources?
y/n: y
trying URL 'https://wbc.upm.edu.my/cran/bin/windows/contrib/3.3/RAD_0.3.zip'
Content type 'application/zip' length 68253 bytes (66 KB) downloaded 0 bytes
Error in read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) : cannot open the connection
In addition: Warning messages:
1: In download.file(url, destfile, method, mode = "wb", ...) :
downloaded length 0 != reported length 68253
2: In unzip(zipname, exdir = dest) : error 1 in extracting from zip file
3: In read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) :

cannot open compressed file 'RAD/DESCRIPTION', probable reason 'No such file or directory'

Option 2:

install_github(repo = "mmolaro/Surus", subdir = "src/RAD")
Downloading GitHub repo mmolaro/Surus@master from URL https://api.github.com/repos/mmolaro/Surus/zipball/master
Installing RAD
"C:/PROGRA~~1/R/R-33~~1.1/bin/i386/R" --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL
"C:/Users/admin/AppData/Local/Temp/RtmpiQuqgz/devtools1a443f6a3c5/mmolaro-Surus-ffbf216/src/RAD"
--library="C:/Users/admin/Documents/R/win-library/3.3" --install-tests

installing source package 'RAD' ...
** libs
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-33~~1.1/include" -DNDEBUG -I"C:/Users/admin/Documents/R/win-library/3.3/Rcpp/include" -I"C:/Users/admin/Documents/R/win-library/3.3/RcppEigen/include" -I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall -mtune=core2 -c RcppExports.cpp -o RcppExports.o
c:/Rtools/mingw_32/bin/g++ -I"C:/PROGRA~~1/R/R-33~~1.1/include" -DNDEBUG -I"C:/Users/admin/Documents/R/win-library/3.3/Rcpp/include" -I"C:/Users/admin/Documents/R/win-library/3.3/RcppEigen/include" -I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall -mtune=core2 -c anomaly_detection.cpp -o anomaly_detection.o
anomaly_detection.cpp: In function 'Rcpp::List RcppRPCA(Eigen::MatrixXd, double, double, bool)':
anomaly_detection.cpp:181:15: warning: suggest parentheses around comparison in operand of '&' [-Wparentheses]
while (iter < maxIter & diff > tol) {
^
c:/Rtools/mingw_32/bin/g++ -shared -s -static-libgcc -o RAD.dll tmp.def RcppExports.o anomaly_detection.o -Ld:/Compiler/gcc-4.9.3/local330/lib/i386 -Ld:/Compiler/gcc-4.9.3/local330/lib -LC:/PROGRA~~1/R/R-33~~1.1/bin/i386 -lR
installing to C:/Users/admin/Documents/R/win-library/3.3/RAD/libs/i386
** R
** tests
** preparing package for lazy loading
Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) :
there is no package called 'quadprog'
ERROR: lazy loading failed for package 'RAD'
removing 'C:/Users/admin/Documents/R/win-library/3.3/RAD'
restoring previous 'C:/Users/admin/Documents/R/win-library/3.3/RAD'
Error: Command failed (1)

Could you please help me to install RAD

AugmentedDickeyFuller test not correct

In the adf test, smaller value of test statistic means the null hypothesis is rejected. That is, unit root is rejected, the time series is stationary, no diff is needed.

Therefore this code is wrong
if (t <= PVALUE_THRESHOLD) { this.needsDiff = true; } else { this.needsDiff = false; }

It should be
if (t <= PVALUE_THRESHOLD) { this.needsDiff = false; } else { this.needsDiff = true; }

RPCA method in RAD doesn't work with multivariate data optimally

I haven't looked at the implementation details in the Pig code, but at least in the R interfaces, multivariate dataframes are treated one timeseries at a time.

This is not intuitive given the rational behind using RPCA for anomaly detection, in particular the existence of many dimensional datasets.

A data point that violates the lower dimensional subspace assumption by changing relationship between two variables could be easily missed in a one variable at a time perspective.

"RAD_RcppRPCA" not available for .Call() for package "RAD"

I cannot properly install this package and I think it may be Windows related. I have installed it using Surus, with the code provided in the README, and when I run AnomalyDetection.rpca I get the following error:

Error in .Call("RAD_RcppRPCA", PACKAGE = "RAD", X, Lpenalty, Spenalty, :
"RAD_RcppRPCA" not available for .Call() for package "RAD"

The same procedure on a Mac works fine.

I also tried building Surus by cloning the project and using the "Clean & Build" option in RStudio. From that I receive the following error:

ERROR: The build directory does not contain a DESCRIPTION
file so cannot be built as a package.
Build directory: ~/Bitbucket/tempsa/Surus

Am I missing something obvious?

Error in AnomalyDetection.rpca

Hi,

I am using the R version of Anomaly Detection and am coming across an error. Here are the details:

Error:

AnomalyDetection.rpca(ts)
Error in data.frame(X.transform = X.transform, L.transform = L.transform, :
arguments imply differing number of rows: 231, 227
In addition: Warning message:
In matrix((j - j.global.mean)/j.global.sd, nrow = frequency) :
data length [227] is not a sub-multiple or multiple of the number of rows [7]

Here's how to reproduce it:

ts <- c(54,28,56,98,105,3,96,70,56,56,30,48,42,42,70,2,48,63,70,66,99,1,54,112,21,56,3,9,4,8,14,15,2,20,8,8,1,3,8,6,6,10,1,8,9,7,6,9,6,18,8,16,28,30,6,4,32,20,16,16,88,2,6,16,12,12,20,4,2,16,18,14,12,18,2,8,24,30,8,10,81,196,6,32,21,27,8,7,9,9,40,8,40,24,10,6,18,52,120,196,180,27,32,100,112,96,484,2,18,88,78,78,120,18,16,108,91,72,108,10,654,2169,1252,2480,4704,4995,612,174,3328,2940,2424,2504,8008,104,666,2184,1806,1866,2930,380,87,1736,2817,2135,1896,2736,187,18,90,40,80,140,150,30,20,160,100,80,80,132,3,9,24,18,18,30,6,3,24,27,21,18,27,3,117,76,152,252,285,39,4,256,190,144,144,572,2,36,104,102,114,170,34,13,136,162,140,126,171,15,258,513,404,808,1344,1425,270,152,912,1000,736,720,4312,86,168,776,564,576,990,188,86,472,918,728,576,819,89)

AnomalyDetection.rpca(ts)

Kindly let me know how the issue can be resolved. I am stuck on a project due to the above issue.

Thanks.

mvn test failed

C:\OpenSource\Netflix\Surus\Surus>mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building scoring 0.1.2
[INFO] ------------------------------------------------------------------------
Downloading: http://artifacts.netflix.com/ext-releases-local/amazon/hadoop-core-emr/1.0.3/hadoop-core-emr-1.0.3.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.891s
[INFO] Finished at: Wed Jan 21 18:30:02 PST 2015
[INFO] Final Memory: 9M/241M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project scoring: Could not resolve dependencies for project netflix:scoring:jar:0.1.2: Failed to collect dependencies at amazo
n:hadoop-core-emr:jar:1.0.3: Failed to read artifact descriptor for amazon:hadoop-core-emr:jar:1.0.3: Could not transfer artifact amazon:hadoop-core-emr:pom:1.0
.3 from/to netflix-ext-releasess (http://artifacts.netflix.com/ext-releases-local): artifacts.netflix.com: Unknown host artifacts.netflix.com -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
C:\OpenSource\Netflix\Surus\Surus>

Cannot install R package

Hi,

The algorithm you've implemented seems great and I'm eager to try it on a project. However, I cannot install the R package: Here is the output:

install_github(repo = "Surus", username = "Netflix", subdir = "resources/R/RAD")
Downloading github repo Netflix/Surus@master
Installing RAD
"C:/PROGRA~~1/RRO/R-31~~1.2/bin/x64/R" --vanilla CMD INSTALL
"C:\Users\Stergios.-\AppData\Local\Temp\RtmpIv9Qjq\devtools1eec79a21cb8\Netflix-Surus-a1c24b9\resources\R\RAD"
--library="C:/Users/Stergios.-/My Documents/R/win-library/3.1" --install-tests

installing source package 'RAD' ...
** libs
Warning: running command 'make -f "C:/PROGRA~~1/RRO/R-31~~1.2/etc/x64/Makeconf" -f "C:/PROGRA~~1/RRO/R-31~~1.2/share/make/winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="RAD.dll" WIN=64 TCLBIN=64 OBJECTS="RcppExports.o anomaly_detection.o"' had status 127
ERROR: compilation failed for package 'RAD'
removing 'C:/Users/Stergios.-/My Documents/R/win-library/3.1/RAD'
Error: Command failed (1)
In addition: Warning message:
Username parameter is deprecated. Please use Netflix/Surus

What could I try?

initial value for mu is not consistent with the paper

the parameter "mu" in Candes paper (http://statweb.stanford.edu/~candes/papers/RobustPCA.pdf, see Eqn. 5.1) is actually the inverse of "mu" in Zhou's paper (http://arxiv.org/abs/1001.2363, see Eqn. 15). Therefore, in the anomalydetection.cpp, it should be

double mu = 4_X.lpNorm<1>()/(m_n);

instead of

double mu = m_n / (4_X.lpNorm<1>());

to be consistent with both papers. This also makes sense because in the getDynamicMu(), mu is set to be linear with "sigma", not with inverse of "sigma".

Unable to compile ScorePMML

Unable to compile ScorePMML, it looks like more dependencies are missing

mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building scoring 0.1.2
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.jpmml:pmml-evaluator:jar:1.0.19 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details
[WARNING] The POM for junit:junit-dep:jar:4.8.2 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details
[WARNING] The POM for org.apache.pig:pig:jar:0.14.0 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details
[INFO]
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ scoring ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/rajesh/work/projects/os/Surus/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ scoring ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/rajesh/work/projects/os/Surus/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[25,20] package org.dmg.pmml does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[26,20] package org.dmg.pmml does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[27,20] package org.dmg.pmml does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[28,20] package org.dmg.pmml does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[29,20] package org.dmg.pmml does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[34,25] package org.jpmml.manager does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[35,25] package org.jpmml.manager does not exist
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[41,22] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[42,22] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[43,22] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[49,13] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[49,68] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[115,17] cannot find symbol
symbol: class PMML
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[130,24] cannot find symbol
symbol: variable IOUtil
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[141,40] cannot find symbol
symbol: variable IOUtil
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[146,40] cannot find symbol
symbol: variable IOUtil
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[153,17] cannot find symbol
symbol: class PMMLManager
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[153,47] cannot find symbol
symbol: class PMMLManager
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[156,17] cannot find symbol
symbol: class ModelManager
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[156,103] cannot access org.jpmml.manager.ModelManagerFactory
class file for org.jpmml.manager.ModelManagerFactory not found
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[159,53] cannot access org.jpmml.manager.Consumer
class file for org.jpmml.manager.Consumer not found
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[160,49] cannot find symbol
symbol: method getPredictedFields()
location: variable evaluator of type org.jpmml.evaluator.Evaluator
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[161,53] cannot find symbol
symbol: method getOutputFields()
location: variable evaluator of type org.jpmml.evaluator.Evaluator
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[183,18] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[210,22] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[214,25] cannot find symbol
symbol: class DataField
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[227,22] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[231,25] cannot find symbol
symbol: class OutputField
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[280,21] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[310,21] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[314,21] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[ERROR] /home/rajesh/work/projects/os/Surus/src/main/java/org/surus/pig/ScorePMML.java:[318,21] cannot find symbol
symbol: class FieldName
location: class org.surus.pig.ScorePMML
[INFO] 32 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.329s
[INFO] Finished at: Tue Feb 03 11:37:48 IST 2015
[INFO] Final Memory: 17M/158M
[INFO] ------------------------------------------------------------------------

Compilation failed for package 'RAD'

Hi,

I'm trying to install the RAD package, but when I follow the given command I got "compilation failed for package ‘RAD’" error and it wouldn't install.

I'm running R 3.2.2 on Mac.

Any help is appreciated!

Thanks.

RAD.java throws an NPE when isForceDiff is not passed as constructor arg

The constructor for the class RAD expects an optional argument for the class member isForceDiff which is a Boolean wrapper (line 46). If the argument is not passed in, the member stays null.

line 46: this.isForceDiff = Boolean.parseBoolean(parameters[3]);

This throws an NPE at line 181 where the code checks if (this.isForceDiff).
line 181: } else if (this.isForceDiff) {

To overcome this, isForceDiff has to be passed at least as False.

Fix: The code should check for null at line 181 or make isForceDiff primitive for it to default to false.

*** Unless I am missing something here ***