mjskay / artool Goto Github PK
View Code? Open in Web Editor NEWR Package for Aligned Rank Transform for Nonparametric Factorial ANOVAs
R Package for Aligned Rank Transform for Nonparametric Factorial ANOVAs
Like this:
lsm <- lsmeans(artlm(m.art, "X1:X2"), ~ X1:X2)
contrast(lsm, interaction = "pairwise")
From Russ:
You can cut it down to fewer with a different contrast family, e.g. "consec" for comparing consecutive levels. You could even specify interaction = c("dunnett", "consec") or something to get different comparisons for X1 than for X2.
I am wondering what would be the reason beyond the message given by R for receiving this warning message: "F values of ANOVAs on aligned responses not of interest are not all ~0. ART may not be appropriate."
Also, is there any way to address this or should the test not be used at all on data that generates this warning message?
E.g. A vs B within C (from the existing interactions vignette)
I asked about this 5 days ago on StackExchange/CrossValidated but I am afraid the question got buried before anyone who could answer it saw it, so I hope you don't mind me asking here.
I tried using ART ANOVA recently for two-way analyses, but summary.art
keeps warning me that "F values of ANOVAs on aligned responses not of interest are not all ~0". According to Wobbrock et al., 2011, this indicates imperfect stripping of the effects that are not the one being aligned for. From what I understand the summary.art
compares the values to 0 using all.equal
with the default tolerance, which is sqrt(.Machine$double.eps). This is ca. 1.5*10-8.
For example, I am getting this warning even when the maximum F-value after ANOVA on aligned responses is 0.0003. ANOVA with 'response="aligned"' shows that these non-zero F-values occurred in the case of aligning by interactions. For some dependent variables, this F-value reaches 0.07.
Am I doing something wrong to get these F-values that are not exactly 0? Is it reasonably normal to get such values and the comparison is just excessively strict? How far from 0 can one get before the results of ART ANOVA start getting unreliable? In what way do they become less reliable? I would really appreciate any information, context, or pointer!
I tried to re-run code in this vignette. I created data using tibble()
as shown in the vignette.
Using the tibble as input for art()
, when I call art.con(m.art, "X2")
, I receive this error:
Error: Can't use character names to index an unnamed vector.
However, if I use R native data frame as input for art()
, it produces the result as shown in the vignette.
library(dplyr)
library(ARTool)
n_per_group = 150
df = tibble(
X1 = factor(c(rep("A", n_per_group), rep("B", n_per_group))),
X2 = factor(rep(c("C","D","E"), n_per_group * 2/3)),
Y = rnorm(
n_per_group * 2,
(X1 == "B")
+ 2* (X2 == "D")
+ 2 * (X1 == "B" & X2 == "D")
- 2 * (X1 == "A" & X2 == "D")
+ 2 * (X2 == "E")
)
)
# does not work
m.art = art(Y ~ X1*X2, data = df)
anova(m.art)
art.con(m.art, "X2")
### Error: Can't use character names to index an unnamed vector.
# works
m.art = art(Y ~ X1*X2, data = as.data.frame(df))
anova(m.art)
art.con(m.art, "X2")
See here: https://www.r-pkg.org/services#badges
Hi
I am experiencing a strange error, and I am not sure if it a problem with ARTool library or Rstudio.
I run my model, e.g.
m<-art(Y ~ X1 * X2 * X3 + (1|RE), data=data) without any problem, m object exists in my environment and it looks OK,
but when I try anova(m) or art.con(m) R studio crashes and gives the following error:
[509935:509935:20240121,232749.348089:ERROR elf_dynamic_array_reader.h:61] tag not found
[509935:509935:20240121,232749.348089:ERROR elf_dynamic_array_reader.h:61] tag not found
This error occurs only when I try to use ARTool library and not with functions from other libraries...
Any idea on the error would be very helpful!!
(..I have already tried uninstalling/reinstalling Rstudio, R and ARTool..)
My environment is
Ubuntu 22.04.3
R 4.3
Rstudio 2023.12.0 Build 369
ARTool v 0.11.1
Thank you!!
Regarding the example on the front page for repeated measures, it seems that other websites use an error term of Error(Tray / (Moisture*Fertilizer))
instead of simply Error(Tray)
. I've tried both ways for a 1-way repeated measures ANOVA and it doesn't seem to change the result. Can you explain if this is an error/bug or the expected functionality (for both 1-way and 2-way repeated measures ANOVAs)?
In case people want to use phia instead of lsmeans
E.g.
testInteractions(artlm(m.art, "X1:X2"), pairwise=c("X1","X2"))
Hello,
I’m writing because your R package ARTool suggests the lsmeans package. For almost a year, the new package emmeans has been available, and I am planning to ask the CRAN team to deprecate it in the next few months. It is quite easy to make the transition to emmeans. There is a vignette providing some tips for how to do this at https://cran.r-project.org/web/packages/emmeans/vignettes/transition-from-lsmeans.html, and there is even a script for converting old R sources, vignettes, and workspaces.
Please let me know if you have any questions. I will keep in touch on this matter as is necessary.
Sincerely,
Russ Lenth
Hi, @mjskay. Hope to find you well. I got a question about the art.con.
Test Environment:
ARTool: 0.11.0
R: 4.0.5
System: macOs Big Sur Version 11.2.3
dependent packages: all lastest version
Following the examples on https://cran.r-project.org/web/packages/ARTool/vignettes/art-contrasts.html
# all codes above are exactly the same as those on the website and all worked fine
# except when executing this statement:
contrast(emmeans(artlm.con(m.art, "X1"), ~ X1), method = "pairwise")
>Error: Can't use character names to index an unnamed vector.
19.vec_as_location(i, n, names = names, arg = arg)
18.doTryCatch(return(expr), name, parentenv, handler)
17.tryCatchOne(expr, names, parentenv, handlers[[1L]])
16.tryCatchList(expr, classes, parentenv, handlers)
15.tryCatch(vec_as_location(i, n, names = names, arg = arg), vctrs_error_subscript_type = function(err) { err <<- err i })
14.vec_as_location2_result(i, n = n, names = names, negative = "error", missing = missing, arg = arg)
13.result_get(vec_as_location2_result(i, n = n, names = names, negative = "error", missing = missing, arg = arg))
12.vec_as_location2(j, n, names)
11.withCallingHandlers(expr, vctrs_error_subscript = function(cnd) { cnd$subscript_arg <- j_arg cnd$subscript_elt <- "column" if (isTRUE(assign) && !isTRUE(cnd$subscript_action %in% c("negate"))) { ...
10.subclass_col_index_errors(vec_as_location2(j, n, names), j_arg = j_arg, assign = assign)
9.vectbl_as_col_location2(j, length(x) + 1L, j_arg = j_arg, assign = TRUE)
8.`[[<-.tbl_df`(`*tmp*`, f.concatenated.variable, value = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, ...
7.`[[<-`(`*tmp*`, f.concatenated.variable, value = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, ...
6.generate.art.concatenated.df(m.f.parsed, df, f.parsed)
5.artlm.con.internal(m, f.parsed, response, factor.contrasts, ...)
4.artlm.con(m.art, "X1")
3.is(object, "emmGrid")
2.emmeans(artlm.con(m.art, "X1"), ~X1)
1.contrast(emmeans(artlm.con(m.art, "X1"), ~X1), method = "pairwise")
Just a small note: artlm(m.art, "X1"), ~ X1) works fine, but artlm.con does not .
I am not sure what causes the problem as the code is exactly the same, but I assumed that the problem is in the generate.art.concatenated.df
function, or probably due to different dependent packages version. TBH, before I upgraded R from 3.6.2 to 4.0.5, the example above showed a different error:
# all codes above are exactly the same as those on the website and all worked fine
# (which means contrast(emmeans(artlm.con(m.art, "X1"), ~ X1), method = "pairwise") prints the results exactly as what the website shows.)
# except when executing this statement:
art.con(m.art, "X1:X2")
>Error: Can't assign to columns that don't exist "X1X2".
I am wondering if you can share your R environment (such as R version, and packages version)
Similarly, when running arc.con on my experiment data, although it throws a different error, it was still in the generate.art.concatenated.df
function:
m = art(Y1 ~ X*Z + (1|D), df = data)
# Tests of differences in pairwise combinations of levels between factors in interactions
art.con(m, "X:Z", adjust = "bonferroni", interaction = FALSE, method = "pairwise")
> Error in if (!is.data.frame(groups) || tail(names(groups), 1L) != ".rows") { : missing value where TRUE/FALSE needed
8.new_grouped_df(data, groups)
7.`names<-.grouped_df`(`*tmp*`, value = NULL)
6.`names<-`(`*tmp*`, value = NULL)
5.unname(art.con.df[, f.interaction.variables.string.vec])
4.do.call(paste, c(unname(art.con.df[, f.interaction.variables.string.vec]), sep = ","))
3.generate.art.concatenated.df(m.f.parsed, df, f.parsed)
2.artlm.con.internal(m, f.parsed, response, factor.contrasts, ...)
1.art.con(m, "X:Z", adjust = "bonferroni", interaction = FALSE, method = "pairwise")
FYI, my data looks like:
$ Y1 : num [1:576]
$ D : int [1:576]
$ X : Factor w/ 4 levels
$ B : Factor w/ 4 levels
$ Z : Factor w/ 3 levels
$ Y2: num [1:576]
I tried to run the tests in this repo, and all tests passed... it confuses me now...
Thank you in advance,
Would be nice for the vignettes to link to each other, since they are HTML after all.
When I do
art_model <- art(values ~ X*Y + Error(participant / (X*Y)), data = my_data)
and run
art.con(art_model, "X:Y")
I receive
Error in eval(predvars, data, env) : object 'X' not found
Mind that my_data
is a data.frame and X
, Y
and participant
are factors. values
is a num.
Mind that the following commands do work, and produce sensible results:
summary(art_model)
anova(art_model)
art.con(art_model, "X")
art.con(art_model, "Y")
And likewise, if I create the model instead with a shorter Error term like this:
art_model <- art(values ~ X*Y + Error(participant), data = my_data)
The original problematic line works and produces sensible results:
art.con(art_model, "X:Y")
Additional alternations, in the hope to provide more help – mind the adjusted Error terms and error message:
art_model <- art(values ~ X*Y + Error(participant / (X)), data = my_data)
art.con(art_model, "X:Y")
=> Error in eval(predvars, data, env) : object 'X' not found
art_model <- art(values ~ X*Y + Error(participant / (Y)), data = my_data)
art.con(art_model, "X:Y")
=> Error in eval(predvars, data, env) : object 'Y' not found
My environment:
Thanks for the help @mjskay , and thanks for ARTool! :)
Currently datasets like Higgins1990Table5 use codings like "1", "2", "3", ... for factors, which can look like numeric data in examples and printed tables even though they aren't. I suspect this contributes to people passing in numeric data for factors (see issue #5).
Simple solution here is to rename the levels to things that are obviously strings and not numeric.
checking tests ... ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
Type 'q()' to quit R.
> library(testthat)
> library(ARTool)
Loading required package: lme4
Loading required package: Matrix
Loading required package: Rcpp
Loading required package: car
>
> test_check("ARTool")
Error in library(lsmeans) : there is no package called 'lsmeans'
Calls: test_check ... lapply -> FUN -> sys.source2 -> eval -> eval -> library
Execution halted
You can use skip_if_not_installed()
to avoid this problem
Hi,
I am reading your vignette "Contrast tests with ART" (2021 version). The example used seems a between-subject factorial design.
I am running similar contrast tests, but my experiment is mixed design (one is between-subject factor, and the other is a within-subject design). When using package 'phia' to analyze the pairwiese comparison for interaction (the difference of differences), the statistic is "chisq", instead of t or F value. If that normal? thanks!
Hello,
First of all, I would like to thank you for this wonderful package. I found no bugs, I only have an issue concerning the publication of the results tables from the anova(art object) function. Is there an easy way to produce the output in a publication-ready format ?
Cheers.
something like "Method:" or "Underlying Anovas:"
I should have made the switch awhile back. Two things:
Once it is in the DL https://doi.org/10.1145/3472749.3474784
I would like to limit pairwise comparisons to each time point of a mixed model repeated measures:
m <- art(
score ~ trt * time + Error(id),
data = mydf
)
anova(m)
The above shows significance for trt and trt:time interaction.
contrast(emmeans(artlm(m.art.lg, "trt:time"), ~ trt:time), method = "pairwise", by = "time")
will give me what I want and do pairwise comparisons between treatments (trt) at each time point only. However, I know that this will use ART and not ART-C, so the results are not valid.
contrast(emmeans(artlm.con(m.art.lg, "trt:time"), ~ trttime), method = "pairwise")
is overkill and does not report any significant differences for any pair.
contrast(emmeans(artlm(m.art.lg, "trt:time"), ~ trttime), method = "pairwise", by = "time")
is what I want, but I get the error "Error in .find.by.rows(args, by) : 'by' variables are not all in the grid".
Would you be able to change it so that the original factor columns are (e.g., in this case, the time and trt columns) are retained as well as the concatenated trttime factors column for this to work?
This package uses function car::Anova()
that implements Kenward-Roger's approximation (KR) to find the degrees of freedom in mixed-effects models. KR could be very slow for a large number of data points. The implementation of KR in the car
package is slower than the implementation in the pbkrtest
package.
This issue could causes ART to behave strangely from the users' perspective
art()
result to a variable is working as expected.art()
without assigning the result to any variable) becomes unresponsive. This was because printing an art
object also runs ANOVA behind the scene.Luke (2017) found that KR and Satterthwaite’s approximation are comparable in terms of Type 1 error. Satterthwaite's method is less time-consuming.
I suggest the following:
car::Anova()
with a call to lmerTest::anova()
. The latter function uses KR from the pbkrtest
package that is a bit faster than the car
package. From the current implementation of the ARTool
, I extracted the model using artlm()
and convert it to lmerModLmerTest
class before calling anova()
:art_result <- art(y ~ x1 * x2 + (1 | id), data = df_wpm_bkgnd)
m_art_typing_style <-
artlm(art_result, "x1") %>%
as_lmerModLmerTest()
anova(m_art_typing_style, ddf= "Kenward-Roger")
art()
"Satterthwaite"
for the ddf
parameter above.I think some are stale; update from zenodo. Might want to add this to the CRAN submission checklist too.
I don't know if you have seen this paper: https://journal.r-project.org/archive/2016/RJ-2016-027/RJ-2016-027.pdf
On page 376, it states:
"This indicates that the art function from ARTool uses the wrong Higgins and Tashtoush formula for aligning the data in mixed designs. It uses the formula for between x between designs, instead of using the formula for mixed (In Higgins’s terms: split-plot or repeated measures) designs."
I did not test this myself, but I was wondering whether you can confirm this as a potential issue? Thanks!
I open this issue in connection with the comment I posted on an older issue opened by @casiez (now closed).
Given our results, which have now been submitted for review at JoVI, we would like to raise awareness about the risks of using ART.
An ANOVA on an ART finds a significant effect of X2 on Y in the following dataset: data.csv
library(dplyr)
library(ARTool) # version 0.11.1
library(lme4)
library(plotly)
library(bootES)
df = read.csv("https://github.com/mjskay/ARTool/files/9513347/data.csv", sep=";")
# assume 'S' is the name of the subjects column
# assume 'X1' is the name of the first factor column (3 levels: H, L and M)
# assume 'X2' is the name of the second factor column (2 levels: P and T)
# assume 'X3' is the name of the third factor column (3 levels: 1, 2 and 3)
# 'X1' , 'X2' and 'X3' are within-subjects variables
# assume 'Y' is the name of the response column
# run the ART procedure on 'df'
df$X1 = factor(df$X1)
df$X2 = factor(df$X2)
df$X3 = factor(df$X3)
m = art(Y ~ X1 * X2 * X3 + (1|S), data=df) # linear mixed model syntax; see lme4::lmer
summary(m)
This outputs:
Aligned Rank Transform of Factorial Model
Call:
art(formula = Y ~ X1 * X2 * X3 + (1 | S), data = df)
Column sums of aligned responses (should all be ~0):
X1 X2 X3 X1:X2 X1:X3 X2:X3 X1:X2:X3
0 0 0 0 0 0 0
F values of ANOVAs on aligned responses not of interest (should all be ~0):
Min. 1st Qu. Median Mean 3rd Qu. Max.
0 0 0 0 0 0
Which suggests everything is right to run the ANOVA.
anova(m)
Here is the output:
## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: art(Y)
##
## F Df Df.res Pr(>F)
## 1 X1 1326.60837 2 289 < 2.22e-16 ***
## 2 X2 7.65014 1 289 0.0060424 **
## 3 X3 21.37221 2 289 2.2061e-09 ***
## 4 X1:X2 1.22885 2 289 0.2941532
## 5 X1:X3 4.48146 4 289 0.0015789 **
## 6 X2:X3 1.50265 2 289 0.2242741
## 7 X1:X2:X3 0.31126 4 289 0.8703617
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
There is a significant main effect of X2 (F(2,289)=7.6, p=0.006).
We now create a figure representing the mean values for the 2 levels of X2 (P and T) and their 95% CI:
Y_X2_P = df %>% filter(X2 == "P")
Y_X2_P_mean = mean(Y_X2_P$Y)
Y_X2_P_CI_lower = bootES(Y_X2_P$Y)$bounds[1]
Y_X2_P_CI_upper = bootES(Y_X2_P$Y)$bounds[2]
Y_X2_T = df %>% filter(X2 == "T")
Y_X2_T_mean = mean(Y_X2_T$Y)
Y_X2_T_CI_lower = bootES(Y_X2_T$Y)$bounds[1]
Y_X2_T_CI_upper = bootES(Y_X2_T$Y)$bounds[2]
fig <- plot_ly(
x = c("P", "T"),
y = c(Y_X2_P_mean, Y_X2_T_mean),
error_y = list(symmetric=F, array=c(Y_X2_P_CI_upper - Y_X2_P_mean, Y_X2_T_CI_upper - Y_X2_T_mean),
arrayminus=c(Y_X2_P_mean - Y_X2_T_CI_lower, Y_X2_T_mean - Y_X2_T_CI_lower), color = '#000000'),
type = "bar"
)
fig
According to this figure and the overlapping of the 95% CI, it seems very unlikely the significant effect exists.
Also a Wilcoxon test does not find a significant effect:
dataAggr = df %>% group_by(S, X2) %>%
summarize(Y = mean(Y))
P = dataAggr %>% filter(X2 == "P")
T = dataAggr %>% filter(X2 == "T")
wilcox.test(P$Y, T$Y, paired=TRUE)
Output:
Wilcoxon signed rank test with continuity correction
data: P$Y and T$Y
V = 117, p-value = 0.1769
alternative hypothesis: true location shift is not equal to 0
Is there something wrong in the code above? Is it a problem with the dataset or a problem with the ART itself?
Emmeans is the successor to lsmeans
E.g., why Error(Tray) works instead of Error(Tray/Fertilizer)
short answer: there is only one observation per Tray.
*long answer (rough): *
For more on split-plot designs, see an explanation here: http://stats.stackexchange.com/a/13837. Relating that example to the ARTool example, Diet corresponds to Moisture, Time to Fertilizer, and Subject to Tray.
Strictly speaking, following a split-plot design, the model specification would be:
m <- art(DryMatter ~ Moisture*Fertilizer + Error(Tray/Fertilizer), data=Higgins1990Table5)
However, we can simplify the model specification for this particular dataset. Error(Tray/Fertilizer) is equivalent to Error(Tray + Tray:Fertilizer). Since there is only one observation per Tray * Fertilizer in the data, the within-Tray error cannot be distinguished from any interaction between Tray and Fertilizer. Thus, the Tray:Fertilizer error term can be dropped from the model specification, yielding:
m <- art(DryMatter ~ Moisture*Fertilizer + Error(Tray), data=Higgins1990Table5)
You can verify that both specifications yield the same results. If there were multiple observations for each Tray*Fertilizer, this would not be the case, and the first specification (with Error(Tray/Fertilizer)) would be preferred.
(don't close this; will re-use and update for each release)
devtools::install_dev_deps()
update.packages(ask = FALSE)
devtools::check(remote = TRUE, manual = TRUE, run_dont_test = TRUE)
spelling::spell_check_package()
devtools::document()
devtools::install()
revdepcheck::revdep_check(num_workers = 4)
devtools::check(remote = TRUE, manual = TRUE, run_dont_test = TRUE)
devtools::check_win_release()
devtools::check_win_devel()
rhub::check(platform = "solaris-x86-patched", env_vars = c("_R_CHECK_FORCE_SUGGESTS_" = "false"))
cran-comments.md
devtools::release()
After package goes live on CRAN:
dev
onto master
and cran
Template loosely based on r-lib/usethis#338
Hello. I applied art to data having observations from 30 subjects. The dependent variable is ranged between 0 - 100 and independent variable has 3 levels.
art_m_block2<- art(Percentage.Executed.Lane.Changes ~ Adjacant.Gap + (1|Subject.Number), data = data)
After fitting the art, I printed a summary
##Print Summary
summary(art_m_block2)
Is the output correct? When I applied art on Higgins1990Table5, I got 0s in F values of ANOVAs on aligned responses, not of interest (should all be ~0). However, in my model, there are no values. What Am I missing?
Thank you so much for your time.
Here is the output of summary(art_m_block2)
Aligned Rank Transform of Factorial Model
Call:
art(formula = Percentage.Executed.Lane.Changes ~ Adjacant.Gap +
(1 | Subject.Number), data = data)
Column sums of aligned responses (should all be ~0):
Adjacant.Gap
0
F values of ANOVAs on aligned responses not of interest (should all be ~0):
Min. 1st Qu. Median Mean 3rd Qu. Max.
Seems a possibly common mistake to pass numerics to Error() when factors are intended. Might be worth a warning, with an arg to disable.
Using aov. Probably just detect Error terms to make the choice.
When rownames_to_column()
is called on the anova results, the text is replaced with numbers.
# make some data
expand.grid(
aaa = LETTERS[1:4],
bbb = LETTERS[1:4],
subject = 1:10
) %>%
mutate_all(factor) %>%
mutate(measure = rnorm(n())) %>%
# ART ANOVA
art(measure ~ aaa * bbb + Error(subject/bbb), .) %>%
anova() %>%
# get row names as a column
rownames_to_column()
To:
At least partial eta^2 and Cohen's d. Possibly with a cautionary note, and maybe add more explicit support for them.
After using the "art()" function, the "art" class data were put into the "anova" function,then errors remind in R studio :
Error in Anova.III.lm(mod, error, singular.ok = singular.ok, ...) : there are aliased coefficients in the model,
what should i do to remove the aliased data and fix this error?
thanks for your help
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.