master and dev-version should be enough for now, right?

Give a little more explanation what the book is about, where the project is at right now and where it is headed?

Deploys from Travis can be found here https://advanced-r-solutions.rbind.io

Next steps:

• set up CI for https://github.com/henningsway/Advanced-R-Solutions
• make Travis play nice with packrat
  • useful links
    • rstudio/packrat#330
    • https://github.com/BlackDuckCoPilot/example-packrat-travis
    • rstudio/packrat#352
• avoid using knitr cache where possible (to reduce build time on travis)
  no need, because knitr cache could be enabled for travis also
• test with for some time with the fork (merge upstream changes or so)
• (optional) update packages
  (won't do, because the sources will be saved and the repo will increase in size without much benefit)
• set up CI for main repo
• (optional) bring book to bookdown.org (see rstudio/bookdown.org#25)
  • link book on bookdown-website to new netlify url (https://bookdown.org/home/about/#how-to-get-your-book-listed-on-bookdown-org) or
  • push to RSConnect from Travis *https://github.com/slopp/rsc-ci-test
    Not planned atm
• (eventually): speed up rendering on book on Travis (relies on knitr-cache, which is only available locally atm)

Let's eventually try to finish one chapter to see, what kind of result we are aiming for and what steps are needed to get there.

We can do this, once the 90% milestone is reached.

TO DO

• Modify-in-place -> 1. Explain why the following code doesn’t create a circular list...
• improve first exercise of copy-on-modify
• double check graphic (remove x)
• check if 2nd exercise of object size can be improved
  • comment: don't think I can improve here.
  • @Tazinho why is there a code chunk as well as inline-code in this answer?
  • Edit: I think it might be nice to give the answer in a sentence (inline code), but also provide the code on how to get there for the reader.
• check 3rd object-size exercise some time
  • not us right now :)
  • observe hadley/adv-r#1324
• fix graphic for last exercise (median subtraction)
• @henningsway read once through the chapter
• @henningsway review the final exercise (3rd object size ["predicition of object sizes"])

chapter expressions:

assign3 <- function(x, value, env = parent.frame()){
      eval(substitute(x <- as.name(x)), env)
  eval(substitute(x <- value), env)
}

Q: The following two calls look the same, but are actually different:

(a <- call("mean", 1:10))
#> mean(1:10)
(b <- call("mean", quote(1:10)))
#> mean(1:10)
identical(a, b)
#> [1] FALSE

What’s the difference? Which one should you prefer?

A: call evalulates its ... arguments. So in the first call 1:10 will be evaluated
to an integer (1, 2, 3, …, 10) and in the second call quote()
compensates the effect of the evaluation, so that b’s second element will be
the expression 1:10 (which is again a call):

  as.list(a)
#> [[1]]
#> mean
#> 
#> [[2]]
#>  [1]  1  2  3  4  5  6  7  8  9 10
as.list(b)
#> [[1]]
#> mean
#> 
#> [[2]]
#> 1:10

We can create an example, where we can see the consequences directly:

  # h <- call("mean", z)
  ##> Error: object 'z' not found

  h <- call("mean", quote(z))
# eval(h)
##> Error in mean(z) : object 'z' not found
z <- 1:10
eval(h)
#> [1] 5.5

From my intuition I would prefer the second version,
since it behaves more like lazy evaluation,
but I am not sure if this is the correct answer as indended by the exercise.

Let's see what we still have to do content-wise.

Malte will create and commit an Rmd-file, which contains unsolved exercises. When new exercises should come up in the 2nd Ed. we will also add them here.

Vectors

Atomic vectors

• 1. How do you create scalars of type raw and complex? (See ?raw and ?complex)...
• 2. Test your knowledge of vector coercion rules by predicting the output of the following...
• 3. Why is 1 == "1" true? Why is -1 < FALSE true? Why is "one" < 2 false?...
• 4. Why is the default missing value, NA, a logical vector? What’s special about logical...
• 5. Precisely what do is.atomic(), is.numeric(), and is.vector() test for?...

Attributes

• 1. How is setNames() implemented? How is unname() implemented? Read the source...
• 2. What does dim() return when applied to a 1d vector? When might you use NROW() or...
• 3. How would you describe the following three objects? What makes them different to ...
• 4. An early draft used this code to illustrate structure():...

S3 atomic vectors

• 1. What sort of object does table() return? What is its type? What attributes does it ...
• 2. What happens to a factor when you modify its levels?...
• 3. What does this code do? How do f2 and f3 differ from f1?...

Lists

• 1. List all the ways that a list differs from an atomic vector....
• 2. Why do you need to use unlist() to convert a list to an atomic vector? Why doesn’t...
• 3. Compare and contrast c() and unlist() when combining a date and date-time into a ...

Data frames and tibbles

• 1. Can you have a data frame with 0 rows? What about 0 columns?....
• 2. What happens if you attempt to set rownames that are not unique?...
• 3. If df is a data frame, what can you say about t(df), and t(t(df))? Perform...
• 4. What does as.matrix() do when applied to a data frame with columns of different ...

I would like to discuss this excercise in detail. (edit: the solution can be found here and looks really good https://github.com/asnr/advanced-r-answers/blob/master/metaprogramming/nse.md just want to understand a bit more in very detail before updating...)

This is the solution from the book.

Q: What happens if you use quote() instead of substitute() inside of subset2()?

A: We will get "Error in eval(expr, envir, enclos) : object 'x' not found" in the above example. We are not sure if the problem relates to finding the right expression or the correct environment for evaluation.

This is the solution from the repo

What happens if you use quote() instead of substitute() inside of subset2()?

subset4 <- function(x, condition) {
  condition_call <- substitute(condition)
  r <- eval(condition_call, x)
  x[r & !is.na(r), , drop = FALSE]
}

subset5 <- function(x, condition) {
  condition_call <- quote(condition)
  r <- eval(condition_call, x)
  x[r & !is.na(r), , drop = FALSE]
}

sample_df2 <- data.frame(x = 1:10)
subset4(sample_df2, x > 8)
#     x
# 9   9
# 10 10

subset5(sample_df2, x > 8)
# Error in eval(expr, envir, enclos) : object 'x' not found

# Why is this the case?  Remember that the difference between `substitute` and `quote`
# is that `quote` just returns the expression whereas `substitute` can modify the
# expression within an environment, performing substitutions.
#
# Therefore `substitute` will contain the information about what the dataframe is, whereas
# `quote` will not.

Discussion

I am not happy with both. The first one is not finished and the 2nd one is not detailed enough...
I would really like to understand better, why eval(condition, sample_df2) here (when called inside a function and condition is a supplied expression which is quoted inside the function) does not work, while many details are unclear. See the following:

> rm(list = ls(all = TRUE))
> # This works
> subset2 <- function(x, condition) {
+     condition_call <- substitute(condition)
+     r <- eval(condition_call, x)
+     x[r, ]
+ }
> 
> sample_df2 <- data.frame(x = 1:10)
> subset2(sample_df2, x > 8)
[1]  9 10
> 
> # This fails
> subset2q <- function(x, condition) {
+     condition_call <- quote(condition)
+     r <- eval(condition_call, x)
+     x[r, ]
+ }
> subset2q(sample_df2, x > 8)
Error in eval(expr, envir, enclos) : object 'x' not found 
> # However, outside of a function both work
> eval(substitute(x>8), sample_df2)
 [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE
> eval(quote(x>8), sample_df2)
 [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE
> 
> # It also does use x (or other vars), when x is in the global environment
> x = 1:100
> subset2q(sample_df2, x > 8)
 [1]  9 10 NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[22] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[43] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[64] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[85] NA NA NA NA NA NA NA NA
> rm(x)
> y = 1:100
> subset2q(sample_df2, y > 8)
 [1]  9 10 NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[22] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[43] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[64] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[85] NA NA NA NA NA NA NA NA
> 
> # We can look where x is evaluated after substituting / quoting
> test_quote <- function(x, condition) {
+     condition_call <- quote(condition)
+     x <- list2env(x)
+     y <- environment()
+     list(x_env = x, env = y, eval(condition_call, x))
+ }
> test_quote(sample_df2, {list(eval_env = environment(), par_eval_env = parent.frame())})
$x_env
<environment: 0x00000000165fc0f0>

$env
<environment: 0x00000000165fad90>

[[3]]
[[3]]$eval_env
<environment: R_GlobalEnv>

[[3]]$par_eval_env
<environment: R_GlobalEnv>


> 
> test_subs <- function(x, condition) {
+     condition_call <- substitute(condition)
+     x <- list2env(x)
+     y <- environment()
+     list(x_env = x, env = y, eval(condition_call, x))
+ }
> test_subs(sample_df2, {list(eval_env = environment(), par_eval_env = parent.frame())})
$x_env
<environment: 0x00000000165c5840>

$env
<environment: 0x00000000165fe478>

[[3]]
[[3]]$eval_env
<environment: 0x00000000165c5840>

[[3]]$par_eval_env
<environment: 0x00000000165c53e0>


> # So when quoted expressions are evaluated inside a function it seems that they
> # are always evaluated inside the global environment. Is that really the message
> # from this exercise or is sth. different happening here. Especially, what is
> # happening inside substitute, which is not happening inside quote ?
> 
> # This doesn't seem to be the only difference.
> f_q <- function(cond)quote(cond)
> f_q(a > b)
cond
> f_s <- function(cond)substitute(cond)
> f_s(a > b)
a > b
> # Also note that this would work:
> subset2q_cheet <- function(x, condition) {
+     condition_call <- quote(condition)
+     r <- eval(quote(x > 8), x)
+     x[r, ]
+ }
> subset2q_cheet(sample_df2, anything)
[1]  9 10
> # Since this time the correct environment is found...
> subset2q_cheet2 <- function(x, condition) {
+     condition_call <- quote(condition)
+     x = list2env(x)
+     y = environment()
+     list(x_env = x, env = y, eval(quote({list(x > 8, environment())}), x))
+     # x[r, ]
+ }
> subset2q_cheet2(sample_df2, anything)
$x_env
<environment: 0x00000000157d63a8>

$env
<environment: 0x00000000157d5fa8>

[[3]]
[[3]][[1]]
 [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE

[[3]][[2]]
<environment: 0x00000000157d63a8>

> # Superstrange that this works
> subset2q <- function(x, condition) {
+     condition_call <- condition
+     r <- eval(condition_call, x)
+     x[r, ]
+ }
> subset2q(sample_df2, quote(x > 8))
[1]  9 10

see here
https://github.com/peterhurford/adv-r-book-solutions
and here
peterhurford/adv-r-book-solutions#37

Functions

• Let's also check: #65

Function fundamentals

• 3. A good rule of thumb is that an anonymous function should fit on one line and shouldn’t need to ...

Lazy evaluation

• 4. What does this function return? Why? Which principle does it illustrate?

Function forms

• 7. What are valid names for user-created infix functions?

Hadley has started to include some "Parts" from I to V in his book. -> compare https://adv-r.hadley.nz/

I think it would be nice to find out how, these were created and to style our sidebar in a similar way.

see here for the development version
http://adv-r.hadley.nz/

When the new version is available, we will change also the order of the solutions to match the new adv r version.

Excercises, which are not in the new version, will move to the appendix of the solutions book

Edit: Update Advanced R Version link to 2nd Edition, when we finished the exercises / review

... back to the old one.

Afterwards I will minimize the bookdown version into a one-page book and finish the redirection issue
#86

Goal:
Enable users and contributers which fork the project to directly build the book.

Next steps:

• locally recompile R 3.5.1 with -enable-memory-profiling so all exercises can be rendered
• assure that book can be build directly after cloning https://github.com/henningsway/Advanced-R-Solutions
• eventually integrate with master of our main repository

List for @hadley
Will be updated soon.

In general the status of solutions is at the start of the solutions book prerequisites.

@henningsway Please post any questions to exercises, especially unclear exercises, here (if there is no better place). I will update posted questions in the start post to have everything in one place.

The order is the same as in the new version of Advanced R.

I Foundations

Names and Values

• hadley/adv-r#1292

Data structures

Subsetting

4. Functions

• 4.6.2.4 (on.exit() and capture.output() / capture.output2()) is a bit cumbersome and our answer should be more precise. Maybe a hint in the question?

Memory

Environments

Debugging, condition handling, and defensive programming

II Functional programming

Functional programming

Functionals

Function operators

III Object oriented programming

Introduction

Base types

S3

• 13.2.5.6 "What would a constructor function for lm objects, new_lm(), look like? Why is a constructor function less useful for linear models?"
  • I have no idea about the 2nd part. I could imagine everything and nothing, since I am not that experienced in OO-Systems.

S4

• 14.2.5.1 "In the definition of the generic, why is it necessary to repeat the name of the generic twice?"
  • I am not sure in which direction this should go. Is it ment literally? I think that we need it for method dispatch and prevent the automatic construction of an ANY method, but I am not sure if this is really the intention of this question.

R6

Trade-offs

IV Metaprogramming

Introduction

Expressions:

• 18.1.1.2 (still don't understand, maybe add a code example in answer of both ways)

Tidy evaluation

Non-standard evaluation

Code generation

Translating R Code

• E1. it is not clear why </script> would need to be escaped

V Performance

Performance

Optimising Code

High performance functions with Rcpp

and also higher resolution for the cover.

• https://advanced-r-solutions.rbind.io/names-and-values.html

Solve

Copy-on-modify

Object Size

Modify-in-place

• 1. Explain why the following code doesn’t create a circular list.

• make a proposal for Q&A layout also

Currently under revision:
Malte & Henning: OoField guide in new AdvR version
Peter: Expressions (mostly done in #33; some comments of the pull request have to be cleared)
Robert (+ help of Peter): Domain Specific Languages

this can be done via rmarkdowns spellchecker. after #14 and #15

• check every `
• check every codespace within `` and codeblocks if they have brackets
• make sure that chunks are in inconsolata font
• check code chunks. Handle intendier errors via chunk options instead of several workarounds...
• check that inline comments have 2 spaces in front
• check full stop after every sentence, also in front of code blocks (here mostly colons).

• also send http to https

https://yihui.name/en/2017/11/301-redirect/

Hey ya'll,

I appreciate you putting together solutions for this lovely book.

I don't think your solution to 32.1 is correct though. I believe we should be capturing the user input as part of the function like this:

is.expression_component <- function(x) {
  x = substitute(x)
  (is.atomic(x) & length(x) == 1) | is.call(x) | is.name(x) | is.pairlist(x)
}

The reason for this R will evaluate the input when you call is.atomic(x) making the call is.expression_component(x <- 1) return true because is.atomic(x) is true. It should return true because is.call(x) is true.

Let me know if you disagree!

When starting with an exercise a lot of things can be confusing. However, we already produced some content that might be easy to refactorize. Here is the place to collect any guidance on this topic.

• Old exercises:
  • Sometimes Hadley drops some exercises or reformulates them. When we update the exercises in our book, we don't drop these exercises, but collect them at the bottom under a paragraph "old exercises". It might be really helpful to look there first. When you're lucky the exercise was already solved. (Note: exercises might also be moved between chapters)
  • Answers to the first edition of our book (look at the last chapters in our sidebar)
  • Answers from Roberts and Peters repository. We didn't merge the solutions completely so there might be a good chance that an exercise (or a very similar old one) is already solved.
• If you really have no idea (after some serious thinking):
  • Open an issue here. If you're eager also open an issue at @Hadley`s Advanced R repo and reference to the issue in our repo.
  • The same holds for exercises that you can answer but need to be improved on @hadley's side.

This looks great. Can I "bless" this solution set in some way?

Subsetting

Selecting multiple arguments

• 1. Fix each of the following common data frame subsetting errors:...
• 2. Why does x <- 1:5; x[NA] yield five missing values? (Hint: why is it...
• 3. What does upper.tri() return? How does subsetting a matrix with it...
• 4. Why does mtcars[1:20] return an error? How does it differ from the...
• 5. Implement your own function that extracts the diagonal entries from a...
• 6. What does df[is.na(df)] <- 0 do? How does it work?

Selecting a single element

• 1. Brainstorm as many ways as possible to extract the third value from...
• 2. Given a linear model, e.g., mod <- lm(mpg ~ wt, data = mtcars)...

Applications

• 1. How would you randomly permute the columns of a data frame? (This...
• 2. How would you select a random sample of m rows from a data...
• 3. How could you put the columns in a data frame in alphabetical order?...

Names and values

Binding basics

• 1. Explain the relationship between a, b, c and d in the following code:...
• 2. The following code accesses the mean function in multiple different ways. ...
• 3. By default, base R data import functions, like read.csv(), will...
• 4. What rules does make.names() use to convert non-syntactic names into ...
• 5. I slightly simplified the rules that govern syntactic names. Why is .123e1...

Copy-on-modify

• 1. Why is tracemem(1:10) not useful?
• 2. Explain why tracemem() shows two copies when you run this code. Hint:...
• 3. Sketch out the relationship between the following objects:
• 4. What happens when you run this code:

Object size

• 1. In the following example, why are object.size(y) and obj_size(y) so ...
• 2. Take the following list. Why is its size somewhat misleading?...
• 3. Predict the output of the following code:...

Modify-in-place

• 1. Explain why the following code doesn’t create a circular list.
• 2. Wrap the two methods for subtracting medians into two functions, then use ...
• 3. What happens if you attempt to use tracemem() on an environment?

I can't remember exactly how you do this, but it would make it contributions from the community much more likely.

This would make pull requests more straightforward. It would seem also be preferable to name the files according to their chapter. :)

Conditions

Signalling conditions

• 2. What does the appendLF argument to message() do? How is it related to cat()?

Custom-conditions

• 1. Inside a package, it’s occassionally useful to check that a package is installed before using it...
• 2. Inside a package you often need to stop with an error when something is not right. Other ...

Applications

• 1. Create suppressConditions() that works like suppressMessages() and supressWarnings() but ...
• 2. Compare the following two implementations of message2error(). What is the main advantage of ...
• 3. How would you modify the catch_cnds() defined if you wanted to recreate the original ...
• 4. Why is catching interrupts dangerous? Run this code to find out. ...

(At top page on https://github.com/Tazinho/Advanced-R-Solutions)

after solving #14

• Please add new url to repo

• also clearly deprecate content (and or redirect) on bookdown url.
  • plz also see rstudio/bookdown.org#27 (comment)

Thanks. :)

Functions

Function fundamentals

• 1. Given name, like "mean", match.fun() lets you find a function...
• 2. It’s possible (although typically not useful) to call an anonymous...
• 3. A good rule of thumb is that an anonymous function should fit on one...
• 4. What function allows you to tell if an object is a function? What...
• 5. This code makes a list of all functions in the base package....
• 6. What are the three important components of a function?...
• 7. When does printing a function not show what environment it was...

Lexical Scoping

• 1. What does the following code return? Why? Describe how each of the...
• 2. What are the four principles that govern how R looks for values?...
• 3. What does the following function return? Make a prediction before...

Lazy evaluation

• 1. What important property of && make x_ok() work?...
• 2. What does this function return? Why? Which principle does it ...
• 3. What does this function return? Why? Which principle does it ...
• 4. In hist(), the default value of xlim is range(breaks), the default ...
• 5. Explain why this function works. Why is it confusing?
• 6. How many arguments are required when calling library()?

... (dot-dot-dot)

• 1. Explain the following results:...
• 2. In the following call, explain how to find the documentation for the....
• 3. Why does plot(1:10, col = "red") only colour the points, not the...

Exiting a function

• 1. What does load() return? Why don’t you normally see these...
• 2. What does write.table() return? What would be more useful?...
• 3. How does the chdir parameter of source() compare to...
• 4. Write a function that opens a graphics device, runs the supplied code...
• 5. We can use on.exit() to implement a simple version of ...

Function forms

• 1. Rewrite the following code snippets into prefix form:...
• 2. Clarify the following list of odd function calls:...
• 3. Explain why the following code fails:...
• 4. Create a replacement function that modifies a random location in a...
• 5. Write your own version of + that will paste its inputs together if they...
• 6. Create a list of all the replacement functions found in the base ...
• 7. What are valid names for user-created infix functions?...
• 8. Create an infix xor() operator....
• 9. Create infix versions of the set functions intersect(), union(), ...

Process:

• capture state of 06.10.18
• check subsection titles for changes
• update exercises
  • add new ones
  • delete/ move old ones
• enumerate accordingly (1.,2.,3. instead of 1.,1.,1.,...)
• mark started exercises as orange and check if it has a todo
• add libraries to the start under prerequisities
• add indentation (3 spaces in front of text)

Will try to structure some of my open questions in this issue.

remove files that are not strictly necessary for the compilation of the book from version control.

Attempting to solve

E) 19.2.6.1 - E) 19.2.6.4

hadley/adv-r#800

Environments

Environment basics

• 4. Explain why e[[1]] and e[c("a", "b")] don’t make sense when e is an environment.
• 5. Create a version of env_poke() that will only bind new names, never re-bind old names. Some ...

Recursing over environments

• 1. Modify where() to return all environments that contain a binding for name. Carefully think through what type of object the function will need to return.

Special environments

• 1. How is search_envs() different fo env_parents(global_env())?
• 2. Draw a diagram that shows the enclosing environments of this function ...

The call-stack

• 1. Write a function that lists all the variables defined in the environment in which it was called. It ...

also

• reduce picture size (some of them are a little too large)

Use a tribble in the rmarkdown chunk instead of an external xls file.

Vectors

Atomic vectors

• 1. How do you create scalars of type raw and complex? (See ?raw and ?complex)
• 5. Precisely what do is.atomic(), is.numeric(), and is.vector() test for?

Attributes

• 1. How is setNames() implemented? How is unname() implemented? Read the source code.
• 2. What does dim() return when applied to a 1d vector? When might you use NROW() or NCOL()?

S3 atomic vectors

• 1. What sort of object does table() return? What is its type? What attributes does it have? How ...

Lists

• 1. List all the ways that a list differs from an atomic vector.
• 3. Compare and contrast c() and unlist() when combining a date and date-time into a single vector.

Data frames and tibbles

• 2. What happens if you attempt to set rownames that are not unique?
• 3. If df is a data frame, what can you say about t(df), and t(t(df))? Perform some ...

moved to and merged with #123

18.3.3 Exercises ("Expressions") #5 asks:

How does mode<- work? How does it use call()?

The solution given in our book annotates a particular function. However, it looks like the book annotates mode. It doesn't look like mode contains call(), though it does contain "call".

I think we should be annotating mode<-, not mode. But mode<- also does not use call at all.

The answer I gave in my repo states:

It dynamically creates a transformation function (e.g., as.numeric,as.logical, etc.) by using paste0, and then executes that by using eval(call()) with that function and the item to be transformed.

The first part is accurate but it doesn't look like mode<- uses eval(call()). I thought maybe this is an R version problem, but @robertzk confirms mode<- in R 3.3 is still the same. I also can't find anything about it in the R CHANGELOG.

What am I missing?

The content and the exercises of Advanced R 2nd Ed are still changing (less and less), see hadley/adv-r#1259

At the moment the process of checking for changes in Adv R and updating our book accordingly (e.g. #74) is manual and somewhat tedious, but doable.

We also need to update a list of open exercises (#73), when Adv R changes.

If anyone has a smart ideas, how to improve these processes, please sketch them out here.