oreilly / couchdb-guide Goto Github PK

From http://books.couchdb.org/relax/design-documents/design-documents

There are two main kinds of documents, in my experience. The first kind i
s like something a word processor would save, or a user profile. With that =
sort of data, you want to denormalize as much as you possibly can. Basicall=
y you want to be able to load the document in one request, and get somethin=
g that makes sense enough to display.

Whats the second kind? Also there was no obvious connection for me
between a word processor document and user profile data. So Im
initially asking myself: is the user profile the second kind of
document?

There are two main kinds of documents. The first is similar to what a
word processor would save. You want to denormalize as much as possible
in order to load the document with a single request. That single
request should contain enough information to make sense when
displayed. The second is kind of document is ....

(the virtual document?)

From http://books.couchdb.org/relax/reference/change-notifications

The style option works for long polling request just like for regular polling requests.

s/style/feed ?

s/thier/their/

From http://books.couchdb.org/relax/reference/clustering

Oversharding is a technique where you partition the cluster so that there=
are multiple shards on each physical machine. Moving a partition from one =
machine to another is simpler than spitting it into smaller partitions, as =
the configuration map of the cluster used by the proxy only needs to change=
to point to shards at thier new homes, rather than adding new logical shar=
ds. It=92s also less resource intensive to move a partition than to split i=
t into many.

From http://books.couchdb.org/relax/reference/conflict-management

This is an interesting conflicts scenario in that we helped a British bro
adcasting company build a solution for that is now in production. The basic=
set up is this: to guarantee that the company=92s website are online 24/7 =
even in the event of the loss of a data center they have multiple data cent=
ers backing the website. The "loss" of a data center is a rare occasion, bu=
t it can be as simple as network outages where the data center is still ali=
ve and well, but can=92t be reached by anyone.

Change "company=92s website are" to "company=92s website is" or "company=92=
s
websites are".

From http://books.couchdb.org/relax/reference/clustering

In this section we=92ll see how we can use CouchDB=92s replication filter=
s ...

From http://books.couchdb.org/relax/reference/clustering

In the Replication chapter we discussed how to do filtered replication.

So where is the documentation for replication filters?

From http://books.couchdb.org/relax/design-documents/design-documents

You can treat documents as state machines, with a combination of user inp
ut and background processing managing document state. You=92d use a view by=
state to pull out the relevant document - changing its state would move it=
in the view.

In this section you mention both types of documents but I believe that
you mean you can treat virtual documents as state machines?

From http://books.couchdb.org/relax/reference/change-notifications

Networks are a tricky beast and sometimes you don=92t know whether there =
are no changes coming or your network connection went stale. If you add ano=
ther query parameter heartbeat=3DN, where N is a number CouchDB will send y=
ou a newline character each N milliseconds. As long as you are receiving ne=
w line characters you know there are no new change notifications, but Couch=
DB is still there ready to send you the next one when it occurs.

Question: why would one care about getting a heartbeat signal under
the polling case. Shouldn't this paragraph be under the continuous or
long polling cases. Admittedly, I'm reading this to learn about the
_changes api, so maybe I'm missing something

From http://books.couchdb.org/relax/reference/clustering

Because Lounge allocates a portion of the hash (known as a keyspace) to e=
ach node, you can add as many nodes as you like. Because the hash function =
produces hexidecimal strings that bare no apparent relation to your docids,=
and because we dispatch requests based on the first few characters, we ens=
ure that all nodes see roughly equal load. And bcause the hash function is =
consitent, Lounge will take any arbitrary docid from an HTTP request URI an=
d point it to the same node each time.

Should be:

...bear no apparent...

Also,

...because the hash function is consistent...

"hosts" should be "host's"

From http://books.couchdb.org/relax/design-documents/lists

Now that we=92ve rendered and sent the file=92s head, it=92s time to star=
t rendering the list itself. Each list item will be the result of convertin=
g a view row to a virtual hosts configuration element. The first thing we d=
o is call getRow() to get a row of the view.

From http://books.couchdb.org/relax/reference/conflict-management

You now have a document in each of the databases that has different infor
mation. This situation is called a conflict. Conflicts occur in distributed=
systems. They are a natural state of your data. How does CouchDB=92s repli=
cation system deal with conflicts? [fix story-to-textbook-lingo-switch]

Just drop this paragraph completely -- youve already established the
basic story, and the concept of conflicts will be clear to your
readers. The only meat of this paragraph is the final sentence --
that we care about how CouchDBs replication system deals with
conflicts.

In fact, I suggest you go ahead and tell the reader here how CouchDB
would resolve the phone number conflict, and then get into the nitty
gritty that follows.

From http://books.couchdb.org/relax/design-documents/design-documents

From the second request, you should see a response like:

I wish this section covered building design docs in Futon instead of
using curl.

From http://books.couchdb.org/relax/reference/change-notifications

The continuous changes API allows you to receive change notifications as =
they are coming in using a single HTTP connection. You make a request to th=
e continuous changes API and both you and CouchDB will hold the connection =
open "forever" and CouchDB will send you new lines for notifications when t=
he occur and =97 opposed to long polling =97 keeps the HTTP connection open=
, waiting to send the next notification.

Actually it won't stay open forever - there is a default timeout of
60s. See http://wiki.apache.org/couchdb/HTTP_database_API#Changes.

The req parameter of the filter function includes a member userCtx, the u=
ser context. It includes information about the user that already has been a=
uthenticated over HTTP earlier in the phase of the request. Specifically re=
q.userCtx.name includes the user name of the user that made the filtered =
changes request. We can be sure that the user is who he says he is because =
he has been authenticated against one of the authenticating schemes in Co=
uchDB. With this, we don=92t even need the dynamic filter parameter (althou=
gh it can still be useful in other situations).

From http://books.couchdb.org/relax/design-documents/design-documents

The bigger picture we=92re working on is like an app server, but differen
t in one crucial regard: rather than let the developer do whatever they wan=
t (loop of a list of docids and make queries, make queries based on the res=
ults of other queries, etc) we=92re defining "safe" transformations, like v=
iew, show, list and update.

I think this should be a new section called The Future

And if I may be so bold: You may want to mention the "of the web"
quote and remind the reader of the transformation rails made to all
web development. Then note that that was the first step in the
transformation of web development, to more easily map the UI to the
db, but it assumed the db was correct. If we look at the full backend
stack and consider railsing (bad word) the full backend, then the
relational database stands in the way. Just like mercurial and git
are replacing svn and cvs, and rails did with struts, so too will
couch replace mysql et al.

Maybe too much, but its big and grand, and that section starts with a
sidenote of lameness about update and something else, and tries to end
big, but it could be bigger.

From http://books.couchdb.org/relax/reference/conflict-management

Suppose you are sitting in a coffee shop working on your book. Chris come
s over and tells you about his new phone. The new phone came with a new num=
ber and you have Chris dictate it while you change it using your laptop=92s=
address book application.

If I understand CouchDB correctly, conflict documents will only arise
(asynchronously) when working with multiple (possibly disconnected)
instances of CouchDB. When working with a single instance of CouchDB,
a change will either be stored without conflict or rejected with a
(synchronous) conflict error. Please consider adding this somewhere.

From http://books.couchdb.org/relax/design-documents/design-documents

Design documents are a special type of CouchDB document which contains ap
plication code. Because it runs inside a database, the application API is h=
ighly structured. We=92ve seen JavaScript views, and other functions in the=
previous chapters. , and as talk about how functions in a design document =
are related within applications.

I think the last sentence needs to be reworked. "In this section we=92ll
take a look at how the specific function APIs themselves," doesnt
make sense to me. Perhaps "In this section well take a look at how
the specific APIs function," is what was intended?

There is a typo in coNvering

From http://books.couchdb.org/relax/design-documents/design-documents

There are other design document functions which are being introduced at t
he time of this writing, _update and _filter which we aren=92t convering in=
depth here

From http://books.couchdb.org/relax/reference/clustering

Consistent hashing solves the problem of how to break a single logical da
tabase up evenly across a set of partitions, which can the be distributed a=
cross multiple servers. It does not address the problem of how to ensure th=
at data you=92ve stored is safe from loss due to hardware or software failu=
re. If you are serious about your data, you can=92t consider it saved until=
you have at least two copies of it, preferably in different geographical l=
ocations.

"preferably in different geographical locations". This is the only
mention of having an architecture where all nodes are not in one
location.

The scaling out portion below is based on the assumption that every
front-end node will be serving requests for all types of data and all
users. In fact this could be its own chapter on distributed
architecture with couchdb.

"There are two main kinds of documents, in my experience."

Bad punctuation. Lose the comma.

From http://books.couchdb.org/relax/design-documents/design-documents

There are two main kinds of documents, in my experience. The first kind i
s like something a word processor would save, or a user profile. With that =
sort of data, you want to denormalize as much as you possibly can. Basicall=
y you want to be able to load the document in one request, and get somethin=
g that makes sense enough to display.

From http://books.couchdb.org/relax/design-documents/design-documents

CouchDB=92s Map Reduce queries are stored in the views field. This is how
Futon displays and allows you to edit Map Reduce queries. View indexes are=
stored on a per design doc basis, according to a fingerprint of the functi=
on=92s text contents. This means that if you edit attachments, validations,=
or any other non-view (or language) fields on the design doc, the views wi=
ll not be regenerated. However, if you change a map or a reduce function, t=
he view index will be deleted and a new index built for the new view functi=
ons.

When the term "View indexes" is used in the 3rd sentence, I am
unfamiliar with the reference. I noticed that the usage of the word
"indexes" also appears in previous chapters. Would it be possible to
add more explanation?

From http://books.couchdb.org/relax/reference/clustering

CouchDB Lounge is a proxy-based partitioning and clustering application, =
originally developed for Meebo, a Web-based instant messaging service. Loun=
ge comes with two major components, one that handles simple GET and PUT req=
uests for documents, and the other which distribtues view requests.

typo: distribtues

From http://books.couchdb.org/relax/reference/change-notifications

Networks are a tricky beast and sometimes you don=92t know whether there =
are no changes coming or your network connection went stale. If you add ano=
ther query parameter heartbeat=3DN, where N is a number CouchDB will send y=
ou a newline character each N milliseconds. As long as you are receiving ne=
w line characters you know there are no new change notifications, but Couch=
DB is still there ready to send you the next one when it occurs.

What does this have to do with polling?

Simple polling is request and response and that's it. There should be
no further responses from the server in simple polling mode.

This paragraph should be moved to another section.

"The first kind is like something a word processor would save, or a
user profile."

Bad punctuation. Lose the comma.

From http://books.couchdb.org/relax/design-documents/design-documents

There are two main kinds of documents, in my experience. The first kind i
s like something a word processor would save, or a user profile. With that =
sort of data, you want to denormalize as much as you possibly can. Basicall=
y you want to be able to load the document in one request, and get somethin=
g that makes sense enough to display.

With this file set up, you can push your CouchApp with the command coucha=
pp push, which will push the application to the "default" database. CouchAp=
p also supports alternate environments. To push your application to a devel=
opment database, you could use couchapp push dev. In our experience, taking=
the time to setup a good .couchapprc is always worth it. Another benefit i=
s that it keeps your passwords off the screen when you are working.

Note that JSON syntax uses commas as separators only. Trailing commas
will cause your push to fail!

From http://books.couchdb.org/relax/design-documents/design-documents

CouchDB is designed to work best when there is a one-to-one correspondence between applications and design documents.

Because of its significance, this statement should be emphasized
(italics or bold).

From http://books.couchdb.org/relax/reference/conflict-management

CouchDB guarantees that each instance that sees the same conflict comes u
p with the same winning and losing revisions. It does so by running a deter=
ministic algorithm to pick the winner. Application should not rely on the d=
etails of this algorithm and must always resolve conflicts. We=92ll tell yo=
u how it works anyway.

Change "Application should" to "Applications should"

From http://books.couchdb.org/relax/reference/conflict-management

Let=92s go through an illustrated example of how conflicts emerge an how
to solve them in super slow-motion. Figure 4-1 illustrates the basic setup:=
we have two CouchDB databases and we are replicating from database A to da=
tabase B. To keep this simple we assume triggered replication and not conti=
nuous replication and we don=92t replicate back from database B to A. All o=
ther replication scenarios can be reduced to this setup, so this explains e=
verything we need to know.

Change "emerge an how" to "emerge and how"

From http://books.couchdb.org/relax/reference/change-notifications

Is this para misplaced? What sense does a heartbeat make with a
polling query, which returns immediately even with no changes. Seems
to fit in long polling or continuous modes only.

Networks are a tricky beast and sometimes you don=92t know whether there =
are no changes coming or your network connection went stale. If you add ano=
ther query parameter heartbeat=3DN, where N is a number CouchDB will send y=
ou a newline character each N milliseconds. As long as you are receiving ne=
w line characters you know there are no new change notifications, but Couch=
DB is still there ready to send you the next one when it occurs.

If you merely want to peruse the source code while reading along with the=
book, it is available as standard zip or tar downloads. To get the zip ver=
sion access the following URL from your browser, which will redirect to the=
latest zip file of Sofa. If you prefer, a tar file is available as well.

Wrong link targets: Links points to couchapp instead of sofa

From http://books.couchdb.org/relax/reference/conflict-management

You now have a document in each of the databases that has different infor
mation. This situation is called a conflict. Conflicts occur in distributed=
systems. They are a natural state of your data. How does CouchDB=92s repli=
cation system deal with conflicts? [fix story-to-textbook-lingo-switch]

Just drop this paragraph completely -- youve already established the
basic story, and the concept of conflicts will be clear to your
readers. The only meat of this paragraph is the final sentence --
that we care about how CouchDBs replication system deals with
conflicts.

In fact, I suggest you go ahead and tell the reader here how CouchDB
would resolve the phone number conflict, and then get into the nitty
gritty that follows.

From http://books.couchdb.org/relax/reference/conflict-management

CouchDB does not attempt to merge the conflicting revision. Your applicat
ion dictates how the merging should be done. The choice of picking the winn=
ing revision is arbitrary. In the case of the phone number, there is no way=
for a computer to decide on the right revision. This is not specific to Co=
uchDB, no other software can this (ever had your contacts sync tool for you=
r phone ask you which contact from which source to take?).

Missing "do": no other software [do] can this

s/benifits/benefits/

From http://books.couchdb.org/relax/reference/clustering

To frame the same concept with Web architecture, because CouchDB uses HTT=
P, the proxy can partition documents according to the request URL, without =
inspecting the body. This is a core principal behind REST, and is one of th=
e many benifits using HTTP affords us. In practice this is accomplished by =
running the hash function against the request URI, and comparing the result=
to find the the portion of the keyspace allocated. Lounge then looks up th=
e associated shard for the hash in a configuration table, forwarding the HT=
TP request to%2

"We=92ve seen JavaScript views, and other functions in the previous
chapters." has a misplaced comma. Lose the comma so it reads:
"We=92ve seen JavaScript views and other functions in the previous
chapters."

Also, the sentence is false. Views are covered in the next chapter,
not the previous one.

From http://books.couchdb.org/relax/design-documents/design-documents

Design documents are a special type of CouchDB document which contains ap
plication code. Because it runs inside a database, the application API is h=
ighly structured. We=92ve seen JavaScript views, and other functions in the=
previous chapters. In this section we=92ll take a look at the function API=
s, and talk about how functions in a design document are related within app=
lications.

From http://books.couchdb.org/relax/reference/clustering

Consistent hashing solves the problem of how to break a single logical da
tabase up evenly across a set of partitions, which can the be distributed a=
cross multiple servers. It does not address the problem of how to ensure th=
at data you=92ve stored is safe from loss due to hardware or software failu=
re. If you are serious about your data, you can=92t consider it saved until=
you have at least two copies of it, preferably in different geographical l=
ocations.

"preferably in different geographical locations". This is the only
mention of having an architecture where all nodes are not in one
location.

The scaling out portion below is based on the assumption that every
front-end node will be serving requests for all types of data and all
users. In fact this could be its own chapter on distributed
architecture with couchdb.

From http://books.couchdb.org/relax/reference/clustering

To frame the same concept with Web architecture, because CouchDB uses HTT
P, the proxy can partition documents according to the request URL, without =
inspecting the body. This is a core principal behind REST, and is one of th=
e many benifits using HTTP affords us. In practice this is accomplished by =
running the hash function against the request URI, and comparing the result=
to find the the portion of the keyspace allocated. Lounge then looks up th=
e associated shard for the hash in a configuration table, forwarding the HT=
TP request to the back-end CouchDB server.

s/benifits/benefits

From http://books.couchdb.org/relax/reference/clustering

This chapter could use some diagrams. For example, take the N-
partition and spread it across N servers. Or show how it can be
spread across M servers.

"wel" should be "well"

From http://books.couchdb.org/relax/design-documents/lists

Headers give list and show functions the ability to provide the content type response that the client prefers, as wel as other nifty things, like cookies. Note that cookies are also parsed into a JSON representation. Thanks Mochiweb!

From http://books.couchdb.org/relax/reference/clustering

Consistent hashing solves the problem of how to break a single logical da
tabase up evenly across a set of partitions, which can the be distributed a=
cross multiple servers. It does not address the problem of how to ensure th=
at data you=92ve stored is safe from loss due to hardware or software failu=
re. If you are serious about your data, you can=92t consider it saved until=
you have at least two copies of it, preferably in different geographical l=
ocations.

"preferably in different geographical locations". This is the only
mention of having an architecture where all nodes are not in one
location.

The scaling out portion below is based on the assumption that every
front-end node will be serving requests for all types of data and all
users. In fact this could be its own chapter on distributed
architecture with couchdb.

From http://books.couchdb.org/relax/reference/change-notifications

The req parameter of the filter function includes a member userCtx, the u=
ser context. It includes information about the user that already has been a=
uthenticated over HTTP earlier in the phase of the request. Specifically re=
q.userCtx.name includes the user name of the user that make the filtered ch=
anges request. We can be sure that the user is who he says he is because he=
as ben authenticated against one of the authenticating schemes in CouchDB.=
With this, we don=92t even need the dynamic filter parameter (although it =
can still be useful in other situations).

s/he as ben/he has been/

From http://books.couchdb.org/relax/reference/conflict-management

Suppose you are sitting in a coffee shop working on your book. Chris come
s over and tells you about his new phone. The new phone came with a new num=
ber and you have Chris dictate it while you change it using your laptop=92s=
address book application.

If I understand CouchDB correctly, conflict documents will only arise
(asynchronously) when working with multiple (possibly disconnected)
instances of CouchDB. When working with a single instance of CouchDB,
a change will either be stored without conflict or rejected with a
(synchronous) conflict error. Please consider adding this somewhere.

From http://books.couchdb.org/relax/reference/conflict-management

Finally, we tell CouchDB which version we like to be latest revision by resolving the conflict. Now both databases have the same data.

Change "version we like to be" to "version we would like to be"

From http://books.couchdb.org/relax/design-documents/lists

It seems like the (intended?) default Content-Type for show
functions is text/html, while for the list functions it's
application/json -- it would be good to mention this in the "List
functions" chapter. (I had to look in the test suite to discover start
({"headers":...}) function.) Also, in the final text, maybe a cross-
reference to the "Viewing Lists of Blog Posts" chapter, specifically
to the provides() function would be great.

--karmi

"he as ben" presumably should be "he has been"

From http://books.couchdb.org/relax/reference/change-notifications

The req parameter of the filter function includes a member userCtx, the u=
ser context. It includes information about the user that already has been a=
uthenticated over HTTP earlier in the phase of the request. Specifically re=
q.userCtx.name includes the user name of the user that make the filtered ch=
anges request. We can be sure that the user is who he says he is because he=
as ben authenticated against one of the authenticating schemes in CouchDB.=
With this, we don=92t even need the dynamic filter parameter (although it =
can still be useful in other situations).

s/Consertive/Conservative/

From http://books.couchdb.org/relax/reference/clustering

We=92ve seen that even a cluster with depth of 2 can hold a vast amount o=
f data. Basic arithmetic shows us the by applying the same process to creat=
e a cluster with 3 layers of proxies, we can manage 262 petabytes on thousa=
nds of machines. Consertive estimates for the latency introduced by each la=
yer is about 100 ms, so even without performance tuning we should see overa=
ll response times of 300ms even with a tree of depth 3, and should be able =
to manage queries over exabyte datasets in under a second.

From http://books.couchdb.org/relax/design-documents/design-documents

A design document is a CouchDB document with an id that begins with _desi
gn/. For instance, the example blog application, Sofa, is stored in a desig=
n document with the id _design/sofa. Design documents are just like any oth=
er CouchDB document: they replicate along with the other documents in their=
database, and track edit conflicts with the rev parameter

This paragraph seems to be written from the point of view of the
CouchDB developer rather than the (newbie) CouchDB user. I would
write it this way:

It is stored in the database like any other document. As such, it
replicates along with the other documents in the applications
database. Edit conflicts are tracked with the rev parameter.

The design document is identified by an id that begins with _design/.
The Sofa application would have a design document identified by
_design/sofa.

From http://books.couchdb.org/relax/reference/conflict-management

As long as all end users can get to their data, the split brain is not sc
ary. Resolving the split brain situation and bringing up the connection tha=
t links the data centers and staring synchronization again is where it gets=
hairy. Arbitrary conflict resolution, like CouchDB does by default can lea=
d to unwanted effects on the user=92s side. Data could revert to an earlier=
stage and leave the impression that changes weren=92t reliably saved while=
in fact they were.

Change "staring synchronization" to "starting synchronization"

From http://books.couchdb.org/relax/reference/clustering

Using CouchDB at web scale likely requires CouchDB clusters that can be s=
caled dynamically. Growing sites must continuously add more storage capacit=
y, so we need a strategy to increase the size of our cluster without taking=
it down. Some workloads can result in temporary growth in data size, in wh=
ich case we=92ll also need a process for shrinking the cluster without an i=
nterruption in service.

I think diagram(s) or config file examples would be helpful for this
entire section. The phrase, "A picture is worth a thousand words"
certainly applies here. IMHO, clustering in a scalable fashion will
be a deciding factor for those considering CouchDB in the enterprise.

From http://books.couchdb.org/relax/reference/conflict-management

You now have a document in each of the databases that has different infor
mation. This situation is called a conflict. Conflicts occur in distributed=
systems. They are a natural state of your data. How does CouchDB=92s repli=
cation system deal with conflicts? [fix story-to-textbook-lingo-switch]

Just drop this paragraph completely -- youve already established the
basic story, and the concept of conflicts will be clear to your
readers. The only meat of this paragraph is the final sentence --
that we care about how CouchDBs replication system deals with
conflicts.

In fact, I suggest you go ahead and tell the reader here how CouchDB
would resolve the phone number conflict, and then get into the nitty
gritty that follows.

From http://books.couchdb.org/relax/reference/conflict-management

Suppose you are sitting in a coffee shop working on your book. Chris come
s over and tells you about his new phone. The new phone came with a new num=
ber and you have Chris dictate it while you change it using your laptop=92s=
address book application.

If I understand CouchDB correctly, conflict documents will only arise
(asynchronously) when working with multiple (possibly disconnected)
instances of CouchDB. When working with a single instance of CouchDB,
a change will either be stored without conflict or rejected with a
(synchronous) conflict error. Please consider adding this somewhere.