409 Conflict on attempt to add new document

[Roger Rohrbach]

I just ran a program to add 6,412 new documents to a freshly-created, empty database.

All of the documents were added, but CouchDB reported an error (HTTP status 409 Conflict) for one of them. This is troubling for several reasons:

The document did not previously exist, but was being added for the first time, like all the others. It was also in no way different than those; in particular, the JSON did not contain a _rev field.
Despite the error, the document was successfully added.

This would appear, therefore, to be a spurious error. Could this be a bug? This is Cloudant's hosted version (CouchDB 1.0.2, Cloudant build 1.3.22).

[Marcello Nuccio]

Are you really sure all 6,412 documents have different _id?
Marcello

Post by Roger Rohrbach
I just ran a program to add 6,412 new documents to a freshly-created, empty database.
The document did not previously exist, but was being added for the first time, like all the others.  It was also in no way different than those; in particular, the JSON did not contain a _rev field.
Despite the error, the document was successfully added.
This would appear, therefore, to be a spurious error.  Could this be a bug?  This is Cloudant's hosted version (CouchDB 1.0.2, Cloudant build 1.3.22).

[Robert Newson]

Hi Roger,

Cloudant makes 3 copies of each document and, internally, uses a
custom replicator to ensure that all 3 copies are created even if some
copies failed during initial creation. What I suspect has happened
here is that the internal replicator beat you to it. Thus, you'd get a
409, because the document arrived at a node prior to your request as a
result of your request arriving at a different node, but earlier.

BigCouch 0.4 modifies the quorum logic in this scenario to return a
202 status code, instead of 409.

B.

Post by Marcello Nuccio
Are you really sure all 6,412 documents have different _id?
Marcello

Post by Roger Rohrbach
I just ran a program to add 6,412 new documents to a freshly-created, empty database.
The document did not previously exist, but was being added for the first time, like all the others.  It was also in no way different than those; in particular, the JSON did not contain a _rev field.
Despite the error, the document was successfully added.
This would appear, therefore, to be a spurious error.  Could this be a bug?  This is Cloudant's hosted version (CouchDB 1.0.2, Cloudant build 1.3.22).

[Roger Rohrbach]

Robert,

Ah, so it's a BigCouch issue. Thanks for the explanation.

What can I do about this in the interim? I guess I have to check to see if the document was inserted whenever I receive a 409 after PUTing the first revision of a document (as I think of it, I'm pretty sure I was also seeing this with attachments)?

As an aside, I'm not sure 202 is the right status to return in this case. The replication you're describing is an implementation detail, and in fact at least one copy of the document was stored in this example. So, unless BigCouch exposes this replication externally (e.g., provides a link with the 202 response that lets clients determines whether replication completed successfully), I'd say that 200 is appropriate. A non-commital response, absent the ability to follow up, leaves the client in an uncertain position.

Roger

Post by Robert Newson
Cloudant makes 3 copies of each document and, internally, uses a
custom replicator to ensure that all 3 copies are created even if some
copies failed during initial creation. What I suspect has happened
here is that the internal replicator beat you to it. Thus, you'd get a
409, because the document arrived at a node prior to your request as a
result of your request arriving at a different node, but earlier.
BigCouch 0.4 modifies the quorum logic in this scenario to return a
202 status code, instead of 409.

[Roger Rohrbach]

Yes.

Post by Marcello Nuccio
Are you really sure all 6,412 documents have different _id?
Marcello

Post by Roger Rohrbach
I just ran a program to add 6,412 new documents to a freshly-created, empty database.
The document did not previously exist, but was being added for the first time, like all the others. It was also in no way different than those; in particular, the JSON did not contain a _rev field.
Despite the error, the document was successfully added.
This would appear, therefore, to be a spurious error. Could this be a bug? This is Cloudant's hosted version (CouchDB 1.0.2, Cloudant build 1.3.22).