[Yanel-dev] Fwd: Cluster documentation

[Michael Wechner]

On 2/2/11 10:56 PM, Cedric Staub wrote:
> On Wed, Feb 02, 2011 at 08:50:09PM +0100, Balz Schreier wrote:
>> you got it :-) that's what I tried to explain: read, modify and update must
>> be a single atomic operation.
> Ok, so now I understand what you're trying to achieve. What I thought
> of primarily was making sure that files don't get corrupted,

I think our primary concern is that data does not get corrupted and
also that if you have write and read at the same time, that the read
doesn't get incomplete data.

Using checkout/checkin solves the main part of the corruption problem, but
the corruption can happen while setting the lock. The chances for
this depend on how much the application is used for writing and
how modular the data model is and in order to guarantee this in
any case one could introduce a "singleton service" for locking (also see 
below re Amazon S3)

The concurrent "write/read incomplete data" problem could be mainly solved
by buffering the data, but this would still allow chances when "moving 
the buffer".

Having said the above I think it's a question of time/effort versus 
chances if the problems
from above actually happen. I would argue in many cases the chances are 
very small
(this why we are currently running stress tests to have some real data 
on this instead of
a guts feeling).

If we want to really solve this problem properly independent of the 
repository implementation below,
then I think we need to introduce something like a "synchronized 
singleton service" which can be used
by the various cluster apps to get locks.

Cheers

Michael
>   not about
> implementing transactions.
>
> Cedric