Handling Data Conflicts

Merge conflicts can occur when two or more clients write to the same record at the exact same time.

How does deepstream keep track of data consistency?

deepstream uses incrementing version numbers to make sure changes to records happen in sequence and no intermediary update gets lost. Each message created as a result of a set() call contains the version number that the client expects to set the record to.

The server will ensure that the version of an incoming update is exactly one higher than the current version. If it is, the update is applied and propagated to all other subscribed clients. If it's not though, one of two things will happen:

If the incoming version is the same as the existing version

If the version of an incoming update is the same as the current version, deepstream assumes a write conflict. It will keep the current version and send a VERSION_EXISTS error to the client that tried to update the record to the existing version. On the client, this will invoke a MERGE_STRATEGY function. More about merge strategies below.

If the incoming version is lower than or more than 1 higher than the current version

If versions have gotten out of sync, the server will attempt a reconciliation.

Handling merge conflicts

Merge conflicts are handled by MERGE_STRATEGY functions. These are exposed by the deepstream object and can be set globally when the client is initialized, on a pattern or on a per record basis.

// Set merge strategy globally when initialising the client
client = deepstream('localhost', { mergeStrategy: deepstream.MERGE_STRATEGIES.LOCAL_WINS })

// Set merge strategy on a pattern when initialising the client
client = deepstream('localhost', { mergeStrategy: deepstream.MERGE_STRATEGIES.LOCAL_WINS })
client.record.setMergeStrategyRegExp('name', (localValue, localVersion, remoteValue, remoteVersion, callback) => {
    callback(error, mergedData)
})

// Set merge strategy on a per record basis
rec = ds.record.getRecord('some-record')
rec.setMergeStrategy(deepstream.MERGE_STRATEGIES.REMOTE_WINS)

By default, LOCAL_WINS and REMOTE_WINS are available. It is also possible to implement custom merge strategies, e.g.

// Accept remote title, but keep local content
rec.setMergeStrategy(( record, remoteData, remoteVersion, callback ) => {
    callback( null, {
        title: remoteData.title,
        content: record.get( 'content' )
    });
});

Avoiding merge conflicts

The more granular you structure your records, the rarer merge conflicts will be. Generally, deepstream is better at coping with a large number of small records than with a few very large ones. Especially when it comes to records with high upstream and downstream rates, e.g. am item on an auction site with quickly updating prices, it might make sense to make the upstream (e.g. the bids a client submits) a separate record or model them as events or RPCs

Atomic multi-path updates with setMulti

A common source of self-inflicted conflicts is a compound mutation that requires several related set() calls — for example, updating two related fields, or maintaining the linked-list pointers inside a Dequeue (unshift, push, insertEntry, removeEntry each touch 2–3 paths).

With multiple back-to-back set() calls you bump the version multiple times, and each bump independently races every other writer. Two concurrent set calls a record can leave it in a half-applied state where one succeeds but the other doesn't.

record.setMulti(patches) batches all of those path updates into a single message — one version bump, one race window, all-or-nothing application. The merge-strategy story is unchanged: a conflicting setMulti update triggers the same merge-strategy callback that a single set would.

// Instead of three independent writes that can race with each other...
record.set('a', 1)
record.set('b', 2)
record.set('c', 3)

// ...batch them into one atomic update:
record.setMulti([
  { path: 'a', data: 1 },
  { path: 'b', data: 2 },
  { path: 'c', data: 3 }
])

This requires server 10.1+ and client 7.1+. On older servers the batched write is rejected with INVALID_MESSAGE_DATA; use setMultiWithAck (or pass a callback) to detect this and fall back to setEntries or sequential set calls.