So in the last major RxDB versions, the focus was set to improvements of the storage engine. This is done. RxDB has now multiple RxStorage implementations, a better query planner and an improved test suite to ensure everything works correct. This let to huge improvements in write and query performance and decreased the initial pageload of RxDB based applications.
In the new major version
13.0.0, the focus was set to improvements to the replication protocol.
When I first implemented the GraphQL replication a few years ago, I had a specific use case in mind and designed the whole protocol and replication plugins around that use case.
But the time has shown, that the old replication protocol is a big downside of RxDB:
- The replication relied on the backend to solve all conflicts. This was easy to implement into RxDB because the whole responsibility was given away to the person that has to implement a compatible backend.
- In each point in time, the replication did either push or pull documents, but never in parallel. This slows done the whole replication process and makes RxDB not usable for the implementation of features like multi-user-real-time-collaboration or when many read- and write operations have to happen in a short timespan.
- After each
pullhad to be run to check if the backend had changed the state to solve a conflict.
- The replication protocol did not support attachments and was not designed to ever support them.
So in version
13.0.0 I replaced the whole replication plugins with a new replication protocol. The main goals have been:
- Push- and Pull in parallel.
- Use the data in the changestream (optional) to decrease replication latency.
- Implement the conflict resolution into RxDB so that the client resolves its own conflicts and does not rely on the backend.
- Decrease the complexity for a compatible backend implementation. The new protocol relies on a dumb backend. This will open compatibility with many other use cases like implementing Offline-First in Supabase or using CouchDB but having a faster replication compared to the native CouchDB replication.
- Make it possible to use CRDTs instead of a conflict resolution.
- Design a the protocol in a way to make it possible to add attachments replication in the future.
On the RxDocument level, the replication works like git, where the fork/client contains all new writes and must be merged with the master/server before it can push its new state to the master/server.
A---B1---C1---X master/server state \ / B1---C2 fork/client state
For more details, read the documentation about the new replication protocol.
Backends that have been compatible with the previous RxDB versions
12 and older, will not work with the new replication protocol. To learn how to do that, either read the docs or check out the GraphQL example.
Other breaking changes
- RENAMED the
validate-ajvto be in equal with the other validation plugins.
is-my-json-validvalidation is no longer supported until this bug is fixed.
REFACTORED the schema validation plugins, they are no longer plugins but now they get wrapped around any other RxStorage.
- It allows us to configure which
RxDatabaseinstance must use the validation and which does not. In production it often makes sense to validate user data, but you might not need the validation for data that is only replicated from the backend.
REFACTORED the key compression plugin, it is no longer a plugin but now a wrapper around any other RxStorage.
REFACTORED the encryption plugin, it is no longer a plugin but now a wrapper around any other RxStorage.
Store the password hash in the same write request as the database token to improve performance.
REMOVED support for temporary documents see here
REMOVED RxDatabase.broadcastChannel The broadcast channel has been moved out of the RxDatabase and is part of the RxStorage. So it is not longer exposed via
liveIntervaloption of the replication plugins. It was an edge case feature with wrong defaults. If you want to run the pull replication on interval, you can send a
RESYNCevent manually in a loop.
RxErrorlike in the rest of the RxDB code.
- REMOVED the option to filter out replication documents with the push/pull modifiers #2552 because this does not work with the new replication protocol.
CHANGE default of replication
liveto be set to
true. Because most people want to do a live replication, not a one time replication.
serverplugin is now called
CHANGED Attachment data is now always handled as
Blobbecause Node.js does support
Blobsince version 18.0.0 so we no longer have to use a
Bufferbut instead can use Blob for browsers and Node.js
REFACTORED the layout of
RxChangeEventto better match the RxDB requirements and to fix the 'deleted-document-is-modified-but-still-deleted' bug.
When used with Node.js, RxDB now requires Node.js version
18.0.0 or higher.
Other non breaking or internal changes
- REMOVED many unused plugin hooks because they decreased the performance.
- REMOVE RxStorageStatics
- CHANGE removed default usage of
md5as default hashing. Use a faster non-cryptographic hash instead.
- ADD option to pass a custom hash function when calling
- ADD option to pass a custom hash function when calling
Intto represent timestamps in GraphQL.
FIXED multiple problems with encoding attachments data. We now use the
js-base64library which properly handles utf-8/binary/ascii transformations.
In the RxDB internal
_meta.lwtfield, we now use 2 decimals number of the unix timestamp in milliseconds.
Migration to the new version
Stored data of the previous RxDB versions is not compatible with RxDB
13.0.0. So if you want to keep that data, you have to migrate it in a way. For most use cases you might want to just drop the data from the client and re-sync it again from the backend. To keep the data locally, you might want to use the storage migration plugin.