# RxDB Documentation
> Authoritative reference documentation for RxDB, a reactive, local-first NoSQL database for JavaScript with offline support and explicit replication.
This file contains all documentation content in a single document following the llmstxt.org standard.
## RxDB Docs
import { Overview } from '@site/src/components/overview';
# RxDB Documentation
---
## Quickstart
import {Steps} from '@site/src/components/steps';
import {TriggerEvent} from '@site/src/components/trigger-event';
import {Tabs} from '@site/src/components/tabs';
import {NavbarDropdownSyncList} from '@site/src/components/navbar-dropdowns';
import { IconQuickstart } from '@site/src/components/icons/quickstart';
import { HeadlineWithIcon } from '@site/src/components/headline-with-icon';
# }>RxDB Quickstart
Welcome to the RxDB Quickstart. Here we'll learn how to create a simple real-time app with the RxDB database that is able to store and query data persistently in a browser and does realtime updates to the UI on changes.
### Installation
Install the RxDB library and the RxJS dependency:
```bash
npm install rxdb rxjs
```
### Pick a Storage
RxDB is able to run in a wide range of JavaScript runtimes like browsers, mobile apps, desktop and servers. Therefore different storage engines exist that ensure the best performance depending on where RxDB is used.
#### LocalStorage
Use this for the simplest browser setup and very small datasets. It has a tiny bundle size and works anywhere [localStorage](./articles/localstorage.md) is available, but is not optimized for large data or heavy writes.
```ts
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
let storage = getRxStorageLocalstorage();
```
#### IndexedDB π
The premium [IndexedDB storage](./rx-storage-indexeddb.md) is a high-performance, browser-native storage with a smaller bundle and faster startup compared to Dexie-based IndexedDB. Recommended when you have [π premium](/premium/) access and care about performance and bundle size.
```ts
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
let storage = getRxStorageIndexedDB();
```
#### Dexie.js
[Dexie.js](./rx-storage-dexie.md) is a friendly wrapper around IndexedDB and is a great default for browser apps when you don't use premium. It's reliable, works well for medium-sized datasets, and is free to use.
```ts
import {
getRxStorageDexie
} from 'rxdb/plugins/storage-dexie';
let storage = getRxStorageDexie();
```
#### SQLite
[SQLite](./rx-storage-sqlite.md) is ideal for React Native, Capacitor, Electron, Node.js and other hybrid or native environments. It gives you a fast, durable database on disk. Use the π premium storage for production; a trial version exists for quick experimentation.
**Premium SQLite (Node.js example)**
```ts
import {
getRxStorageSQLite,
getSQLiteBasicsNode
} from 'rxdb-premium/plugins/storage-sqlite';
// Provide the sqliteBasics adapter for your runtime, e.g. Node.js, React Native, etc.
// For example in Node.js you would derive
// sqliteBasics from a sqlite3-compatible library:
import sqlite3 from 'sqlite3';
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsNode(sqlite3)
});
```
**SQLite trial storage (Node.js, free)**
```ts
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsNodeNative
} from 'rxdb/plugins/storage-sqlite';
import { DatabaseSync } from 'node:sqlite';
const storage = getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
});
```
#### Expo Filesystem π
For React Native and Expo applications, the [Expo Filesystem storage](./rx-storage-filesystem-expo.md) offers superior performance compared to SQLite and Async Storage by utilizing OPFS JSI bindings.
```ts
import {
getRxStorageExpoAsync
} from 'rxdb-premium/plugins/storage-filesystem-expo';
let storage = getRxStorageExpoAsync();
```
#### And more...
There are many more storages such as [MongoDB](./rx-storage-mongodb.md), [DenoKV](./rx-storage-denokv.md), [Filesystem](./rx-storage-filesystem-node.md), [Memory](./rx-storage-memory.md), [Memory-Mapped](./rx-storage-memory-mapped.md), [FoundationDB](./rx-storage-foundationdb.md) and more. [Browse the full list of storages](/rx-storage.html).
Which storage should I use?
RxDB provides a wide range of storages depending on your JavaScript runtime and performance needs.
In the Browser: Use the LocalStorage storage for simple setup and small build size. For bigger datasets, use either the dexie.js storage (free) or the IndexedDB RxStorage if you have π premium access which is a bit faster and has a smaller build size.
In Electron and React Native: Use the SQLite RxStorage if you have π premium access or the SQLite Trial RxStorage for tryouts.
In Capacitor: Use the SQLite RxStorage if you have π premium access, otherwise use the LocalStorage storage.
### Dev-Mode
When you use RxDB in development, you should always enable the [dev-mode plugin](./dev-mode.md), which adds helpful checks and validations, and tells you if you do something wrong.
```ts
import { addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
addRxPlugin(RxDBDevModePlugin);
```
### Schema Validation
[Schema validation](./schema-validation.md) is required when using dev-mode and recommended (but optional) in production. Wrap your storage with the AJV schema validator to ensure all documents match your schema before being saved.
```ts
import { wrappedValidateAjvStorage } from 'rxdb/plugins/validate-ajv';
storage = wrappedValidateAjvStorage({ storage });
```
### Create a Database
A database is the topβlevel container in RxDB, responsible for managing collections, coordinating persistence, and providing reactive change streams.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: storage
});
```
### Add a Collection
An RxDatabase contains [RxCollection](./rx-collection.md)s for storing and querying data. A collection is similar to an SQL table, and individual records are stored in the collection as JSON documents. An [RxDatabase](./rx-database.md) can have as many collections as you need.
Add a collection with a [schema](./rx-schema.md) to the database:
```ts
await myDatabase.addCollections({
// name of the collection
todos: {
// we use the JSON-schema standard
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have maxLength
},
name: {
type: 'string'
},
done: {
type: 'boolean'
},
timestamp: {
type: 'string',
format: 'date-time'
}
},
required: ['id', 'name', 'done', 'timestamp']
}
}
});
```
### Insert a Document
Now that we have an RxCollection we can store some [documents](./rx-document.md) in it.
```ts
const myDocument = await myDatabase.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
done: false,
timestamp: new Date().toISOString()
});
```
### Run a Query
Execute a [query](./rx-query.md) that returns all found documents once:
```ts
const foundDocuments = await myDatabase.todos.find({
selector: {
done: {
$eq: false
}
}
}).exec();
```
### Update a Document
In the first found document, set `done` to `true`:
```ts
const firstDocument = foundDocuments[0];
await firstDocument.patch({
done: true
});
```
### Delete a Document
Delete the document so that it can no longer be found in queries:
```ts
await firstDocument.remove();
```
### Observe a Query
Subscribe to data changes so that your UI is always up-to-date with the data stored on disk. RxDB allows you to subscribe to data changes even when the change happens in another part of your application, another browser tab, or during database [replication/synchronization](./replication.md):
```ts
const observable = myDatabase.todos.find({
selector: {
done: {
$eq: false
}
}
}).$ // get the observable via RxQuery.$;
observable.subscribe(notDoneDocs => {
console.log('Currently have ' + notDoneDocs.length + ' things to do');
// -> here you would re-render your app to show the updated document list
});
```
### Observe a Document Value
You can also subscribe to the fields of a single RxDocument. Add the `$` sign to the desired field and then subscribe to the returned observable.
```ts
myDocument.done$.subscribe(isDone => {
console.log('done: ' + isDone);
});
```
### Sync the Client
RxDB has multiple [replication plugins](./replication.md) to replicate database state with a server.
#### HTTP
```ts
import {
replicateHTTP,
pullQueryBuilderFromRxSchema,
} from "rxdb/plugins/replication-http";
replicateHTTP({
collection: db.todos,
push: {
handler: async (rows) => {
return fetch("https://example.com/api/todos/push", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(rows),
}).then((res) => res.json());
},
},
pull: {
handler: async (lastCheckpoint) => {
return fetch(
"https://example.com/api/todos/pull?" +
new URLSearchParams({
checkpoint: JSON.stringify(lastCheckpoint)
}),
).then((res) => res.json());
},
},
});
```
#### GraphQL
```ts
import { replicateGraphQL } from 'rxdb/plugins/replication-graphql';
replicateGraphQL({
collection: db.todos,
url: 'https://example.com/graphql',
push: { batchSize: 50 },
pull: { batchSize: 50 }
});
```
#### WebRTC (P2P)
The easiest way to replicate data between your clients' devices is the [WebRTC replication plugin](./replication-webrtc.md) that replicates data between devices without a centralized server. This makes it easy to try out replication without having to host anything:
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
replicateWebRTC({
collection: myDatabase.todos,
connectionHandlerCreator: getConnectionHandlerSimplePeer({}),
topic: '', // <- set any app-specific room id here.
secret: 'mysecret',
pull: {},
push: {}
})
```
#### CouchDB
```ts
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
replicateCouchDB({
collection: db.todos,
url: 'http://example.com/todos/',
push: {},
pull: {}
});
```
#### And more...
Explore all [replication plugins](/replication.html), including advanced conflict handling and custom protocols.
## Next steps
You are now ready to dive deeper into RxDB.
- Start reading the full documentation [here](./install.md).
- There is a full implementation of the [quickstart guide](https://github.com/pubkey/rxdb-quickstart) so you can clone that repository and play with the code.
- For frameworks and runtimes like Angular, React Native and others, check out the list of [example implementations](https://github.com/pubkey/rxdb/tree/master/examples).
- Also please continue reading the documentation, join the community on our [Discord chat](/chat/), and star the [GitHub repo](https://github.com/pubkey/rxdb).
- If you are using RxDB in a production environment and are able to support its continued development, please take a look at the [π Premium package](/premium/) which includes additional plugins and utilities.
---
## Installation
import {InstallTabs} from '@site/src/components/install-tabs';
# Install RxDB
## npm
To install the latest release of `rxdb` and its dependencies and save it to your `package.json`, run:
## peer-dependency
You also need to install the peer-dependency `rxjs` if you have not installed it before.
## polyfills
RxDB is coded with ES8 and transpiled to ES5. This means you have to install [polyfills](https://developer.mozilla.org/en-US/docs/Glossary/Polyfill) to support older browsers. For example you can use [core-js](https://github.com/zloirock/core-js) with:
```bash
npm i core-js --save
```
If you need polyfills, you have to import them in your code.
```typescript
import 'core-js/stable';
```
## Polyfill the `global` variable
When you use RxDB with [Angular](./articles/angular-database.md) or other **Webpack** based frameworks, you might get the error `Uncaught ReferenceError: global is not defined`.
This is because some dependencies of RxDB assume a Node.js-specific `global` variable that is not added to browser runtimes by some bundlers.
You have to add them manually, like we do [here](https://github.com/pubkey/rxdb/blob/master/examples/angular/src/polyfills.ts).
```ts
(window as any).global = window;
(window as any).process = {
env: { DEBUG: undefined },
};
```
## Project Setup and Configuration
In the [examples](https://github.com/pubkey/rxdb/tree/master/examples) folder you can find CI tested projects for different frameworks and use cases, while in the [/config](https://github.com/pubkey/rxdb/tree/master/config) folder base configuration files for Webpack, Rollup, Mocha, Karma, TypeScript are exposed.
Consult [package.json](https://github.com/pubkey/rxdb/blob/master/package.json) for the versions of the packages supported.
## Installing the latest RxDB build
If you need the latest development state of RxDB, add it as git dependency into your `package.json`.
```json
"dependencies": {
"rxdb": "git+https://git@github.com/pubkey/rxdb.git#commitHash"
}
```
Replace `commitHash` with the hash of the latest [build-commit](https://github.com/pubkey/rxdb/search?q=build&type=Commits).
## Import
To import `rxdb`, add this to your JavaScript file to import the default bundle that contains the RxDB core:
```typescript
import {
createRxDatabase, // ./rx-database.md
/* ... */
} from 'rxdb';
```
---
## Development Mode
import {Steps} from '@site/src/components/steps';
# Dev Mode
The dev-mode plugin adds many checks and validations to RxDB.
This ensures that you use the RxDB API properly and so the dev-mode plugin should always be used when
using RxDB in development mode.
- Adds readable error messages.
- Ensures that `readonly` JavaScript objects are not accidentally mutated.
- Adds validation check for validity of schemas, queries, [ORM](./orm.md) methods and document fields.
- Notice that the `dev-mode` plugin does not perform schema checks against the data see [schema validation](./schema-validation.md) for that.
:::warning
The dev-mode plugin will increase your build size and decrease the performance. It must **always** be used in development. You should **never** use it in production.
:::
### Import the dev-mode Plugin
```javascript
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { addRxPlugin } from 'rxdb/plugins/core';
```
## Add the Plugin to RxDB
```javascript
addRxPlugin(RxDBDevModePlugin);
```
## Usage with Node.js
```ts
async function createDb() {
if (process.env.NODE_ENV !== "production") {
await import('rxdb/plugins/dev-mode').then(
module => addRxPlugin(module.RxDBDevModePlugin)
);
}
const db = await createRxDatabase( /* ... */ );
}
```
## Usage with [Angular](./articles/angular-database.md)
```ts
import { isDevMode } from '@angular/core';
async function createDb() {
if (isDevMode()){
await import('rxdb/plugins/dev-mode').then(
module => addRxPlugin(module.RxDBDevModePlugin)
);
}
const db = await createRxDatabase( /* ... */ );
// ...
}
```
## Usage with webpack
In the `webpack.config.js`:
```ts
module.exports = {
entry: './src/index.ts',
/* ... */
plugins: [
// set a global variable that can be accessed during runtime
new webpack.DefinePlugin({ MODE: JSON.stringify("production") })
]
/* ... */
};
```
In your source code:
```ts
declare var MODE: 'production' | 'development';
async function createDb() {
if (MODE === 'development') {
await import('rxdb/plugins/dev-mode').then(
module => addRxPlugin(module.RxDBDevModePlugin)
);
}
const db = await createRxDatabase( /* ... */ );
// ...
}
```
## Disable the dev-mode warning
When the dev-mode is enabled, it will print a `console.warn()` message to the console so that you do not accidentally use the dev-mode in production. To disable this warning you can call the `disableWarnings()` function.
```ts
import { disableWarnings } from 'rxdb/plugins/dev-mode';
disableWarnings();
```
## Disable the tracking iframe
When used in localhost and in the browser, the dev-mode plugin can add a tracking iframe to the DOM. This is used to track the effectiveness of marketing efforts of RxDB.
If you have [premium access](/premium/) and want to disable this iframe, you can call `setPremiumFlag()` before creating the database.
```js
import { setPremiumFlag } from 'rxdb-premium/plugins/shared';
setPremiumFlag();
```
---
## TypeScript Setup
import {Steps} from '@site/src/components/steps';
# Using RxDB with TypeScript
In this tutorial you will learn how to use RxDB with TypeScript.
We will create a basic database with one collection and several [ORM](../orm.md)-methods, fully typed!
RxDB directly comes with its typings and you do not have to install anything else, however the latest version of RxDB requires that you are using Typescript v3.8 or newer.
Our way to go is
- First define what the documents look like
- Then define what the collections look like
- Then define what the database looks like
## Declare the types
First you import the types from RxDB.
```typescript
import {
createRxDatabase,
RxDatabase,
RxCollection,
RxJsonSchema,
RxDocument,
} from 'rxdb/plugins/core';
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
```
## Create the base document type
First we have to define the TypeScript type of the documents of a collection:
**Option A**: Create the document type from the schema
```typescript
import {
toTypedRxJsonSchema,
ExtractDocumentTypeFromTypedRxJsonSchema,
RxJsonSchema
} from 'rxdb';
export const heroSchemaLiteral = {
title: 'hero schema',
description: 'describes a human being',
version: 0,
keyCompression: true,
primaryKey: 'passportId',
type: 'object',
properties: {
passportId: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
firstName: {
type: 'string'
},
lastName: {
type: 'string'
},
age: {
type: 'integer'
}
},
required: ['firstName', 'lastName', 'passportId'],
indexes: ['firstName']
} as const; // <- It is important to set 'as const' to preserve the literal type
const schemaTyped = toTypedRxJsonSchema(heroSchemaLiteral);
// aggregate the document type from the schema
export type HeroDocType =
ExtractDocumentTypeFromTypedRxJsonSchema<
typeof schemaTyped
>;
// create the typed RxJsonSchema from the literal typed object.
export const heroSchema: RxJsonSchema = heroSchemaLiteral;
```
**Option B**: Manually type the document type
```typescript
export type HeroDocType = {
passportId: string;
firstName: string;
lastName: string;
age?: number; // optional
};
```
**Option C**: Generate the document type from schema during build time
If your schema is in a `.json` file or generated from somewhere else, you might generate the typings with the [json-schema-to-typescript](https://www.npmjs.com/package/json-schema-to-typescript) module.
## Types for the ORM methods
We also add some ORM-methods for the document.
```typescript
export type HeroDocMethods = {
scream: (v: string) => string;
};
```
## Create [RxDocument](../rx-document.md) Type
We can merge these into our HeroDocument.
```typescript
export type HeroDocument = RxDocument;
```
## Create [RxCollection](../rx-collection.md) Type
Now we can define type for the collection which contains the documents.
```typescript
// we declare one static ORM-method for the collection
export type HeroCollectionMethods = {
countAllDocuments: () => Promise;
}
// and then merge all our types
export type HeroCollection = RxCollection<
HeroDocType,
HeroDocMethods,
HeroCollectionMethods
>;
```
## Create [RxDatabase](../rx-database.md) Type
Before we can define the database, we make a helper-type which contains all collections of it.
```typescript
export type MyDatabaseCollections = {
heroes: HeroCollection
}
```
Now the database.
```typescript
export type MyDatabase = RxDatabase;
```
## Using the types
Now that we have declare all our types, we can use them.
```typescript
/**
* create database and collections
*/
const myDatabase: MyDatabase = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage()
});
const heroSchema: RxJsonSchema = {
title: 'human schema',
description: 'describes a human being',
version: 0,
keyCompression: true,
primaryKey: 'passportId',
type: 'object',
properties: {
passportId: {
type: 'string',
maxLength: 100
},
firstName: {
type: 'string'
},
lastName: {
type: 'string'
},
age: {
type: 'integer'
}
},
required: ['passportId', 'firstName', 'lastName']
};
const heroDocMethods: HeroDocMethods = {
scream: function(this: HeroDocument, what: string) {
return this.firstName + ' screams: ' + what.toUpperCase();
}
};
const heroCollectionMethods: HeroCollectionMethods = {
countAllDocuments: async function(this: HeroCollection) {
const allDocs = await this.find().exec();
return allDocs.length;
}
};
await myDatabase.addCollections({
heroes: {
schema: heroSchema,
methods: heroDocMethods,
statics: heroCollectionMethods
}
});
// add a postInsert-hook
myDatabase.heroes.postInsert(
function myPostInsertHook(
this: HeroCollection, // own collection is bound to the scope
docData: HeroDocType, // documents data
doc: HeroDocument // RxDocument
) {
console.log('insert to ' + this.name + '-collection: ' + doc.firstName);
},
false // not async
);
/**
* use the database
*/
// insert a document
const hero: HeroDocument = await myDatabase.heroes.insert({
passportId: 'myId',
firstName: 'piotr',
lastName: 'potter',
age: 5
});
// access a property
console.log(hero.firstName);
// use a orm method
hero.scream('AAH!');
// use a static orm method from the collection
const amount: number = await myDatabase.heroes.countAllDocuments();
console.log(amount);
/**
* clean up
*/
myDatabase.close();
```
---
## RxDatabase - The Core of Your Realtime Data
# RxDatabase
An RxDatabase Object contains your [collections](./rx-collection.md) and handles the synchronization of change events.
## Creation
The database is created by the asynchronous `.createRxDatabase()` function of the core RxDB module. It has the following parameters:
```javascript
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'heroesdb', // <- name
storage: getRxStorageLocalstorage(), // <- RxStorage
/* Optional parameters: */
password: 'myPassword', // <- password (optional)
multiInstance: true, // <- multiInstance (optional, default: true)
eventReduce: true, // <- eventReduce (optional, default: false)
cleanupPolicy: {} // <- custom cleanup policy (optional)
});
```
### name
The database name is a string which uniquely identifies the database. When two RxDatabases have the same name and use the same `RxStorage`, their data can be assumed as equal and they will share events between each other.
Depending on the storage or adapter this can also be used to define the filesystem folder of your data.
### storage
RxDB works on top of an implementation of the [RxStorage](./rx-storage.md) interface. This interface is an abstraction that allows you to use different underlying databases that actually handle the documents. Depending on your use case you might use a different `storage` with different tradeoffs in performance, bundle size or supported runtimes.
There are many `RxStorage` implementations that can be used depending on the JavaScript environment and performance requirements.
For example you can use the [LocalStorage RxStorage](./rx-storage-localstorage.md) in the browser or use the [MongoDB RxStorage](./rx-storage-mongodb.md) in Node.js.
- [List of RxStorage implementations](./rx-storage.md)
```javascript
// use the LocalStorage that stores data in the browser.
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageLocalstorage()
});
// ...or use the MongoDB RxStorage in Node.js.
import { getRxStorageMongoDB } from 'rxdb/plugins/storage-mongodb';
const dbMongo = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageMongoDB({
connection: 'mongodb://localhost:27017,localhost:27018,localhost:27019'
})
});
```
### password
`(optional)`
If you want to use encrypted fields in the collections of a database, you have to set a password for it. The password must be a string with at least 12 characters.
[Read more about encryption here](./encryption.md).
### multiInstance
`(optional=true)`
When you create more than one instance of the same database in a single javascript-runtime, you should set `multiInstance` to ```true```. This will enable the event sharing between the two instances. For example when the user has opened multiple browser windows, events will be shared between them so that both windows react to the same changes.
`multiInstance` should be set to `false` when you have single instances like a single Node.js process, a React Native app, a Cordova app or a single-window [Electron](./electron-database.md) app which can decrease the startup time because no instance coordination has to be done.
### eventReduce
`(optional=false)`
One big benefit of having a realtime database is that big performance optimizations can be done when the database knows a query is observed and the updated results are needed continuously. RxDB uses the [EventReduce Algorithm](https://github.com/pubkey/event-reduce) to optimize observer or recurring queries.
For better performance, you should always set `eventReduce: true`. This will also be the default in the next major RxDB version.
### ignoreDuplicate
`(optional=false)`
If you create multiple RxDatabase-instances with the same name and same adapter, it's very likely that you have done something wrong.
To prevent this common mistake, RxDB will throw an error when you do this.
In some rare cases like unit-tests, you want to do this intentionally by setting `ignoreDuplicate` to `true`. Because setting `ignoreDuplicate: true` in production will decrease the performance by having multiple instances of the same database, `ignoreDuplicate` is only allowed to be set in [dev-mode](./dev-mode.md).
```js
const db1 = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
ignoreDuplicate: true
});
const db2 = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
// this create-call will not throw because
// you explicitly allow it
ignoreDuplicate: true
});
```
### closeDuplicates
`(optional=false)`
Closes all other RxDatabase instances that have the same storage+name combination.
```js
const db1 = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
closeDuplicates: true
});
const db2 = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
closeDuplicates: true // this create-call will close db1
});
// db1 is now closed.
```
### hashFunction
By default, RxDB will use `crypto.subtle.digest('SHA-256', data)` for hashing. If you need a different hash function or the `crypto.subtle` API is not supported in your JavaScript runtime, you can provide your own hash function instead. A hash function gets a `string`, `ArrayBuffer`, or `Blob` as input and returns a `Promise` that resolves a string. When a `Blob` is received (for attachment digest hashing), convert it to a string or ArrayBuffer before hashing.
```ts
// example hash function that runs in plain JavaScript
import { sha256 } from 'ohash';
import { blobToBase64String } from 'rxdb';
async function myOwnHashFunction(input: string | ArrayBuffer | Blob) {
if (typeof Blob !== 'undefined' && input instanceof Blob) {
input = await blobToBase64String(input);
} else if (input instanceof ArrayBuffer) {
input = new TextDecoder().decode(new Uint8Array(input));
}
return sha256(input);
}
const db = await createRxDatabase({
hashFunction: myOwnHashFunction
/* ... */
});
```
If you get the error message `TypeError: Cannot read properties of undefined (reading 'digest')` this likely means that you are neither running on `localhost` nor on `https` which is why your browser might not allow access to `crypto.subtle.digest`.
### liveQueryUpdateThrottleTime
`(optional, default: 0 = disabled)`
Groups write-triggered live query updates to limit how often RxDB re-evaluates queries during write bursts. See [liveQueryUpdateThrottleTime](./rx-query.md#livequeryupdatethrottletime) for full documentation.
## Methods
### Observe with $
Calling this will return an [RxJS Observable](http://reactivex.io/documentation/observable.html) which streams all write events of the `RxDatabase`.
```javascript
myDb.$.subscribe(changeEvent => console.dir(changeEvent));
```
### exportJSON()
Use this function to create a JSON export from every piece of data in every collection of this database. You can pass `true` as a parameter to decrypt the encrypted data fields of your document.
Before `exportJSON()` and `importJSON()` can be used, you have to add the `json-dump` plugin.
```javascript
import { addRxPlugin } from 'rxdb';
import { RxDBJsonDumpPlugin } from 'rxdb/plugins/json-dump';
addRxPlugin(RxDBJsonDumpPlugin);
```
```javascript
myDatabase.exportJSON()
.then(json => console.dir(json));
```
### importJSON()
To import the JSON dumps into your database, use this function.
```javascript
// import the dump to the database
emptyDatabase.importJSON(json)
.then(() => console.log('done'));
```
### backup()
Writes the current (or ongoing) database state to the filesystem. [Read more](./backup.md)
### waitForLeadership()
Returns a Promise which resolves when the RxDatabase becomes [elected leader](./leader-election.md).
### requestIdlePromise()
Returns a promise which resolves when the database is in idle. This works similar to [requestIdleCallback](https://developer.mozilla.org/de/docs/Web/API/Window/requestIdleCallback) but tracks the idleness of the database instead of the CPU.
Use this for semi-important tasks like cleanups which should not affect the speed of important tasks.
```javascript
myDatabase.requestIdlePromise().then(() => {
// this will run at the moment the database has nothing else to do
myCollection.customCleanupFunction();
});
// with timeout
myDatabase.requestIdlePromise(1000 /* time in ms */).then(() => {
// this will run at the moment the database has nothing else to do
// or the timeout has passed
myCollection.customCleanupFunction();
});
```
### close()
Closes the database's object instance. This is to free up memory and stop all observers and replications.
Returns a `Promise` that resolves when the database is closed.
Closing a database will not remove the database's data. When you create the database again with `createRxDatabase()`, all data will still be there.
```javascript
await myDatabase.close();
```
### remove()
Wipes all documents from the storage. Use this to free up disk space.
```javascript
await myDatabase.remove();
// database instance is now gone
```
You can also clear a database without removing its instance by using `removeRxDatabase()`. This is useful if you want to migrate data or reset the user's state by renaming the database. Then you can remove the previous data with `removeRxDatabase()` without creating a RxDatabase first. Notice that this will only remove the
stored data on the storage. It will not clear the cache of any [RxDatabase](./rx-database.md) instances.
```javascript
import { removeRxDatabase } from 'rxdb';
removeRxDatabase('mydatabasename', 'localstorage');
```
### isRxDatabase
Returns true if the given object is an instance of RxDatabase. Returns false if not.
```javascript
import { isRxDatabase } from 'rxdb';
const is = isRxDatabase(myObj);
```
### collections$
Emits events whenever an [RxCollection](./rx-collection.md) is added or removed to the instance of the RxDatabase. Notice that this only emits the JavaScript instance of the RxCollection class, it does not emit events across browser tabs.
```javascript
const sub = myDatabase.collections$.subscribe(event => {
console.dir(event);
});
await myDatabase.addCollections({
heroes: {
schema: mySchema
}
});
// -> emits the event
sub.unsubscribe();
```
---
## Design Perfect Schemas in RxDB
import {Faq, FaqItem} from '@site/src/components/faq';
# RxSchema
Schemas define the structure of the documents of a collection. Which field should be used as the primary key, which fields should be used as indexes, and what should be encrypted. Every collection has its own schema. With RxDB, schemas are defined with the [JSON Schema](https://json-schema.org/blog/posts/rxdb-case-study) standard which you might know from other projects.
## Example
In this example-schema we define a hero-collection with the following settings:
- the version-number of the schema is 0
- the name-property is the **primaryKey**. This means it's a unique, indexed, required `string` which can be used to definitely find a single document.
- the color-field is required for every document
- the healthpoints-field must be a number between 0 and 100
- the secret-field stores an encrypted value
- the birthyear-field is final which means it is required and cannot be changed
- the skills-attribute must be an array of objects which contain the name and the damage-attribute. There is a maximum of 5 skills per hero.
- Allows adding attachments and storing them encrypted
```json
{
"title": "hero schema",
"version": 0,
"description": "describes a simple hero",
"primaryKey": "name",
"type": "object",
"properties": {
"name": {
"type": "string",
"maxLength": 100 // <- the primary key must have set maxLength
},
"color": {
"type": "string"
},
"healthpoints": {
"type": "number",
"minimum": 0,
"maximum": 100
},
"secret": {
"type": "string"
},
"birthyear": {
"type": "number",
"final": true,
"minimum": 1900,
"maximum": 2050
},
"skills": {
"type": "array",
"maxItems": 5,
"uniqueItems": true,
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"damage": {
"type": "number"
}
}
}
}
},
"required": [
"name",
"color"
],
"encrypted": ["secret"],
"attachments": {
"encrypted": true
}
}
```
## Create a collection with the schema
```javascript
await myDatabase.addCollections({
heroes: {
schema: myHeroSchema
}
});
console.dir(myDatabase.heroes.name);
// heroes
```
## version
The `version` field is a number, starting with `0`.
When the version is greater than 0, you have to provide the `migrationStrategies` to create a collection with this schema.
## primaryKey
The `primaryKey` field contains the fieldname of the property that will be used as primary key for the whole collection.
The value of the primary key of the document must be a `string`, unique, final and required.
### composite primary key
You can define a composite primary key which is composed from multiple properties of the document data.
```javascript
const mySchema = {
keyCompression: true, // set this to true, to enable the keyCompression
version: 0,
title: 'human schema with composite primary',
primaryKey: {
// where should the composed string be stored
key: 'id',
// fields that will be used to create the composed key
fields: [
'firstName',
'lastName'
],
// separator which is used to concat the fields values.
separator: '|'
},
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
firstName: {
type: 'string'
},
lastName: {
type: 'string'
}
},
required: [
'id',
'firstName',
'lastName'
]
};
```
You can then find a document by using the relevant parts to create the composite primaryKey:
```ts
// inserting with composite primary
await myRxCollection.insert({
// id, <- do not set the id, it will be filled by RxDB
firstName: 'foo',
lastName: 'bar'
});
// find by composite primary
const id = myRxCollection.schema.getPrimaryOfDocumentData({
firstName: 'foo',
lastName: 'bar'
});
const myRxDocument = await myRxCollection.findOne(id).exec();
```
## Indexes
RxDB supports secondary indexes which are defined at the schema-level of the collection.
Indexes are only allowed on field types `string`, `integer` and `number`. Some RxStorages allow to use `boolean` fields as index.
Depending on the field type, you must have set some meta attributes like `maxLength` or `minimum`. This is required so that RxDB
is able to know the maximum string representation length of a field, which is needed to craft custom indexes in several `RxStorage` implementations.
**Performance Note:** Having a large `maxLength` for indexed fields and primary keys can negatively impact performance and storage size on many storages. Therefore, you should only set it as large as strictly needed for your application.
:::note
RxDB will always append the `primaryKey` to all indexes to ensure a deterministic sort order of query results. You do not have to add the `primaryKey` to any index.
:::
### Index-example
```javascript
const schemaWithIndexes = {
version: 0,
title: 'human schema with indexes',
keyCompression: true,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
firstName: {
type: 'string',
// string-fields used as an index,
// must have set maxLength.
maxLength: 100
},
lastName: {
type: 'string'
},
active: {
type: 'boolean'
},
familyName: {
type: 'string'
},
balance: {
type: 'number',
// number fields used in an index, must set
// minimum, maximum and multipleOf
minimum: 0,
maximum: 100000,
multipleOf: 0.01
},
creditCards: {
type: 'array',
items: {
type: 'object',
properties: {
cvc: {
type: 'number'
}
}
}
}
},
required: [
'id',
'active' // <- boolean fields that are used in an index must be required.
],
indexes: [
'firstName', // <- this will create a simple index for the `firstName` field
// <- compound-index for these two fields
['active', 'firstName'],
'active'
]
};
```
# internalIndexes
When you use RxDB on the server-side, you might want to use internalIndexes to speed up internal queries. [Read more](./rx-server.md#server-only-indexes)
## attachments
To use attachments in the collection, you have to add the `attachments`-attribute to the schema. [See RxAttachment](./rx-attachment.md).
## default
Default values can only be defined for first-level fields.
Whenever you insert a document unset fields will be filled with default-values.
```javascript
const schemaWithDefaultAge = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
firstName: {
type: 'string'
},
lastName: {
type: 'string'
},
age: {
type: 'integer',
default: 20 // <- default will be used
}
},
required: ['id']
};
```
## final
By setting a field to `final`, you make sure it cannot be modified later. Final fields are always required.
Final fields cannot be observed because they will not change.
Advantages:
- With final fields you can ensure that no-one accidentally modifies the data.
- When you enable the `eventReduce` algorithm, some performance-improvements are done.
```javascript
const schemaWithFinalAge = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
firstName: {
type: 'string'
},
lastName: {
type: 'string'
},
age: {
type: 'integer',
final: true
}
},
required: ['id']
};
```
## Non allowed properties
The schema is not only used to validate objects before they are written into the database, but also used to map getters to observe and populate single fieldnames, key compression and other things. Therefore you can not use every schema which would be valid for the spec of [json-schema.org](http://json-schema.org/).
For example, fieldnames must match the regex `^[a-zA-Z](?:[[a-zA-Z0-9_]*]?[a-zA-Z0-9])?$` and `additionalProperties` is always set to `false`. But don't worry, RxDB will instantly throw an error when you pass an invalid schema into it.
Also the following class properties of `RxDocument` cannot be used as top level fields because they would clash when the RxDocument property is accessed:
```json
[
"collection",
"_data",
"_propertyCache",
"isInstanceOfRxDocument",
"primaryPath",
"primary",
"revision",
"deleted$",
"deleted$$",
"deleted",
"getLatest",
"$",
"$$",
"get$",
"get$$",
"populate",
"get",
"toJSON",
"toMutableJSON",
"update",
"incrementalUpdate",
"updateCRDT",
"putAttachment",
"putAttachmentBase64",
"getAttachment",
"allAttachments",
"allAttachments$",
"modify",
"incrementalModify",
"patch",
"incrementalPatch",
"_saveData",
"remove",
"incrementalRemove",
"close",
"deleted",
"synced"
]
```
## FAQ
With RxDB you can only store plain JSON data inside of a document. You cannot store a JavaScript `new Date()` instance directly. This is for performance reasons and because `Date` is a mutable object where changing it at any time might cause strange problems that are hard to debug.
To store a date in RxDB, you have to define a string field with a `format` attribute:
```json
{
"type": "string",
"format": "date-time"
}
```
When storing the data you have to first transform your `Date` object into a string `Date.toISOString()`.
Because the `date-time` is sortable, you can do whatever query operations on that field and even use it as an index.
In JSON Schema, you make a field nullable by allowing multiple types with an array:
```json
{
"type": ["string", "null"]
}
```
When you use a nullable type like `["string", "null"]`, you should always add that field to the `required` array. If a nullable field is not required, it can end up in three possible states: a string value, `null`, or `undefined` (not set). Having three states instead of two makes your code harder to reason about.
In RxDB it is recommended to **not** store `null` values at all. Instead, define the field as non-required and leave it `undefined` (not set) when there is no value. A field that is not listed in the `required` array can be omitted from a document. This approach works better with RxDB's internal handling and keeps your data cleaner:
```ts
{
"version": 0,
"primaryKey": "id",
"type": "object",
"properties": {
"id": {
"type": "string",
"maxLength": 100
},
"nickname": {
"type": "string"
}
},
"required": ["id"]
// "nickname" is not required, so it can be left undefined (not set)
}
```
By design, RxDB requires that every collection has a schema. This means you cannot create a truly "schema-less" collection where top-level fields are unknown at schema creation time. RxDB must know about all fields of a document at the top level to perform validation, index creation, and other internal optimizations.
However, there is a way to store data of arbitrary structure at sub-fields. To do this, define a property with `type: "object"` in your schema. For example:
```ts
{
"version": 0,
"primaryKey": "id",
"type": "object",
"properties": {
"id": {
"type": "string",
"maxLength": 100
},
"myDynamicData": {
"type": "object"
// Here you can store any JSON data
// because it's an open object.
}
},
"required": ["id"]
}
```
RxDB automatically sets `additionalProperties: false` at the top level of a schema to ensure that all top-level fields are known in advance. This design choice offers several benefits:
- Prevents collisions with [RxDocument](./rx-document.md) class properties:
RxDB documents have built-in class methods (e.g., .toJSON, .save) at the top level. By forbidding unknown top-level properties, we avoid accidental naming collisions with these built-in methods.
- Avoids conflicts with user-defined ORM functions:
Developers can add custom [ORM methods](./orm.md) to RxDocuments. If top-level properties were unbounded, a property name could accidentally conflict with a method name, leading to unexpected behavior.
- Improves TypeScript typings:
If RxDB didn't know about all top-level fields, the document type would effectively become `any`. That means a simple typo like `myDocument.toJOSN()` would only be caught at runtime, not at build time. By disallowing unknown properties, TypeScript can provide strict typing and catch errors sooner.
When you make changes to the schema of a collection, you sometimes can get an error like
`Error: addCollections(): another instance created this collection with a different schema`.
This means you have created a collection before and added document-data to it.
When you now just change the schema, it is likely that the new schema does not match the saved documents inside of the collection.
This would cause strange bugs and would be hard to debug, so RxDB checks if your schema has changed and throws an error.
To change the schema in **production**-mode, do the following steps:
- Increase the `version` by 1
- Add the appropriate [migrationStrategies](https://pubkey.github.io/rxdb/migration-schema.html) so the saved data will be modified to match the new schema
Why does the top-level schema complain about a missing `_id` primary key field?
You encounter an error stating that the top-level schema is missing the `_id` primary key field during [replication](./replication.md). RxDB requires every schema to explicitly define the primary key property. Other databases use an implicit `_id` field. You must add the `_id` property to your schema manually if your backend expects it. You declare `_id` as a string type and set it as the `primaryKey` in your schema definition.
In **development**-mode, the schema-change can be simplified by **one of these** strategies:
- Use the memory-storage so your db resets on restart and your schema is not saved permanently
- Call `removeRxDatabase('mydatabasename', RxStorage);` before creating a new [RxDatabase](./rx-database.md)-instance
- Add a timestamp as suffix to the database-name to create a new one each run like `name: 'heroesDB' + new Date().getTime()`
---
## Master Data - Create and Manage RxCollections
import { NON_PREMIUM_COLLECTION_LIMIT } from '../src/constants';
import {Faq, FaqItem} from '@site/src/components/faq';
# RxCollection
A collection stores documents of the same type.
## Creating a Collection
To create one or more collections you need an [RxDatabase](./rx-database.md) object which has the `.addCollections()` method. Every collection needs a collection name and a valid [RxJsonSchema](./rx-schema.md). Other attributes are optional.
```js
const myCollections = await myDatabase.addCollections({
// key = collectionName
humans: {
schema: mySchema,
statics: {}, // (optional) ORM-functions
// for this collection
methods: {}, // (optional) ORM-functions for documents
attachments: {}, // (optional) ORM-functions for attachments
options: {}, // (optional) Custom parameters
// that might be used in plugins
migrationStrategies: {}, // (optional)
autoMigrate: true, // (optional) [default=true]
cacheReplacementPolicy: function(){}, // (optional) custom
// cache replacement policy
conflictHandler: function(){} // (optional) custom
// conflict handler
},
// you can create multiple collections at once
animals: {
// ...
}
});
```
:::note
Without the Premium Plugins, RxDB allows up to {NON_PREMIUM_COLLECTION_LIMIT} open collections in parallel. If you hit that limit, see the [FAQ on how to remove it](/rx-collection.html#faq).
:::
### name
The name uniquely identifies the collection and should be used to refine the collection in the database. Two different collections in the same database can never have the same name. Collection names must match the following regex: `^[a-z][a-z0-9]*$`.
### schema
The schema defines how the documents of the collection are structured. RxDB uses a schema format, similar to [JSON schema](https://json-schema.org/). Read more about the RxDB schema format [here](./rx-schema.md).
### ORM-functions
With the parameters `statics`, `methods` and `attachments`, you can define ORM functions that are applied to each of these objects that belong to this collection. See [ORM/DRM](./orm.md).
### liveQueryUpdateThrottleTime
Overrides `liveQueryUpdateThrottleTime` set at the database level for this specific collection. See [liveQueryUpdateThrottleTime](./rx-query.md#livequeryupdatethrottletime).
### Migration
With the parameters `migrationStrategies` and `autoMigrate` you can specify how migration between different schema-versions should be done. [See Migration](./migration-schema.md).
## Get a collection from the database
To get an existing collection from the database, call the collection name directly on the database:
```javascript
// newly created collection
const collections = await db.addCollections({
heroes: {
schema: mySchema
}
});
const collection2 = db.heroes;
console.log(collections.heroes === collection2); //> true
```
## Functions
### Observe $
Calling this will return an [rxjs-Observable](https://rxjs.dev/guide/observable) which streams every change to data of this collection.
```js
myCollection.$.subscribe(changeEvent => console.dir(changeEvent));
// you can also observe single event-types with insert$ update$ remove$
myCollection.insert$.subscribe(changeEvent => console.dir(changeEvent));
myCollection.update$.subscribe(changeEvent => console.dir(changeEvent));
myCollection.remove$.subscribe(changeEvent => console.dir(changeEvent));
```
### insert()
Use this to insert new documents into the database. The collection will validate the schema and automatically encrypt any encrypted fields. Returns the new RxDocument.
```js
const doc = await myCollection.insert({
name: 'foo',
lastname: 'bar'
});
```
### insertIfNotExists()
The insertIfNotExists() method attempts to insert a new document into the collection only if a document with the same primary key does not already exist. This is useful for ensuring uniqueness without having to manually check for existing records before inserting or handling [conflicts](./transactions-conflicts-revisions.md).
Returns either the newly added [RxDocument](./rx-document.md) or the previous existing document.
```js
const doc = await myCollection.insertIfNotExists({
name: 'foo',
lastname: 'bar'
});
```
### bulkInsert()
When you have to insert many documents at once, use bulk insert. This is much faster than calling `.insert()` multiple times.
Returns an object with a `success` and `error` arrays.
```js
const result = await myCollection.bulkInsert([{
name: 'foo1',
lastname: 'bar1'
},
{
name: 'foo2',
lastname: 'bar2'
}]);
// > {
// success: [RxDocument, RxDocument],
// error: []
// }
```
:::note
`bulkInsert` will not fail on update conflicts and you cannot expect that on failure the other documents are not inserted. Also, the call to `bulkInsert()` will not throw if a single document errors because of validation errors. Instead it will return the error in the `.error` property of the returned object.
:::
### bulkRemove()
When you want to remove many documents at once, use bulk remove. Returns an object with a `success`- and `error`-array.
```js
const result = await myCollection.bulkRemove([
'primary1',
'primary2'
]);
// > {
// success: [RxDocument, RxDocument],
// error: []
// }
```
Instead of providing the document ids, you can also use the [RxDocument](./rx-document.md) instances. This can have better performance if your code knows them already at the moment of removing them:
```js
const result = await myCollection.bulkRemove([
myRxDocument1,
myRxDocument2,
/* ... */
]);
```
### upsert()
Inserts the document if it does not exist within the collection, otherwise it will overwrite it. Returns the new or overwritten RxDocument.
When the document already exists, any [inline attachments](./rx-attachment.md#inline-attachments-on-insert-and-upsert) in the upsert data are **merged** with existing attachments by default. Pass `{ deleteExistingAttachments: true }` as the second argument to replace all existing attachments instead.
```js
const doc = await myCollection.upsert({
name: 'foo',
lastname: 'bar2'
});
// with options
const doc2 = await myCollection.upsert(docData, { deleteExistingAttachments: true });
```
### bulkUpsert()
Same as `upsert()` but runs over multiple documents. Improves performance compared to running many `upsert()` calls.
Returns an `error` and a `success` array. Accepts an optional second argument for [upsert options](./rx-attachment.md#upsert-behavior-with-attachments).
```js
const docs = await myCollection.bulkUpsert([
{
name: 'foo',
lastname: 'bar2'
},
{
name: 'bar',
lastname: 'foo2'
}
]);
/**
* {
* success: [RxDocument, RxDocument]
* error: [],
* }
*/
```
### incrementalUpsert()
When you run many upsert operations on the same RxDocument in a very short timespan, you might get a `409 Conflict` error.
This means that you tried to run a `.upsert()` on the document, while the previous upsert operation was still running.
To prevent these types of errors, you can run incremental upsert operations.
The behavior is similar to [RxDocument.incrementalModify](./rx-document.md#incrementalModify).
```js
const docData = {
name: 'Bob', // primary
lastName: 'Kelso'
};
myCollection.upsert(docData);
myCollection.upsert(docData);
// -> throws because of parallel update to the same document
myCollection.incrementalUpsert(docData);
myCollection.incrementalUpsert(docData);
myCollection.incrementalUpsert(docData);
// wait until last upsert finished
await myCollection.incrementalUpsert(docData);
// -> works
```
### find()
To find documents in your collection, use this method. [See RxQuery.find()](./rx-query.md#find).
```js
// find all that are older than 18
const olderDocuments = await myCollection
.find()
.where('age')
.gt(18)
.exec(); // execute
```
### findOne()
This does basically what find() does, but it returns only a single document. You can pass a primary value to find a single document more easily.
To find documents in your collection, use this method. [See RxQuery.find()](./rx-query.md#findOne).
```js
// get document with name:foobar
myCollection.findOne({
selector: {
name: 'foo'
}
}).exec().then(doc => console.dir(doc));
// get document by primary, functionally identical to above query
myCollection.findOne('foo')
.exec().then(doc => console.dir(doc));
```
### findByIds()
Find many documents by their id (primary value). This has a way better performance than running multiple `findOne()` or a `find()` with a big `$or` selector.
Returns a `Map` where the primary key of the document is mapped to the document. Documents that do not exist or are deleted, will not be inside of the returned Map.
```js
const ids = [
'alice',
'bob',
/* ... */
];
const docsMap = await myCollection.findByIds(ids);
console.dir(docsMap); // Map(2)
```
:::note
The `Map` returned by `findByIds` is not guaranteed to return elements in the same order as the list of ids passed to it.
:::
### exportJSON()
Use this function to create a JSON export from every document in the collection.
Before `exportJSON()` and `importJSON()` can be used, you have to add the `json-dump` plugin.
```javascript
import { addRxPlugin } from 'rxdb';
import { RxDBJsonDumpPlugin } from 'rxdb/plugins/json-dump';
addRxPlugin(RxDBJsonDumpPlugin);
```
```js
myCollection.exportJSON()
.then(json => console.dir(json));
```
### importJSON()
To import the JSON dump into your collection, use this function.
```js
// import the dump to the database
myCollection.importJSON(json)
.then(() => console.log('done'));
```
Note that importing will fire events for each inserted document.
### remove()
Removes all known data of the collection and its previous versions.
This removes the documents, the schemas, and older schemaVersions.
```js
await myCollection.remove();
// collection is now removed and can be re-created
```
### close()
Removes the collection's object instance from the [RxDatabase](./rx-database.md). This is to free up memory and stop all observers and replications. It will not delete the collection's data. When you create the collection again with `database.addCollections()`, the newly added collection will still have all data.
```js
await myCollection.close();
```
### onClose / onRemove()
With these you can add a function that is run when the collection was closed or removed.
This works even across multiple browser tabs so you can detect when another tab removes the collection
and your application can behave accordingly.
```js
await myCollection.onClose(() => console.log('I am closed'));
await myCollection.onRemove(() => console.log('I am removed'));
```
### isRxCollection
Returns true if the given object is an instance of RxCollection. Returns false if not.
```js
const is = isRxCollection(myObj);
```
## FAQ
No, the javascript instance of the collections will not automatically load into the database on page reloads.
You have to call the `addCollections()` method each time you create your database. This will create the JavaScript object instance of the RxCollection so that you can use it in the RxDatabase. The persisted data will automatically be available in your RxCollection each time you create it.
In the open-source version of RxDB, the amount of RxCollections that can exist in parallel is limited to {NON_PREMIUM_COLLECTION_LIMIT}.
To remove this limit, you can purchase the [Premium Plugins](/premium/) and call the `setPremiumFlag()` function before creating a database:
```ts
import { setPremiumFlag } from 'rxdb-premium/plugins/shared';
setPremiumFlag();
```
---
## RxDocument
An RxDocument is an object which represents the data of a single JSON document stored in a [collection](./rx-collection.md). It can be compared to a single record in a relational database table. You get an `RxDocument` either as return on inserts/updates, or as result-set of [queries](./rx-query.md).
RxDB works on RxDocuments instead of plain JSON data to have more convenient operations on the documents. Also Documents that are fetched multiple times by different queries or operations are automatically de-duplicated by RxDB in memory.
## insert
To insert a document into a collection, you have to call the collection's `.insert()` function.
```js
await myCollection.insert({
name: 'foo',
lastname: 'bar'
});
```
## find
To find documents in a collection, you have to call the collection's `.find()` function. [See RxQuery](./rx-query.md).
```js
const docs = await myCollection.find().exec(); // <- find all documents
```
## Functions
### get()
This will get a single field of the document. If the field is encrypted, it will be automatically decrypted before returning.
```js
const name = myDocument.get('name'); // returns the name
// OR
const name = myDocument.name;
```
### get$()
This function returns an observable of the given path's value.
The current value of this path will be emitted each time the document changes.
```js
// get the live-updating value of 'name'
let isName;
myDocument.get$('name')
.subscribe(newName => {
isName = newName;
});
await myDocument.incrementalPatch({name: 'foobar2'});
console.dir(isName); // isName is now 'foobar2'
// OR
myDocument.name$
.subscribe(newName => {
isName = newName;
});
```
### proxy-get
All properties of an `RxDocument` are assigned as getters so you can also directly access values instead of using the get()-function.
```js
// Identical to myDocument.get('name');
const name = myDocument.name;
// Can also get nested values.
const nestedValue = myDocument.whatever.nestedfield;
// Also usable with observables:
myDocument.firstName$.subscribe(newName => console.log('name is: ' + newName));
// > 'name is: Stefe'
await myDocument.incrementalPatch({firstName: 'Steve'});
// > 'name is: Steve'
```
### update()
Updates the document based on the [Mongo update syntax](https://docs.mongodb.com/manual/reference/operator/update-field/), based on the [mingo library](https://github.com/kofrasa/mingo#updating-documents).
```js
/**
* If not done before, you have to add the update plugin.
*/
import { addRxPlugin } from 'rxdb';
import { RxDBUpdatePlugin } from 'rxdb/plugins/update';
addRxPlugin(RxDBUpdatePlugin);
await myDocument.update({
$inc: {
age: 1 // increases age by 1
},
$set: {
firstName: 'foobar' // sets firstName to foobar
}
});
```
### modify()
Updates a document's data based on a function that mutates the current data and returns the new value.
```js
const changeFunction = (oldData) => {
oldData.age = oldData.age + 1;
oldData.name = 'foooobarNew';
return oldData;
}
await myDocument.modify(changeFunction);
console.log(myDocument.name); // 'foooobarNew'
```
### patch()
Overwrites the given attributes in the document's data.
```js
await myDocument.patch({
name: 'Steve',
age: undefined // setting an attribute to undefined will remove it
});
console.log(myDocument.name); // 'Steve'
```
### Prevent conflicts with the incremental methods {#incrementalModify}
Making a normal change to the non-latest version of an `RxDocument` will lead to a `409 CONFLICT` error because RxDB
uses [revision checks](./transactions-conflicts-revisions.md) instead of transactions.
To make a change to a document, no matter what the current state is, you can use the `incremental` methods:
```js
// update
await myDocument.incrementalUpdate({
$inc: {
age: 1 // increases age by 1
}
});
// modify
await myDocument.incrementalModify(docData => {
docData.age = docData.age + 1;
return docData;
});
// patch
await myDocument.incrementalPatch({
age: 100
});
// remove
await myDocument.incrementalRemove({
age: 100
});
```
### getLatest()
Returns the latest known state of the `RxDocument`.
```js
const myDocument = await myCollection.findOne('foobar').exec();
const docAfterEdit = await myDocument.incrementalPatch({
age: 10
});
const latestDoc = myDocument.getLatest();
console.log(docAfterEdit === latestDoc); // > true
```
### Observe $ {#observe}
Calling this will return an [RxJS Observable](https://rxjs.dev/guide/observable) which emits the current newest state of the RxDocument.
```js
// get all changeEvents
myDocument.$
.subscribe(currentRxDocument => console.dir(currentRxDocument));
```
### remove()
This removes the document from the collection. Notice that this will not purge the document from the store but set `_deleted:true` so that it will be no longer returned on queries.
To fully purge a document, use the [cleanup plugin](./cleanup.md).
```js
myDocument.remove();
```
### Remove and update in a single atomic operation
Sometimes you want to change a document's value and also remove it in the same operation. For example this can be useful when you use [replication](./replication.md) and want to set a `deletedAt` timestamp. Then you might have to ensure that setting this timestamp and deleting the document happens in the same atomic operation.
To do this the modifying operations of a document accept setting the `_deleted` field. For example:
```ts
// update() and remove()
await doc.update({
$set: {
deletedAt: new Date().getTime(),
_deleted: true
}
});
// modify() and remove()
await doc.modify(data => {
data.age = 1;
data._deleted = true;
return data;
});
```
### deleted$
Emits a boolean value, depending on whether the RxDocument is deleted or not.
```js
let lastState = null;
myDocument.deleted$.subscribe(state => lastState = state);
console.log(lastState);
// false
await myDocument.remove();
console.log(lastState);
// true
```
### get deleted
A getter to get the current value of `deleted$`.
```js
console.log(myDocument.deleted);
// false
await myDocument.remove();
console.log(myDocument.deleted);
// true
```
### toJSON()
Returns the document's data as plain JSON object. This will return an **immutable** object. To get something that can be modified, use `toMutableJSON()` instead.
```js
const json = myDocument.toJSON();
console.dir(json);
/* { passportId: 'h1rg9ugdd30o',
firstName: 'Carolina',
lastName: 'Gibson',
age: 33 ...
*/
```
You can also set `withMetaFields: true` to get additional meta fields like the revision, [attachments](./rx-attachment.md) or the deleted flag.
```js
const json = myDocument.toJSON(true);
console.dir(json);
/* { passportId: 'h1rg9ugdd30o',
firstName: 'Carolina',
lastName: 'Gibson',
_deleted: false,
_attachments: { ... },
_rev: '1-aklsdjfhaklsdjhf...'
*/
```
### toMutableJSON()
Same as `toJSON()` but returns a deep cloned object that can be mutated afterwards.
Remember that deep cloning is expensive and should only be done when necessary.
```js
const json = myDocument.toMutableJSON();
json.firstName = 'Alice'; // The returned document can be mutated
```
:::note All methods of RxDocument are bound to the instance
When you get a method from a `RxDocument`, the method is automatically bound to the document's instance. This means you do not have to use things like `myMethod.bind(myDocument)` like you would do in jsx.
:::
### isRxDocument
Returns true if the given object is an instance of RxDocument. Returns false if not.
```js
const is = isRxDocument(myObj);
```
## Document Lifetime and Immutability
**RxDocument instances are immutable.** Each instance represents a snapshot of the document at the time it was fetched or last written. Modifying a document does not update existing instances of it - it creates a new `RxDocument` instance with the updated data. The old instance retains its original data.
```js
const doc = await myCollection.findOne('foobar').exec();
console.log(doc.age); // 10
await doc.incrementalPatch({ age: 20 });
// The original instance still has the old data
console.log(doc.age); // 10
// Use getLatest() to get the updated state
console.log(doc.getLatest().age); // 20
```
**RxDB de-duplicates document instances.** When the same document is fetched multiple times without any writes in between, RxDB returns the same instance to save memory. Once a write occurs, subsequent fetches return a new instance reflecting the updated state.
**Calling non-incremental write methods on an outdated instance throws a `CONFLICT` error.** If you hold a reference to a document and another operation modifies that document in the meantime, calling `.patch()`, `.update()`, or `.modify()` on the outdated instance will fail with a conflict error. See [Transactions, Conflicts and Revisions](./transactions-conflicts-revisions.md) for details on how RxDB handles conflicts.
To avoid this, either:
- Use the [incremental methods](#incrementalModify) (`incrementalPatch`, `incrementalModify`, `incrementalUpdate`) which always fetch the latest state before applying changes.
- Call `getLatest()` to get the current state before writing.
- Re-query the collection to get a fresh document.
**How long to keep a reference to an `RxDocument`.** Treat an `RxDocument` like plain JSON data - it is a snapshot valid at the time of retrieval. RxDB manages query result caching internally via [event-reduce](./rx-query.md), so you do not need to cache documents yourself. For components that display document data and need live updates, subscribe to the document's `$` observable instead of holding a static reference.
---
## RxQuery
import {BetaBlock} from '@site/src/components/beta-block';
import {Faq, FaqItem} from '@site/src/components/faq';
# RxQuery
To find documents inside of an [RxCollection](./rx-collection.md), RxDB uses the RxQuery interface that handles all query operations: it serves as the main interface for fetching documents, relies on a MongoDB-like [Mango Query Syntax](https://github.com/cloudant/mango), and provides three types of queries: [find()](#find), [findOne()](#findOne) and [count()](#count). By caching and de-duplicating results, RxQuery ensures efficient in-memory handling, and when queries are observed or re-run, the [EventReduce algorithm](https://github.com/pubkey/event-reduce) speeds up updates for a fast real-time experience and queries that run more than once.
## find()
To create a basic `RxQuery`, call `.find()` on a collection and insert selectors. The result-set of normal queries is an array with documents.
```js
// find all that are older than 18
const query = myCollection
.find({
selector: {
age: {
$gt: 18
}
}
});
```
## findOne() {#findOne}
A findOne-query has only a single [RxDocument](./rx-document.md) or `null` as result-set.
```js
// find alice
const query = myCollection
.findOne({
selector: {
name: 'alice'
}
});
```
```js
// find the youngest one
const query = myCollection
.findOne({
selector: {},
sort: [
{age: 'asc'}
]
});
```
```js
// find one document by the primary key
const query = myCollection.findOne('foobar');
```
## exec()
Returns a `Promise` that resolves with the result-set of the query.
```js
const query = myCollection.find();
const results = await query.exec();
console.dir(results); // > [RxDocument,RxDocument,RxDocument..]
```
On `.findOne()` queries, you can call `.exec(true)` to ensure your document exists and to make TypeScript handling easier:
```ts
// docOrUndefined can be RxDocument or null
// which then has to be handled to be typesafe.
const docOrUndefined = await myCollection.findOne().exec();
// with .exec(true), it will throw if the document
// cannot be found and always return type RxDocument
const doc = await myCollection.findOne().exec(true);
```
## Observe $ {#observe}
An `BehaviorSubject` [see](https://medium.com/@luukgruijs/understanding-rxjs-behaviorsubject-replaysubject-and-asyncsubject-8cc061f1cfc0) that always has the current result-set as value.
This is extremely helpful when used together with UIs that should always show the same state as what is written in the database.
```js
const query = myCollection.find();
const querySub = query.$.subscribe(results => {
console.log('got results: ' + results.length);
});
// > 'got results: 5' // BehaviorSubjects emit on subscription
await myCollection.insert({/* ... */}); // insert one
// > 'got results: 6' // $.subscribe() was called again with the new results
// stop watching this query
querySub.unsubscribe()
```
## update()
Runs an [update](./rx-document.md#update) on every RxDocument of the query-result.
```js
// to use the update() method, you need to add the update plugin.
import { RxDBUpdatePlugin } from 'rxdb/plugins/update';
addRxPlugin(RxDBUpdatePlugin);
const query = myCollection.find({
selector: {
age: {
$gt: 18
}
}
});
await query.update({
$inc: {
age: 1 // increases age of every found document by 1
}
});
```
## patch() / incrementalPatch()
Runs the [RxDocument.patch()](./rx-document.md#patch) function on every RxDocument of the query result.
```js
const query = myCollection.find({
selector: {
age: {
$gt: 18
}
}
});
await query.patch({
age: 12 // set the age of every found to 12
});
```
## modify() / incrementalModify()
Runs the [RxDocument.modify()](./rx-document.md#modify) function on every RxDocument of the query result.
```js
const query = myCollection.find({
selector: {
age: {
$gt: 18
}
}
});
await query.modify((docData) => {
docData.age = docData.age + 1; // increases age of every found document by 1
return docData;
});
```
## remove() / incrementalRemove()
Deletes all found documents. Returns a promise which resolves to the deleted documents.
```javascript
// All documents where the age is less than 18
const query = myCollection.find({
selector: {
age: {
$lt: 18
}
}
});
// Remove the documents from the collection
const removedDocs = await query.remove();
```
On `.findOne()` queries, `.remove()` returns `null` when no document matches. You can call `.remove(true)` to throw if the document is missing, similar to `.exec(true)`:
```ts
// returns null if no document matches
const removed = await myCollection.findOne('foobar').remove();
// throws if no document matches, return type is always RxDocument
const removed = await myCollection.findOne('foobar').remove(true);
```
## doesDocumentDataMatch()
Returns `true` if the given document data matches the query.
```js
const documentData = {
id: 'foobar',
age: 19
};
myCollection.find({
selector: {
age: {
$gt: 18
}
}
}).doesDocumentDataMatch(documentData); // > true
myCollection.find({
selector: {
age: {
$gt: 20
}
}
}).doesDocumentDataMatch(documentData); // > false
```
## Query Builder Plugin
To use chained query methods, you can also use the `query-builder` plugin.
```ts
// add the query builder plugin
import { addRxPlugin } from 'rxdb';
import { RxDBQueryBuilderPlugin } from 'rxdb/plugins/query-builder';
addRxPlugin(RxDBQueryBuilderPlugin);
// now you can use chained query methods
const query = myCollection.find().where('age').gt(18);
const result = await query.exec();
```
## Query Examples
Here some examples to learn quickly how to write queries without reading the docs.
- [Pouch-find-docs](https://github.com/pouchdb/pouchdb/blob/master/packages/node_modules/pouchdb-find/README.md) - learn how to use mango-queries
- [mquery-docs](https://github.com/aheckmann/mquery/blob/master/README.md) - learn how to use chained-queries
```js
// directly pass search-object
myCollection.find({
selector: {
name: { $eq: 'foo' }
}
})
.exec().then(documents => console.dir(documents));
/*
* find by using sql equivalent '%like%' syntax
* This example will e.g. match 'foo' but also 'fifoo' or 'foofa' or 'fifoofa'
* Notice that in RxDB queries, a regex is
* represented as a $regex string with the
* $options parameter for flags.
* Using a RegExp instance is not allowed
* because they are not JSON.stringify()-able
* and also RegExp instances are mutable which
* could cause undefined behavior when the
* RegExp is mutated
* after the query was parsed.
*/
myCollection.find({
selector: {
name: { $regex: '.*foo.*' }
}
})
.exec().then(documents => console.dir(documents));
// find using a composite statement eg: $or
// This example checks where name is either foo
// or if name is not existent on the document
myCollection.find({
selector: { $or: [ { name: { $eq: 'foo' } }, { name: { $exists: false } }] }
})
.exec().then(documents => console.dir(documents));
// do a case insensitive search
// This example will match 'foo' or 'FOO' or 'FoO' etc...
myCollection.find({
selector: { name: { $regex: '^foo$', $options: 'i' } }
})
.exec().then(documents => console.dir(documents));
// chained queries
myCollection.find().where('name').eq('foo')
.exec().then(documents => console.dir(documents));
```
:::note RxDB will always append the primary key to the sort parameters
For several performance optimizations, like the [EventReduce algorithm](https://github.com/pubkey/event-reduce), RxDB expects all queries to return a deterministic sort order that does not depend on the insert order of the documents. To ensure a deterministic ordering, RxDB will always append the primary key as last sort parameter to all queries and to all indexes.
This works in contrast to most other databases where a query without sorting would return the documents in the order in which they had been inserted to the database.
:::
## Setting a specific index
By default, the query will be sent to the RxStorage, where a query planner will determine which one of the available indexes must be used.
But the query planner cannot know everything and sometimes will not pick the most optimal index.
To improve query performance, you can specify which index must be used, when running the query.
```ts
const query = myCollection
.findOne({
selector: {
age: {
$gt: 18
},
gender: {
$eq: 'm'
}
},
/**
* Because the developer knows that 50% of the documents are 'male',
* but only 20% are below age 18,
* it makes sense to enforce using the
* ['gender', 'age'] index to improve
* performance.
* This could not be known by the query
* planner which might have chosen
* ['age', 'gender'] instead.
*/
index: ['gender', 'age']
});
```
## Count
When you only need the amount of documents that match a query, but you do not need the document data itself, you can use a count query for **better performance**.
The performance difference compared to a normal query differs depending on which [RxStorage](./rx-storage.md) implementation is used.
```ts
const query = myCollection.count({
selector: {
age: {
$gt: 18
}
}
// 'limit' and 'skip' MUST NOT be set for count queries.
});
// get the count result once
const matchingAmount = await query.exec(); // > number
// observe the result
query.$.subscribe(amount => {
console.log('Currently has ' + amount + ' documents');
});
```
:::note
Count queries have a better performance than normal queries because they do not have to fetch the full document data out of the storage. Therefore it is **not** possible to run a `count()` query with a selector that requires fetching and comparing the document data. So if your query selector **does not** fully match an index of the schema, it is not allowed to run it. These queries would have no performance benefit compared to normal queries but have the tradeoff of not using the fetched document data for caching.
:::
```ts
/**
* The following will throw an error because
* the count operation cannot run on any specific index range
* because the $regex operator is used.
*/
const query = myCollection.count({
selector: {
age: {
$regex: 'foobar'
}
}
});
/**
* The following will throw an error because
* the count operation cannot run on any specific index range
* because there is no ['age' ,'otherNumber'] index
* defined in the schema.
*/
const query = myCollection.count({
selector: {
age: {
$gt: 20
},
otherNumber: {
$gt: 10
}
}
});
```
If you want to count these kinds of queries, you should do a normal query instead and use the length of the result set as counter. This has the same performance as running a non-fully-indexed count which has to fetch all document data from the database and run a query matcher.
```ts
// get count manually once
const resultSet = await myCollection.find({
selector: {
age: {
$regex: 'foobar'
}
}
}).exec();
const count = resultSet.length;
// observe count manually
const count$ = myCollection.find({
selector: {
age: {
$regex: 'foobar'
}
}
}).$.pipe(
map(result => result.length)
);
/**
* To allow non-fully-indexed count queries,
* you can also specify that by setting allowSlowCount: true
* when creating the database.
*/
const database = await createRxDatabase({
name: 'mydatabase',
allowSlowCount: true, // set this to true [default=false]
/* ... */
});
```
### `allowSlowCount`
To allow non-fully-indexed count queries, you can also specify that by setting `allowSlowCount: true` when creating the database.
Doing this is mostly not wanted, because it would run the counting on the storage without having the document stored in the RxDB document cache.
This is only recommended if the RxStorage is running remotely like in a WebWorker and you do not always want to send the document-data between the worker and the main thread. In this case you might only need the count-result instead to save performance.
## RxQuery instances are immutable
Because RxDB is a reactive database, we can do heavy performance-optimisation on query-results which change over time. To be able to do this, RxQueries have to be immutable.
This means, when you have a `RxQuery` and run a `.where()` on it, the original RxQuery object is not changed. Instead the where-function returns a new `RxQuery`-Object with the changed where-field. Keep this in mind if you create RxQueries and change them afterwards.
Example:
```javascript
const queryObject = myCollection.find().where('age').gt(18);
// Creates a new RxQuery object, does not modify previous one
queryObject.sort('name');
const results = await queryObject.exec();
console.dir(results); // result-documents are not sorted by name
const queryObjectSort = queryObject.sort('name');
const results = await queryObjectSort.exec();
console.dir(results); // result-documents are now sorted
```
### isRxQuery
Returns true if the given object is an instance of RxQuery. Returns false if not.
```js
const is = isRxQuery(myObj);
```
## liveQueryUpdateThrottleTime
When set to a positive number (milliseconds), write-triggered live query updates are grouped with [RxJS auditTime](https://rxjs.dev/api/operators/auditTime) before `_ensureEqual()` runs. This limits how often RxDB re-evaluates a live query during write bursts, which avoids repeated full-result rebuilds for queries with large result sets.
The first query result is always emitted immediately so subscriptions still behave like a `BehaviorSubject` on startup. Only subsequent change-triggered reruns are throttled.
The value can be set at the database level (applies to all collections) or overridden per collection via `liveQueryUpdateThrottleTime` in `addCollections`.
```ts
// set at database level - applies to all collections
const db = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
liveQueryUpdateThrottleTime: 100 // ms
});
// or override per collection
const collections = await db.addCollections({
heroes: {
schema: heroSchema,
liveQueryUpdateThrottleTime: 200 // ms, overrides the database-level value
}
});
```
## Design Decisions
Like most other noSQL-Databases, RxDB uses the [mango-query-syntax](https://github.com/cloudant/mango) similar to MongoDB and others.
- We use the JSON-based Mango Query Syntax because:
- Mango Queries work better with TypeScript compared to SQL strings.
- Mango Queries are composable and easy to transform by code without joining SQL strings.
- Queries can be run very fast and efficient with only a minimal query planner to plan the best indexes and operations.
- NoSQL queries can be optimized with the [EventReduce](https://github.com/pubkey/event-reduce) algorithm to improve performance of observed and cached queries.
## FAQ
No, RxDB does not support partial document retrieval. Because RxDB is a client-side database with limited memory, it caches and de-duplicates entire documents across multiple queries. Even if you only need a few fields, most storages must still fetch the entire JSON data, so subselecting fields would not significantly improve performance. Therefore, RxDB always returns full documents. If you only need certain fields, you can filter them out in your application code or consider storing just the necessary data in a separate collection.
RxDB runs entirely on the client side. Any "aggregation" or data processing you might do within RxDB would still happen in the same JavaScript environment as your application code. Therefore, there's no real performance advantage or difference between doing the aggregation in RxDB vs. doing it in your own code after fetching the data. As a result, RxDB doesn't provide built-in aggregation methods. Instead, just query the documents you need and perform any calculations directly in your app's code.
RxDB is a client-side database and does not provide built-in cross-collection queries or transactions. Instead, you can execute multiple queries in your JavaScript code and combine their results as needed. Because everything runs in the same environment, this approach offers the same performance you would get if cross-collection queries were built in - without the added complexity.
RxDB relies on various storage engines as its backend, and these storage engines generally do not support case-insensitive search natively, like [IndexedDB](./rx-storage-indexeddb.md) or [FoundationDB](./rx-storage-foundationdb.md). This limitation arises from the design of these engines, which prioritize efficiency and flexibility for specific types of queries rather than universal features like case-insensitivity. Although RxDB does not offer built-in support for case-insensitive search, there are two common workarounds:
- **Store Data in a Meta-Field for Lowercase Search**: To enable case-insensitive search, you can store an additional field in your documents where the relevant text data is preprocessed and saved in lowercase.
```ts
const document = {
name: 'John Doe',
nameLowercase: 'john doe' // Meta-field
};
await myCollection.insert(document);
const query = myCollection.find({
selector: {
nameLowercase: { $eq: 'john doe' }
}
});
```
- **Use a Regex Query**: Regular expressions can perform case-insensitive searches. For example:
```ts
const query = myCollection.find({
selector: {
name: { $regex: '^john doe$', $options: 'i' } // Case-insensitive regex
}
});
```
However, this method has a significant downside: regex queries often cannot leverage indexes efficiently. As a result, they may be slower, especially for large datasets.
---
## RxStorage Layer - Choose the Perfect RxDB Storage for Every Use Case
import { IconStorage } from '@site/src/components/icons/storage';
import { HeadlineWithIcon } from '@site/src/components/headline-with-icon';
# }>RxStorage
RxDB is not a self-contained database. Instead the data is stored in an implementation of the [RxStorage interface](https://github.com/pubkey/rxdb/blob/master/src/types/rx-storage.interface.d.ts). This allows you to **switch out** the underlying data layer, depending on the JavaScript environment and performance requirements. For example you can use the SQLite storage for a capacitor app or you can use the LocalStorage RxStorage to store data in localstorage in a browser-based application. There are also storages for other JavaScript runtimes like Node.js, React-Native, NativeScript and more.
## Quick Recommendations
- In the Browser: Use the [LocalStorage](./rx-storage-localstorage.md) storage for simple setup and small build size. For bigger datasets, use either the [dexie.js storage](./rx-storage-dexie.md) (free) or the [IndexedDB RxStorage](./rx-storage-indexeddb.md) if you have [π premium access](/premium/) which is a bit faster and has a smaller build size.
- In [Electron](./electron-database.md) and [ReactNative](./react-native-database.md): Use the [SQLite RxStorage](./rx-storage-sqlite.md) if you have [π premium access](/premium/) or the [trial-SQLite RxStorage](./rx-storage-sqlite.md) for tryouts. For ultimate performance in Expo and React Native, use the [Expo Filesystem RxStorage](./rx-storage-filesystem-expo.md).
- In Capacitor: Use the [SQLite RxStorage](./rx-storage-sqlite.md) if you have [π premium access](/premium/), otherwise use the [localStorage](./rx-storage-localstorage.md) storage.
## Configuration Examples
The RxStorage layer of RxDB is very flexible. Here are some examples on how to configure more complex settings:
### Storing much data in a browser securely
Lets say you build a browser app that needs to store a big amount of data as securely as possible. Here we can use a combination of the storages (encryption, IndexedDB, compression, schema-checks) that increase security and reduce the stored data size.
We use the schema-validation on the top level to ensure schema-errors are clearly readable and do not contain [encrypted](./encryption.md)/[compressed](./key-compression.md) data. The encryption is used inside of the compression because encryption of compressed data is more efficient.
```ts
import { wrappedValidateAjvStorage } from 'rxdb/plugins/validate-ajv';
import { wrappedKeyCompressionStorage } from 'rxdb/plugins/key-compression';
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
const myDatabase = await createRxDatabase({
storage: wrappedValidateAjvStorage({
storage: wrappedKeyCompressionStorage({
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
})
})
})
});
```
### High query Load
Also we can utilize a combination of storages to create a database that is optimized to run complex queries on the data really fast. Here we use the sharding storage together with the worker storage. This allows to run queries in parallel multithreading instead of a single JavaScript process. Because the worker initialization can slow down the initial page load, we also use the [localstorage-meta-optimizer](./rx-storage-localstorage-meta-optimizer.md) to improve initialization time.
```ts
import { getRxStorageSharding } from 'rxdb-premium/plugins/storage-sharding';
import { getRxStorageWorker } from 'rxdb-premium/plugins/storage-worker';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
import {
getLocalstorageMetaOptimizerRxStorage
} from 'rxdb-premium/plugins/storage-localstorage-meta-optimizer';
const myDatabase = await createRxDatabase({
storage: getLocalstorageMetaOptimizerRxStorage({
storage: getRxStorageSharding({
storage: getRxStorageWorker({
workerInput: 'path/to/worker.js',
storage: getRxStorageIndexedDB()
})
})
})
});
```
### Low Latency on Writes and Simple Reads
Here we create a storage configuration that is optimized to have a low latency on simple reads and writes. It uses the memory-mapped storage to fetch and store data in memory. For persistence the OPFS storage is used in the main thread which has lower latency for fetching big chunks of data when at initialization the data is loaded from disk into memory. We do not use workers because sending data from the main thread to workers and backwards would increase the latency.
```ts
import {
getLocalstorageMetaOptimizerRxStorage
} from 'rxdb-premium/plugins/storage-localstorage-meta-optimizer';
import {
getMemoryMappedRxStorage
} from 'rxdb-premium/plugins/storage-memory-mapped';
import { getRxStorageOPFSMainThread } from 'rxdb-premium/plugins/storage-worker';
const myDatabase = await createRxDatabase({
storage: getLocalstorageMetaOptimizerRxStorage({
storage: getMemoryMappedRxStorage({
storage: getRxStorageOPFSMainThread()
})
})
});
```
## All RxStorage Implementations List
### Memory
A storage that stores the data as plain data in the memory of the JavaScript process. Really fast and can be used in all environments. [Read more](./rx-storage-memory.md)
### LocalStorage
The localStorage based storage stores the data inside of a browsers [localStorage API](./articles/localstorage.md). It is the easiest to set up and has a small bundle size. **If you are new to RxDB, you should start with the LocalStorage RxStorage**. [Read more](./rx-storage-localstorage.md)
### π IndexedDB
The IndexedDB `RxStorage` is based on plain IndexedDB. For most use cases, this has the best performance together with the OPFS storage. [Read more](./rx-storage-indexeddb.md)
### π OPFS
The OPFS `RxStorage` is based on the File System Access API. This has the best performance of all other non-in-memory storage, when RxDB is used inside of a browser. [Read more](./rx-storage-opfs.md)
### π Filesystem Node
The Filesystem Node storage is best suited when you use RxDB in a Node.js process or with [electron.js](./electron.md). [Read more](./rx-storage-filesystem-node.md)
### Storage Wrapper Plugins
#### π Worker
The worker RxStorage is a wrapper around any other RxStorage which allows to run the storage in a WebWorker (in browsers) or a Worker Thread (in Node.js). By doing so, you can take CPU load from the main process and move it into the worker's process which can improve the perceived performance of your application. [Read more](./rx-storage-worker.md)
#### π SharedWorker
The SharedWorker RxStorage is a wrapper around any other RxStorage which allows to run the storage in a SharedWorker (only in browsers). By doing so, you can take CPU load from the main process and move it into the worker's process which can improve the perceived performance of your application. [Read more](./rx-storage-shared-worker.md)
#### Remote
The Remote RxStorage is made to use a remote storage and communicate with it over an asynchronous message channel. The remote part could be on another JavaScript process or even on a different host machine. Mostly used internally in other storages like Worker or Electron-ipc. [Read more](./rx-storage-remote.md)
#### π Sharding
On some `RxStorage` implementations (like IndexedDB), a huge performance improvement can be done by sharding the documents into multiple database instances. With the sharding plugin you can wrap any other `RxStorage` into a sharded storage. [Read more](./rx-storage-sharding.md)
#### π Memory Mapped
The memory-mapped [RxStorage](./rx-storage.md) is a wrapper around any other RxStorage. The wrapper creates an in-memory storage that is used for query and write operations. This memory instance stores its data in an underlying storage for persistence.
The main reason to use this is to improve query/write performance while still having the data stored on disk. [Read more](./rx-storage-memory-mapped.md)
#### π Localstorage Meta Optimizer
The [RxStorage](./rx-storage.md) Localstorage Meta Optimizer is a wrapper around any other RxStorage. The wrapper uses the original RxStorage for normal collection documents. But to optimize the initial page load time, it uses [localstorage](./articles/localstorage.md) to store the plain key-value metadata that RxDB needs to create databases and collections. This plugin can only be used in browsers. [Read more](./rx-storage-localstorage-meta-optimizer.md)
#### Electron IpcRenderer & IpcMain
To use RxDB in [electron](./electron-database.md), it is recommended to run the RxStorage in the main process and the [RxDatabase](./rx-database.md) in the renderer processes. With the rxdb electron plugin you can create a remote RxStorage and consume it from the renderer process. [Read more](./electron.md)
### Third Party based Storages
#### π Expo Filesystem
The Expo Filesystem storage brings blazing-fast OPFS capabilities to React Native and Expo applications, bypassing the bridge via JSI bindings for maximum performance. This is the fastest storage engine for React Native. [Read more](./rx-storage-filesystem-expo.md)
#### π SQLite
The SQLite storage has great performance when RxDB is used on **Node.js**, **Electron**, **React Native**, **Cordova** or **Capacitor**. [Read more](./rx-storage-sqlite.md)
#### Dexie.js
The Dexie.js based storage is based on the Dexie.js IndexedDB wrapper library. [Read more](./rx-storage-dexie.md)
#### MongoDB
To use RxDB on the server side, the MongoDB RxStorage provides a way of having a secure, scalable and performant storage based on the popular MongoDB NoSQL database. [Read more](./rx-storage-mongodb.md)
#### DenoKV
To use RxDB in Deno. The DenoKV RxStorage provides a way of having a secure, scalable and performant storage based on the Deno Key Value Store. [Read more](./rx-storage-denokv.md)
#### FoundationDB
To use RxDB on the server side, the FoundationDB RxStorage provides a way of having a secure, fault-tolerant and performant storage. [Read more](./rx-storage-foundationdb.md)
---
## RxDB LocalStorage - The Easiest Way to Persist Data in Your Web App
import {Steps} from '@site/src/components/steps';
# RxStorage LocalStorage
RxDB can persist data in various ways. One of the simplest methods is using the browserβs built-in [LocalStorage](./articles/localstorage.md). This storage engine allows you to store and retrieve [RxDB documents](./rx-document.md) directly from the browser without needing additional plugins or libraries.
> **Recommended Default for using RxDB in the Browser**
>
> We highly recommend using LocalStorage for a quick and easy RxDB setup, especially when you want a minimal project configuration. For professional projects, the [IndexedDB RxStorage](./rx-storage-indexeddb.md) is recommended in most cases.
## Key Benefits
1. **Simplicity**: No complicated configurations or external dependencies - LocalStorage is already built into the browser.
2. **Fast for small Datasets**: Writing and Reading small sets of data from localStorage is really fast as shown in [these benchmarks](./articles/localstorage-indexeddb-cookies-opfs-sqlite-wasm.md#performance-comparison).
4. **Ease of Setup**: Just import the plugin, import it, and pass `getRxStorageLocalstorage()` into `createRxDatabase()`. Thatβs it!
## Limitations
While LocalStorage is the easiest way to get started, it does come with some constraints:
1. **Limited Storage Capacity**: Browsers often limit LocalStorage to around [5 MB per domain](./articles/localstorage.md#understanding-the-limitations-of-local-storage), though exact limits vary.
2. **Synchronous Access**: LocalStorage operations block the main thread. This is usually fine for small amounts of data but can cause performance bottlenecks with heavier use.
Despite these limitations, LocalStorage remains a great default option for smaller projects, prototypes, or cases where you need the absolute simplest way to persist data in the browser.
## How to use the LocalStorage RxStorage with RxDB
### Import the Storage
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
```
### Create a Database
```ts
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageLocalstorage()
});
```
### Add a Collection
```ts
await db.addCollections({
tasks: {
schema: {
title: 'tasks schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' }
},
required: ['id', 'title', 'done']
}
}
});
```
### Insert a document
```ts
await db.tasks.insert({ id: 'task-01', title: 'Get started with RxDB', done: false });
```
### Query documents
```ts
const nonDoneTasks = await db.tasks.find({
selector: {
done: {
$eq: false
}
}
}).exec();
```
## Mocking the LocalStorage API for testing in Node.js
While the `localStorage` API only exists in browsers, you can use the LocalStorage based storage in [Node.js](./nodejs-database.md) by using the mock that comes with RxDB.
This is intended to be used in unit tests or other test suites:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import {
getRxStorageLocalstorage,
getLocalStorageMock
} from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageLocalstorage({
localStorage: getLocalStorageMock()
})
});
```
---
## Instant Performance with IndexedDB RxStorage
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {PremiumBlock} from '@site/src/components/premium-block';
# IndexedDB RxStorage
The IndexedDB [RxStorage](./rx-storage.md) is based on plain IndexedDB and can be used in browsers, [electron](./electron-database.md) or [hybrid apps](./articles/mobile-database.md).
Compared to other [browser based storages](./articles/browser-database.md), the IndexedDB storage has the smallest write- and read latency, the fastest initial page load
and the smallest build size. Only for big datasets (more than 10k documents), the [OPFS storage](./rx-storage-opfs.md) is better suited.
While the IndexedDB API itself can be very slow, the IndexedDB storage uses many tricks and performance optimizations, some of which are described [here](./slow-indexeddb.md). For example it uses custom index strings instead of the native IndexedDB indexes, batches cursors for faster bulk reads and many other improvements. The IndexedDB storage also operates on [Write-ahead logging](https://en.wikipedia.org/wiki/Write-ahead_logging) similar to SQLite, to improve write latency while still ensuring consistency on writes.
## IndexedDB performance comparison
Here is some performance comparison with other storages. Compared to the non-memory storages like [OPFS](./rx-storage-opfs.md) and [WASM SQLite](./rx-storage-sqlite.md), IndexedDB has the smallest build size and fastest write speed. Only OPFS is faster on queries over big datasets. See [performance comparison](./rx-storage-performance.md) page for a comparison with all storages.
## Using the IndexedDB RxStorage
To use the indexedDB storage you import it from the [RxDB Premium π](/premium/) npm module and use `getRxStorageIndexedDB()` when creating the [RxDatabase](./rx-database.md).
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageIndexedDB({
/**
* For better performance, queries run with a batched cursor.
* You can change the batchSize to optimize the query time
* for specific queries.
* You should only change this value when
* you are also doing performance measurements.
* [default=300]
*/
batchSize: 300
})
});
```
## Overwrite/Polyfill the native IndexedDB
[Node.js](./nodejs-database.md) has no IndexedDB API. To still run the IndexedDB `RxStorage` in Node.js, for example to run unit tests, you have to polyfill it.
You can do that by using the [fake-indexeddb](https://github.com/dumbmatter/fakeIndexedDB) module and pass it to the `getRxStorageIndexedDB()` function.
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
//> npm install fake-indexeddb --save
const fakeIndexedDB = require('fake-indexeddb');
const fakeIDBKeyRange = require('fake-indexeddb/lib/FDBKeyRange');
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageIndexedDB({
indexedDB: fakeIndexedDB,
IDBKeyRange: fakeIDBKeyRange
})
});
```
## Storage Buckets
The [Storage Buckets API](https://wicg.github.io/storage-buckets/) provides a way for sites to organize locally stored data into groupings called "storage buckets". This allows the user agent or sites to manage and delete buckets independently rather than applying the same treatment to all the data from a single origin. [Read More](https://developer.chrome.com/docs/web-platform/storage-buckets?hl=en)
To use different storage buckets with the RxDB IndexedDB Storage, you can use a function instead of a plain object when providing the `indexedDB` attribute:
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageIndexedDB({
indexedDB: async(params) => {
const myStorageBucket =
await navigator.storageBuckets
.open('myApp-' + params.databaseName);
return myStorageBucket.indexedDB;
},
IDBKeyRange
})
});
```
## Limitations of the IndexedDB RxStorage
- It is part of the [RxDB Premium π](/premium/) plugin that must be purchased. If you just need a storage that works in the browser and you do not have to care about performance, you can use the [LocalStorage storage](./rx-storage-localstorage.md) instead.
- The IndexedDB storage requires support for [IndexedDB v2](https://caniuse.com/indexeddb2), it does not work on Internet Explorer.
---
## Supercharged OPFS Database with RxDB
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_DATA_OPFS, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {PremiumBlock} from '@site/src/components/premium-block';
# Origin Private File System (OPFS) Database with the RxDB OPFS-RxStorage
With the [RxDB](https://rxdb.info/) OPFS storage you can build a fully featured database on top of the [Origin Private File System](https://web.dev/opfs) (OPFS) browser API. Compared to other storage solutions, it has a way better performance.
## What is OPFS
The **Origin Private File System (OPFS)** is a native browser storage API that allows web applications to manage files in a private, sandboxed, **origin-specific virtual filesystem**. Unlike [IndexedDB](./rx-storage-indexeddb.md) and [LocalStorage](./articles/localstorage.md), which are optimized as object/key-value storage, OPFS provides more granular control for file operations, enabling byte-by-byte access, file streaming, and even low-level manipulations.
OPFS is ideal for applications requiring **high-performance** file operations (**3x-4x faster compared to IndexedDB**) inside of a client-side application, offering advantages like improved speed, more efficient use of resources, and enhanced security and privacy features.
### OPFS limitations
From the beginning of 2023, the Origin Private File System API is supported by [all modern browsers](https://caniuse.com/native-filesystem-api) like Safari, Chrome, Edge and Firefox. Only Internet Explorer is not supported and likely will never get support.
It is important to know that the most performant synchronous methods like [`read()`](https://developer.mozilla.org/en-US/docs/Web/API/FileSystemSyncAccessHandle/read) and [`write()`](https://developer.mozilla.org/en-US/docs/Web/API/FileSystemSyncAccessHandle/write) of the OPFS API are **only available inside of a [WebWorker](./rx-storage-worker.md)**.
They cannot be used in the main thread, an iFrame or even a [SharedWorker](./rx-storage-shared-worker.md).
The OPFS [`createSyncAccessHandle()`](https://developer.mozilla.org/en-US/docs/Web/API/FileSystemFileHandle/createSyncAccessHandle) method that gives you access to the synchronous methods is not exposed in the main thread, only in a Worker.
While there is no concrete **data size limit** defined by the API, browsers will refuse to store more [data at some point](./articles/indexeddb-max-storage-limit.md).
If no more data can be written, a `QuotaExceededError` is thrown which should be handled by the application, like showing an error message to the user.
## How the OPFS API works
The OPFS API is pretty straightforward to use. First you get the root filesystem. Then you can create files and directories on that. Notice that whenever you _synchronously_ write to, or read from a file, an `ArrayBuffer` must be used that contains the data. It is not possible to synchronously write plain strings or objects into the file. Therefore the `TextEncoder` and `TextDecoder` API must be used.
Also notice that some of the methods of `FileSystemSyncAccessHandle` [have been asynchronous](https://developer.chrome.com/blog/sync-methods-for-accesshandles) in the past, but are synchronous since Chromium 108. To make it less confusing, we just use `await` in front of them, so it will work in both cases.
```ts
// Access the root directory of the origin's private file system.
const root = await navigator.storage.getDirectory();
// Create a subdirectory.
const diaryDirectory = await root.getDirectoryHandle('subfolder', {
create: true,
});
// Create a new file named 'example.txt'.
const fileHandle = await diaryDirectory.getFileHandle('example.txt', {
create: true,
});
// Create a FileSystemSyncAccessHandle on the file.
const accessHandle = await fileHandle.createSyncAccessHandle();
// Write a sentence to the file.
let writeBuffer = new TextEncoder().encode('Hello from RxDB');
const writeSize = accessHandle.write(writeBuffer);
// Read file and transform data to string.
const readBuffer = new Uint8Array(writeSize);
const readSize = accessHandle.read(readBuffer, { at: 0 });
const contentAsString = new TextDecoder().decode(readBuffer);
// Write an exclamation mark to the end of the file.
writeBuffer = new TextEncoder().encode('!');
accessHandle.write(writeBuffer, { at: readSize });
// Truncate file to 10 bytes.
await accessHandle.truncate(10);
// Get the new size of the file.
const fileSize = await accessHandle.getSize();
// Persist changes to disk.
await accessHandle.flush();
// Always close FileSystemSyncAccessHandle if done, so others can open the file again.
await accessHandle.close();
```
A more detailed description of the OPFS API can be found [on MDN](https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system).
## OPFS performance
Because the Origin Private File System API provides low-level access to binary files, it is much faster compared to [IndexedDB](./slow-indexeddb.md) or [localStorage](./articles/localstorage.md). According to the [storage performance test](https://pubkey.github.io/client-side-databases/database-comparison/index.html), OPFS is up to 2x times faster on plain inserts when a new file is created on each write. Reads are even faster.
A good comparison about real world scenarios, are the [performance results](./rx-storage-performance.md) of the various RxDB storages. Here it shows that reads are up to 4x faster compared to IndexedDB, even with complex queries:
## Using OPFS as RxStorage in RxDB
The OPFS [RxStorage](./rx-storage.md) itself must run inside a WebWorker. Therefore we use the [Worker RxStorage](./rx-storage-worker.md) and let it point to the prebuild `opfs.worker.js` file that comes shipped with RxDB Premium π.
```ts
import {
createRxDatabase
} from 'rxdb';
import { getRxStorageWorker } from 'rxdb-premium/plugins/storage-worker';
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageWorker(
{
/**
* This file must be statically served from a webserver.
* You might want to first copy it somewhere outside of
* your node_modules folder.
*/
workerInput: 'node_modules/rxdb-premium/dist/workers/opfs.worker.js'
}
)
});
```
## Using OPFS in the main thread instead of a worker
The `createSyncAccessHandle()` method from the OPFS File System Access API is only available inside of a WebWorker. Therefore you cannot use `getRxStorageOPFS()` in the main thread. Instead, RxDB provides `getRxStorageOPFSMainThread()`, which uses the asynchronous OPFS APIs (such as `FileSystemFileHandle.createWritable()`) under the hood.
Using OPFS from the main thread can also simplify your application architecture by avoiding the WebWorker setup.
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageOPFSMainThread } from 'rxdb-premium/plugins/storage-opfs';
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageOPFSMainThread()
});
```
The main thread and worker variants have different performance patterns. Running the database inside a WebWorker frees up the main thread to perform other tasks and enables faster synchronous file access. This is why the worker can be noticeably faster for operations with many sequential reads, such as *Find 50 docs by ID*.
However, for many insert and bulk operations, the latency overhead of serializing queries and passing messages between the main thread and the worker will outweigh the raw storage performance gains. This means the Main Thread variant can appear faster in some benchmarks. Always test both variants to determine which performs better for your specific use case.
## Building a custom `worker.js`
When you want to run additional plugins like storage wrappers or replication **inside** of the worker, you have to build your own `worker.js` file. You can do that similar to other workers by calling `exposeWorkerRxStorage` like described in the [worker storage plugin](./rx-storage-worker.md).
```ts
// inside of the worker.js file
import { getRxStorageOPFS } from 'rxdb-premium/plugins/storage-opfs';
import { exposeWorkerRxStorage } from 'rxdb-premium/plugins/storage-worker';
const storage = getRxStorageOPFS();
exposeWorkerRxStorage({
storage
});
```
## Setting `usesRxDatabaseInWorker` when a RxDatabase is also used inside of the worker
When you use the OPFS inside of a worker, it will internally use strings to represent operation results. This has the benefit that transferring strings from the worker to the main thread, is way faster compared to complex json objects. The `getRxStorageWorker()` will automatically decode these strings on the main thread so that the data can be used by the RxDatabase.
But using a RxDatabase **inside** of your worker can make sense for example when you want to move the [replication](./replication.md) with a server. To enable this, you have to set `usesRxDatabaseInWorker` to `true`:
```ts
// inside of the worker.js file
import { getRxStorageOPFS } from 'rxdb-premium/plugins/storage-opfs';
const storage = getRxStorageOPFS({
usesRxDatabaseInWorker: true
});
```
If you forget to set this and still create and use a [RxDatabase](./rx-database.md) inside of the worker, you might get the error message "Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'length')".
## OPFS in Electron, React-Native or Capacitor.js
Origin Private File System is a browser API that is only accessible in browsers. Other JavaScript like React-Native or Node.js, do not support it.
**Electron** has two JavaScript contexts: the browser (chromium) context and the Node.js context. While you could use the OPFS API in the browser context, it is not recommended. Instead you should use the Filesystem API of Node.js and then only transfer the relevant data with the [ipcRenderer](https://www.electronjs.org/de/docs/latest/api/ipc-renderer). With RxDB that is pretty easy to configure:
- In the `main.js`, expose the [Node Filesystem](./rx-storage-filesystem-node.md) storage with the `exposeIpcMainRxStorage()` that comes with the [electron plugin](./electron.md)
- In the browser context, access the main storage with the `getRxStorageIpcRenderer()` method.
**React Native** (and Expo) does not have an OPFS API. You could use the ReactNative Filesystem to directly write data. But to get a fully featured database like RxDB it is easier to use the [SQLite RxStorage](./rx-storage-sqlite.md) which starts an SQLite database inside of the ReactNative app and uses that to do the database operations.
**Capacitor.js** is able to access the OPFS API.
## Difference between `File System Access API` and `Origin Private File System (OPFS)`
Often developers are confused with the differences between the `File System Access API` and the `Origin Private File System (OPFS)`.
- The `File System Access API` provides access to the files on the device file system, like the ones shown in the file explorer of the operating system. To use the File System API, the user has to actively select the files from a filepicker.
- `Origin Private File System (OPFS)` is a sub-part of the `File System Standard` and it only describes the things you can do with the filesystem root from `navigator.storage.getDirectory()`. OPFS writes to a **sandboxed** filesystem, not visible to the user. Therefore the user does not have to actively select or allow the data access.
## Learn more about OPFS:
- [WebKit: The File System API with Origin Private File System](https://webkit.org/blog/12257/the-file-system-access-api-with-origin-private-file-system/)
- [Browser Support](https://caniuse.com/native-filesystem-api)
- [Performance Test Tool](https://pubkey.github.io/client-side-databases/database-comparison/index.html)
---
## Lightning-Fast Memory Storage for RxDB
import {Faq, FaqItem} from '@site/src/components/faq';
# Memory RxStorage
{/* keywords:
javascript in-memory database
in memory db
node js in memory database
in memory storage
Nestjs in-memory database
*/}
The Memory [RxStorage](./rx-storage.md) is based on plain in-memory arrays and objects. It can be used in all environments and is made for performance. By storing data directly in RAM, it eliminates disk I/O bottlenecks and operates faster than traditional disk-based databases.
You should use this storage when you need a fast database configuration, such as in unit tests, server-side rendering, or high-throughput data processing.
## How it achieves maximum speed
- **No Disk I/O**: Operations happen entirely in RAM. There is no waiting for disk reads or writes.
- **No Serialization Overhead**: Data remains as JavaScript objects and arrays. It skips the expensive JSON serialization and deserialization steps required by index-based or file-based storages.
- **Binary Search**: It uses pure JavaScript arrays and binary search algorithms on all database operations, ensuring fast queries and index traversals.
- **Small Build Size**: The plugin contains minimal code, keeping your bundle size small.
## Use Cases
### 1. Unit Testing and CI/CD
The Memory storage is the recommended storage for testing RxDB applications. It provides two major benefits: speed and isolation. Because it keeps data only in memory, each test run can start with a clean state without needing to clean up leftover filesystem states or deleting IndexedDB databases.
You can also simulate multi-tab behavior inside a single Node.js process by creating multiple `RxDatabase` instances with the same name and the `ignoreDuplicate: true` setting. They will share the memory state and communicate with each other naturally.
### 2. Server-Side Rendering (SSR)
When rendering React, Vue, or Angular applications on the server, you often need to fetch data, populate a database state, and render the UI. Using the Memory storage ensures your server handles these requests quickly without touching the file system, reducing latency and avoiding disk write locks.
### 3. Caching and Real-Time Processing
For applications handling thousands of events per second, such as real-time analytics dashboards or temporary chat state, the Memory storage acts as a fast data layer. You achieve instantaneous data access for aggregations and queries.
### 4. Memory-Mapped Performance Upgrades
RxDB provides a [Memory-Mapped RxStorage](./rx-storage-memory-mapped.md) which uses the Memory storage as a fast, primary layer and replicates data to a slower persistence storage in the background. This improves initial page load and query times while still keeping data safe on disk.
## Implementation
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageMemory
} from 'rxdb/plugins/storage-memory';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageMemory()
});
```
### Constraints
- **No Persistence**: All data is lost when the JavaScript process exits or the browser tab is closed.
- **Memory Limits**: The dataset is constrained by the available RAM in the JavaScript runtime environment.
## FAQ
The fastest scalable in-memory databases skip expensive disk I/O bindings and bypass JSON serialization bottlenecks by storing data strictly within standard JavaScript V8 variables. **[RxDB](./rx-database.md)**'s Memory Storage plugin utilizes pure algorithmic binary-search indexing over raw array references, offering instantaneous throughput. This makes it an unparalleled choice for Node.js environments processing real-time analytics, rapid CI/CD Server-Side Rendering (SSR) pipelines, or highly volatile chat application states.
---
## Blazing-Fast Node Filesystem Storage
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_NODE, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {PremiumBlock} from '@site/src/components/premium-block';
import {Faq, FaqItem} from '@site/src/components/faq';
# Filesystem Node RxStorage
The Filesystem Node [RxStorage](./rx-storage.md) for RxDB is built on top of the [Node.js Filesystem API](https://nodejs.org/api/fs.html).
It stores data in plain JSON/txt files like any "normal" database does. It is a bit faster compared to the [SQLite storage](./rx-storage-sqlite.md) and its setup is less complex.
Using the same database folder in parallel with multiple Node.js processes is supported when you set `multiInstance: true` while creating the [RxDatabase](./rx-database.md).
### Pros
- Easier setup compared to [SQLite](./rx-storage-sqlite.md)
- [Fast](./rx-storage-performance.md)
## Usage
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageFilesystemNode
} from 'rxdb-premium/plugins/storage-filesystem-node';
import path from 'path';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageFilesystemNode({
basePath: path.join(__dirname, 'my-database-folder'),
/**
* Set inWorker=true if you use this RxStorage
* together with the WebWorker plugin.
*/
inWorker: false
})
});
/* ... */
```
## FAQ
The native `getRxStorageFilesystemNode` adapter does not compile documents into a single monolithic file (like SQLite), but instead serializes and persists document data as distinct JSON/text files directly representing the database tree on the disk. For strict single-file architectures in Node.js, you must mount the specialized **[SQLite RxStorage](./rx-storage-sqlite.md)** plugin, which wraps the entire database state into a single portable `.sqlite` file efficiently using Node's native `sqlite` bindings.
---
## Expo Filesystem RxStorage for React Native
import {BetaBlock} from '@site/src/components/beta-block';
import {PremiumBlock} from '@site/src/components/premium-block';
import {Steps} from '@site/src/components/steps';
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_EXPO, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
# Expo Filesystem RxStorage
The Expo Filesystem [RxStorage](./rx-storage.md) for RxDB is built on top of the [expo-file-system](https://docs.expo.dev/versions/latest/sdk/filesystem/) library, bringing blazing-fast direct filesystem capabilities to React Native and Expo applications.
It stores data in plain files and achieves vastly superior performance compared to traditional React Native storage solutions like Async Storage or SQLite.
### Pros
- **Extreme Performance**: Significantly faster than SQLite and Async Storage in React Native.
- **Easy Integration**: Drops right into any Expo or React Native project.
- Directly uses the Expo FileSystem for minimum overhead without relying on an intermediate database engine.
## Installation
> **Note:** This storage plugin requires at least **Expo SDK 54 (or newer)** or the equivalent React Native `expo-file-system` version to function.
### Install expo-file-system
First, you need to install the `expo-file-system` dependency:
```bash
npx expo install expo-file-system
```
### Install expo-opfs
You also have to install the `expo-opfs` peer dependency:
```bash
npx expo install expo-opfs
```
## Usage
You can import either the **Asynchronous** or **Synchronous** storage from the `rxdb-premium` package.
### Asynchronous API
For standard usage in React Native and Expo, use the asynchronous storage plugin:
```ts
import { createRxDatabase } from 'rxdb';
import {
getRxStorageExpoAsync
} from 'rxdb-premium/plugins/storage-filesystem-expo';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageExpoAsync(),
// Usually false in React Native as there is only one JavaScript process
multiInstance: false
});
/* ... */
```
### Synchronous API
Because the expo filesystem also has a sync API, you can use the sync storage which has faster writes but slower reads.
```ts
import { createRxDatabase } from 'rxdb';
import {
getRxStorageExpoSync
} from 'rxdb-premium/plugins/storage-filesystem-expo';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageExpoSync(),
multiInstance: false
});
/* ... */
```
## How it works (vs. SQLite)
When using SQLite in React Native, every read and write operation has to go through multiple stages: the JavaScript query must be sent to the native side, translated into a SQL string, parsed and planned by the SQLite engine, and then finally executed on disk. For data retrieval, the SQLite rows must then be mapped back into JavaScript objects. This overhead can be significant, especially when handling many operations or large batches of documents.
In contrast, the **Expo Filesystem RxStorage** skips the relational SQL database engine entirely and instead runs on RxDB's own highly-optimized NoSQL storage engine. While document data is efficiently stored via raw file read and write operations using `expo-file-system`, the storage maintains proper indexing and an advanced query engine directly in JavaScript. Because there is no SQL parsing, complex native query planning, or relational mapping involved, this makes reading, writing, and querying data significantly faster. It operates closer to the hardware and handles bulk document serialization dynamically using highly optimized UTF-8 decoding.
## Performance of Expo Filesystem vs SQLite
Because of this streamlined, direct-to-filesystem approach, operations like inserting large batches of documents or executing complex queries are handled with minimal overhead. This makes it one of the absolute fastest local storage engines available for React Native.
Here is a performance comparison of the **Expo Filesystem RxStorage** compared to the **[SQLite RxStorage](./rx-storage-sqlite.md)** (using `expo-sqlite`), tested with RxDB's internal performance testing suite (3000 documents, 4 collections):
## Using with Plain React Native (Bare Workflow)
You can also use this storage in a plain React Native project that does not use the Expo framework. To use Expo modules in a bare React Native app, you must first install the `expo` package to provide the underlying infrastructure:
```bash
npx install-expo-modules@latest
npx expo install expo-file-system
```
*(Note for iOS: You may need to run `npx pod-install` after installation so the native dependencies are linked correctly)*
### Permissions
The `expo-file-system` module requires certain permissions on Android to interact with the filesystem.
**For Expo Projects:**
Installing the module will automatically add the required permissions during the build process. You do not need to configure these manually.
**For Plain React Native (Bare Workflow) Projects:**
You must manually add the following permissions to your `android/app/src/main/AndroidManifest.xml`:
```xml
```
On iOS, no additional permissions or setup are necessary for standard filesystem access.
## FAQ
In React Native, SQLite suffers from translation overhead. Every operation requires sending JavaScript queries to the native side, translating them into SQL statements, running the native query planner, and mapping the relational rows back into JavaScript objects. For bulk operations, this parsing and mapping causes noticeable performance degradation.
A NoSQL document store that avoids relational SQL overhead provides the fastest performance. RxDB paired with the Expo Filesystem RxStorage skips the SQLite engine entirely. It stores documents directly as plain JSON text appended to files, resulting in superior read and write speeds.
You can optimize queries by using proper indexing to prevent full database scans. Switching from a relational SQL database to a local-first NoSQL database that queries directly against raw file data removes the translation steps between JavaScript objects and the storage layer, reducing query resolution times.
You should use a storage engine that supports fast bulk writes. The Expo Filesystem RxStorage manages large batches of documents by serializing them dynamically using highly optimized UTF-8 decoding, writing the entire batch directly to the filesystem in one continuous operation rather than parsing individual SQL insert statements.
Use an [offline-first](./offline-first.md) database like RxDB to maintain a persistent local replica of your data on the device filesystem. Read operations resolve instantly from the local cache, while background [replication](./replication.md) synchronizes data modifications with your remote server asynchronously.
---
## RxDB SQLite RxStorage for Hybrid Apps
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_NODE, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {Steps} from '@site/src/components/steps';
import {Tabs} from '@site/src/components/tabs';
import {Faq, FaqItem} from '@site/src/components/faq';
# SQLite RxStorage
This [RxStorage](./rx-storage.md) is based on [SQLite](https://www.sqlite.org/index.html) and is made to work with **Node.js**, [Electron](./electron-database.md), [React Native](./react-native-database.md) and [Capacitor](./capacitor-database.md) or SQLite via webassembly in the browser. It can be used with different so called `sqliteBasics` adapters to account for the differences in the various SQLite bundles and libraries that exist.
SQLite is a natural fit for RxDB because most platforms - Android, iOS, Node.js, and beyond - already ship with a built-in SQLite engine, delivering robust performance and minimal setup overhead. Its proven reliability, having powered countless applications over the years, ensures a battle-tested foundation for local data. By placing RxDB on top of SQLite, you gain advanced features suited for building interactive, [offline-capable](./offline-first.md) UI apps: [real-time queries](./rx-query.md#observe), reactive state updates, [conflict handling](./transactions-conflicts-revisions.md), [data encryption](./encryption.md), and straightforward [schema management](./rx-schema.md). This combination offers a unified NoSQL-like experience without sacrificing the speed and broad availability that SQLite brings.
## Performance comparison with other storages
The SQLite storage is a bit slower compared to other Node.js based storages like the [Filesystem Storage](./rx-storage-filesystem-node.md) because wrapping SQLite has a bit of overhead and sending data from the JavaScript process to SQLite and backwards increases the latency. However for most hybrid apps the SQLite storage is the best option because it can leverage the SQLite version that comes already installed on the smartphone's OS (iOS and android). Also for desktop Electron apps it can be a viable solution because it is easy to ship SQLite together inside of the Electron bundle.
## Using the SQLite RxStorage
There are two versions of the SQLite storage available for RxDB:
- The **trial version** which comes directly shipped with RxDB Core. It contains an SQLite storage that allows you to try out RxDB on devices that support SQLite, like React Native or Electron. While the trial version does pass the full RxDB storage test-suite, it is not made for production. It is not using indexes, has no [attachment support](./rx-attachment.md), is limited to store 500 non-deleted documents and fetches the whole storage state to run queries in memory. **Use it for evaluation and prototypes only!**
- The **[RxDB Premium π](/premium/) version** which contains the full production-ready SQLite storage. It contains a full load of performance optimizations and full query support. To use the SQLite storage you have to import `getRxStorageSQLite` from the [RxDB Premium π](/premium/) package and then add the correct `sqliteBasics` adapter depending on which sqlite module you want to use. This can then be used as storage when creating the [RxDatabase](./rx-database.md). In the following you can see some examples for some of the most common SQLite packages.
## Trial Version
```ts
// Import the Trial SQLite Storage
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsNodeNative
} from 'rxdb/plugins/storage-sqlite';
// Create a Storage for it, here we use the nodejs-native SQLite module
// other SQLite modules can be used with a different sqliteBasics adapter
import { DatabaseSync } from 'node:sqlite';
const storage = getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
});
// Create a Database with the Storage
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: storage
});
```
## RxDB Premium π
```ts
// Import the SQLite Storage from the premium plugins.
import {
getRxStorageSQLite,
getSQLiteBasicsNodeNative
} from 'rxdb-premium/plugins/storage-sqlite';
// Create a Storage for it, here we use the nodejs-native SQLite module
// other SQLite modules can be used with a different sqliteBasics adapter
import { DatabaseSync } from 'node:sqlite';
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
});
// Create a Database with the Storage
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: storage
});
```
In the following, all examples are shown with the premium SQLite storage. Still they work the same with the trial version.
## SQLiteBasics
Different SQLite libraries have different APIs to create and access the SQLite database. Therefore the library must be massaged to work with the RxDB SQlite storage. This is done in a so-called `SQLiteBasics` interface. RxDB directly ships with a wide range of these for various SQLite libraries that are commonly used. Also creating your own one is pretty simple, check the source code of the existing ones for that.
For example for the `sqlite3` npm library we have the `getSQLiteBasicsNode()` implementation. For `node:sqlite` we have the `getSQLiteBasicsNodeNative()` implementation and so on..
## Using the SQLite RxStorage with different SQLite libraries
### Usage with the **sqlite3 npm package**
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsNode
} from 'rxdb-premium/plugins/storage-sqlite';
/**
* In Node.js, we use the SQLite database
* from the 'sqlite' npm module.
* @link https://www.npmjs.com/package/sqlite3
*/
import sqlite3 from 'sqlite3';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLite({
/**
* Different runtimes have different interfaces to SQLite.
* For example in node.js we have a callback API,
* while in capacitor sqlite we have Promises.
* So we need a helper object that is capable of doing the basic
* sqlite operations.
*/
sqliteBasics: getSQLiteBasicsNode(sqlite3)
})
});
```
### Usage with the **node:sqlite** package
With Node.js version 22 and newer, you can use the "native" [sqlite module](https://nodejs.org/api/sqlite.html) that comes shipped with Node.js.
```ts
import { createRxDatabase } from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsNodeNative
} from 'rxdb-premium/plugins/storage-sqlite';
import { DatabaseSync } from 'node:sqlite';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
})
});
```
### Usage with Webassembly in the Browser
In the browser you can use the [wa-sqlite](https://github.com/rhashimoto/wa-sqlite) package to run SQLite in Webassembly. The wa-sqlite module also allows using persistence with IndexedDB or OPFS. Notice that in general SQLite via Webassembly is slower compared to other storages like [IndexedDB](./rx-storage-indexeddb.md) or [OPFS](./rx-storage-opfs.md) because sending data from the main thread to wasm and backwards is slow in the browser. Have a look at the [performance comparison](./rx-storage-performance.md).
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsWasm
} from 'rxdb-premium/plugins/storage-sqlite';
/**
* In the Browser, we use the SQLite database
* from the 'wa-sqlite' npm module. This contains the SQLite library
* compiled to Webassembly
* @link https://www.npmjs.com/package/wa-sqlite
*/
import SQLiteESMFactory from 'wa-sqlite/dist/wa-sqlite-async.mjs';
import SQLite from 'wa-sqlite';
const sqliteModule = await SQLiteESMFactory();
const sqlite3 = SQLite.Factory(module);
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsWasm(sqlite3)
})
});
```
### Usage with **React Native**
#### 1. Install the package
Install the [react-native-quick-sqlite npm module](https://www.npmjs.com/package/react-native-quick-sqlite)
#### 2. Create the Database
Import `getSQLiteBasicsQuickSQLite` from the SQLite plugin and use it to create a [RxDatabase](./rx-database.md):
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsQuickSQLite
} from 'rxdb-premium/plugins/storage-sqlite';
import { open } from 'react-native-quick-sqlite';
// create database
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
// Set multiInstance to false for React Native
multiInstance: false,
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsQuickSQLite(open)
})
});
```
If `react-native-quick-sqlite` does not work for you, as alternative you can use the [react-native-sqlite-2](https://www.npmjs.com/package/react-native-sqlite-2) library instead:
```ts
import {
getRxStorageSQLite,
getSQLiteBasicsWebSQL
} from 'rxdb-premium/plugins/storage-sqlite';
import SQLite from 'react-native-sqlite-2';
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsWebSQL(SQLite.openDatabase)
});
```
### Usage with **Expo SQLite**
:::info
For Expo apps, the **[Expo Filesystem RxStorage](./rx-storage-filesystem-expo.md)** exists and has significantly better performance compared to SQLite.
:::
Notice that [expo-sqlite](https://www.npmjs.com/package/expo-sqlite) cannot be used on android (but it works on iOS) if you use Expo SDK version 50 or older. Please update to Version 50 or newer to use it.
In the latest expo SDK version, use the `getSQLiteBasicsExpoSQLiteAsync()` method:
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsExpoSQLiteAsync
} from 'rxdb-premium/plugins/storage-sqlite';
import * as SQLite from 'expo-sqlite';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
multiInstance: false,
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsExpoSQLiteAsync(SQLite.openDatabaseAsync)
})
});
```
In older Expo SDK versions, you might have to use the non-async API:
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsExpoSQLite
} from 'rxdb-premium/plugins/storage-sqlite';
import { openDatabase } from 'expo-sqlite';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
multiInstance: false,
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsExpoSQLite(openDatabase)
})
});
```
### Usage with **SQLite Capacitor**
#### 1. Install the sqlite capacitor npm module
Install the [sqlite capacitor npm module](https://github.com/capacitor-community/sqlite)
#### 2. Add the iOS database location
Add the iOS database location to your capacitor config
```json
{
"plugins": {
"CapacitorSQLite": {
"iosDatabaseLocation": "Library/CapacitorDatabase"
}
}
}
```
#### 3. Get the capacitor sqlite wrapper
Use the function `getSQLiteBasicsCapacitor` to get the capacitor sqlite wrapper.
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsCapacitor
} from 'rxdb-premium/plugins/storage-sqlite';
/**
* Import SQLite from the capacitor plugin.
*/
import {
CapacitorSQLite,
SQLiteConnection
} from '@capacitor-community/sqlite';
import { Capacitor } from '@capacitor/core';
const sqlite = new SQLiteConnection(CapacitorSQLite);
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLite({
/**
* Different runtimes have different interfaces to SQLite.
* For example in node.js we have a callback API,
* while in capacitor sqlite we have Promises.
* So we need a helper object that is capable of doing the basic
* sqlite operations.
*/
sqliteBasics: getSQLiteBasicsCapacitor(sqlite, Capacitor)
})
});
```
### Usage with Tauri SQLite
#### 1. Add the Tauri SQL plugin
Add the [Tauri SQL plugin](https://tauri.app/plugin/sql/#setup) to your Tauri project.
#### 2. Add sqlite as your database engine
Make sure to add `sqlite` as your database engine by running `cargo add tauri-plugin-sql --features sqlite` inside `src-tauri`.
#### 3. Use the Tauri SQLite wrapper
Use the `getSQLiteBasicsTauri` function to get the Tauri SQLite wrapper.
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsTauri
} from 'rxdb-premium/plugins/storage-sqlite';
import sqlite3 from '@tauri-apps/plugin-sql';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsTauri(sqlite3)
})
});
```
## Database Connection
If you need to access the database connection for any reason you can use `getDatabaseConnection` to do so:
```ts
import { getDatabaseConnection } from 'rxdb-premium/plugins/storage-sqlite'
```
It has the following signature:
```ts
getDatabaseConnection(
sqliteBasics: SQLiteBasics,
databaseName: string
): Promise;
```
## Known Problems of SQLite in JavaScript apps
- Some JavaScript runtimes do not contain a `Buffer` API which is used by SQLite to store binary attachments data as `BLOB`. You can set `storeAttachmentsAsBase64String: true` if you want to store the attachments data as base64 string instead. This increases the database size but makes it work even without having a `Buffer`.
- The SQlite RxStorage works on SQLite libraries that use SQLite in version `3.38.0 (2022-02-22)` or newer, because it uses the [SQLite JSON](https://www.sqlite.org/json1.html) methods like `JSON_EXTRACT`. If you get an error like `[Error: no such function: JSON_EXTRACT (code 1 SQLITE_ERROR[1])`, you might have a too old version of SQLite.
- To debug all SQL operations, you can pass a log function to `getRxStorageSQLite()` like this. This does not work with the trial version:
```ts
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsCapacitor(sqlite, Capacitor),
// pass log function
log: console.log.bind(console)
});
```
- By default, all tables will be created with the `WITHOUT ROWID` flag. Some tools like drizzle do not support tables with that option. You can disable it by setting `withoutRowId: false` when calling `getRxStorageSQLite()`:
```ts
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsCapacitor(sqlite, Capacitor),
withoutRowId: false
});
```
## FAQ
Yes, starting natively from version `3.38.0`, SQLite includes comprehensive built-in core JSON functions like `JSON_EXTRACT`. The **[RxDB SQLite Storage](./rx-storage.md)** engine utilizes these exact JSON extension methods to seamlessly run complex NoSQL document queries, indexes, and sorting operations directly within the SQLite runtime, bridging the gap between flat tabular paradigms and rich document store flexibility.
You can save and export an active SQLite database by closing the connection and copying its physical `.sqlite` storage file traversing the underlying OS filesystem. If you are operating within a strict sandboxed web environment using WebAssembly, you must extract the SQLite file via exactly matching the `wa-sqlite` export streams, or rely on **[RxDB](./rx-database.md)** JSON export plugins to seamlessly migrate data out of local constraints into raw JSON streams regardless of the active SQLite engine.
## Related
- [React Native Databases](./react-native-database.md)
---
## RxDB Dexie.js Database - Fast, Reactive, Sync with Any Backend
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {Steps} from '@site/src/components/steps';
import {Faq, FaqItem} from '@site/src/components/faq';
# RxStorage Dexie.js
To store the data inside of and [RxDB Database](./rx-database.md) in IndexedDB in the [browser](./articles/browser-database.md), you can use the [Dexie.js](https://github.com/dexie/Dexie.js) based [RxStorage](./rx-storage.md). Dexie.js is a minimal wrapper around IndexedDB and the Dexie.js RxStorage wraps that again to use it for an RxDB database in the browser. For side projects and prototypes that run in a browser, you should use the dexie RxStorage as a default.
## Dexie.js vs IndexedDB Storage
While Dexie.js [RxStorage](./rx-storage.md) can be used for free, most professional projects should switch to our **premium [IndexedDB RxStorage](./rx-storage-indexeddb.md) π** in production:
- It is faster and reduces build size by up to **36%**.
- It has a way [better performance](./rx-storage-performance.md) on reads and writes.
- It stores [attachments](./rx-attachment.md) data as binary instead of base64 which reduces used space by 33%.
- It does not use a [Batched Cursor](./slow-indexeddb.md#batched-cursor) or [custom indexes](./slow-indexeddb.md#custom-indexes) which makes queries slower compared to the [IndexedDB RxStorage](./rx-storage-indexeddb.md).
- It supports **non-required indexes** which is [not possible](https://github.com/pubkey/rxdb/pull/6643#issuecomment-2505310082) with Dexie.js.
- It runs in a **WAL-like mode** (similar to SQLite) for faster writes and improved responsiveness.
- It support the [Storage Buckets API](./rx-storage-indexeddb.md#storage-buckets)
## How to use Dexie.js as a Storage for RxDB
### Import the Dexie Storage
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
```
### Create a Database
```ts
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageDexie()
});
```
## Overwrite/Polyfill the native IndexedDB API with an in-memory version
Node.js has no IndexedDB API. To still run the Dexie `RxStorage` in Node.js, for example to run unit tests, you have to polyfill it.
You can do that by using the [fake-indexeddb](https://github.com/dumbmatter/fakeIndexedDB) module and pass it to the `getRxStorageDexie()` function.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
//> npm install fake-indexeddb --save
const fakeIndexedDB = require('fake-indexeddb');
const fakeIDBKeyRange = require('fake-indexeddb/lib/FDBKeyRange');
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageDexie({
indexedDB: fakeIndexedDB,
IDBKeyRange: fakeIDBKeyRange
})
});
```
## Using Dexie Addons
Dexie.js has its own plugin system with [many plugins](https://dexie.org/docs/DerivedWork#known-addons) for [encryption](./encryption.md), replication or other use cases. With the Dexie.js `RxStorage` you can use the same plugins by passing them to the `getRxStorageDexie()` function.
```ts
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageDexie({
addons: [ /* Your Dexie.js plugins */ ]
})
});
```
## Sync Dexie.js with your Backend in RxDB
Having your local data in sync with a remote backend is a key feature of RxDB. Here are two approaches to achieve this when using the Dexie.js RxStorage:
* **Dexie Cloud** provides a **managed solution**: For quick setups, letting you rely on its Cloud backend and conflict resolution.
* [RxDB's replication](./replication.md): Offers **full control** over your backend, data flow, and [conflict handling](./transactions-conflicts-revisions.md).
Choose the approach that best suits your needs - whether you want to get started quickly with Dexie Cloud or require the adaptability and autonomy of RxDB's native replication.
### A. Use Dexie Cloud Sync
**Dexie Cloud** is an official SaaS solution provided by the Dexie team. It offers automatic synchronization, user management, and conflict resolution out of the box. The primary benefits are:
- **Automatic Sync**: Dexie Cloud keeps your local IndexedDB in sync with its cloud-based backend.
- **User Authentication**: Built-in user management (auth, roles, permissions).
- **Conflict Resolution**: Automated resolution logic on the server side.
#### Install the Dexie Cloud Addon
```bash
npm install dexie-cloud-addon
```
#### Import RxDB and dexie-cloud
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import dexieCloud from 'dexie-cloud-addon';
```
#### Create a Dexie based RxStorage with the Cloud Plugin
```ts
const storage = getRxStorageDexie({
addons: [dexieCloud],
/*
* Whenever a new dexie database instance is created,
* this method will be called.
*/
async onCreate(dexieDatabase, dexieDatabaseName) {
await dexieDatabase.cloud.configure({
databaseUrl: "https://.dexie.cloud",
requireAuth: true // optional
});
}
});
```
#### Create an RxDB Database
```ts
const db = await createRxDatabase({
name: 'mydb',
storage
});
```
### B. Use the RxDB Replication
For **full flexibility** over your backend or conflict resolution strategy, you can use one of **RxDB's many replication plugins** like
- [CouchDB Replication](./replication-couchdb.md) Plugin: Replicate with a CouchDB Server
- [GraphQL Replication](./replication-graphql.md) Plugin: Sync data with any GraphQL endpoint. Useful when you have a custom schema or you want to utilize GraphQL's powerful query features.
- [Custom Replication with REST APIs](./replication-http.md): Implement your own replication by building a pull/push handler that communicates with any RESTful backend.
Below is an example of replicating an RxDB collection with a CouchDB backend using RxDB's CouchDB replication plugin:
#### Import the RxDB with dexie and the CouchDB plugin
```ts
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import { createRxDatabase } from 'rxdb/plugins/core';
```
#### Create an RxDB Database with the Dexie Storage
```ts
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageDexie()
});
```
#### Add a Collection
```ts
await db.addCollections({
humans: {
schema: {
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
age: { type: 'number' }
},
required: ['id', 'name']
}
}
});
```
#### Sync the Collection with a CouchDB Server
```ts
const replicationState = replicateCouchDB({
replicationIdentifier: 'my-couchdb-replication',
collection: db.humans,
// The URL to your CouchDB endpoint
url: 'http://example.com/db/humans'
});
```
## liveQuery - Realtime Queries
Dexie.js offers a feature called `liveQuery` which automatically updates query results as data changes, allowing you to react to these changes in real-time. However, because RxDB intrinsically provides [reactive queries](./rx-query.md#observe), you typically do **not** need to enable live queries through Dexie. Once you have created your database and collections with RxDB, any query you perform can be observed by subscribing to it, for example via `collection.find().$.subscribe(results => { /*... */ })`. This means RxDB takes care of listening for changes and automatically emitting new results - ensuring your UI stays in sync with the underlying data without requiring extra plugins or manual polling.
## Disabling the non-premium console log
We want to be transparent with our community, and you'll notice a console message when using the free Dexie.js based RxStorage implementation. This message serves to inform you about the availability of faster storage solutions within our [π Premium Plugins](/premium/). We understand that this might be a minor inconvenience, and we sincerely apologize for that. However, maintaining and improving RxDB requires substantial resources, and our premium users help us ensure its sustainability. If you find value in RxDB and wish to remove this message, we encourage you to explore our premium storage options, which are optimized for professional use and production environments. Thank you for your understanding and support.
If you already have premium access and want to use the Dexie.js [RxStorage](./rx-storage.md) without the log, you can call the `setPremiumFlag()` function to disable the log.
```js
import { setPremiumFlag } from 'rxdb-premium/plugins/shared';
setPremiumFlag();
```
## Performance comparison with other RxStorage plugins
The performance of the Dexie.js RxStorage is good enough for most use cases but other storages can have way better performance metrics:
## FAQ
Dexie.js is a minimalist, Promise-based wrapper engineered specifically to resolve the notoriously complex callback-driven API of standard IndexedDB. It offers significant advantages over raw IndexedDB by providing an intuitive chainable query API, a far simpler database schema definition process, and extremely robust transaction management. However, Dexie lacks advanced querying capabilities found in Document-oriented NoSQL databases, such as deep-nested JSON querying and comprehensive MongoDB-style selectors. **[RxDB](./rx-database.md)**, which can use Dexie securely as its underlying storage engine, compensates for these limitations by providing a fully reactive advanced NoSQL query engine, robust cross-platform offline replication protocols, and built-in field encryption features that Dexie inherently lacks.
---
## Unlock MongoDB Power with RxDB
import {Steps} from '@site/src/components/steps';
# MongoDB RxStorage
RxDB MongoDB RxStorage is an RxDB [RxStorage](./rx-storage.md) that allows you to use [MongoDB](https://www.mongodb.com/) as the underlying storage engine for your RxDB database. With this you can take advantage of MongoDB's features and scalability while benefiting from RxDB's real-time data synchronization capabilities.
The storage is made to work with any plain MongoDB Server, [MongoDB Replica Set](https://www.mongodb.com/docs/manual/tutorial/deploy-replica-set/), [Sharded MongoDB Cluster](https://www.mongodb.com/docs/manual/sharding/) or [Atlas Cloud Database](https://www.mongodb.com/atlas/database).
## Limitations of the MongoDB RxStorage
- Multiple Node.js servers using the same MongoDB database is currently not supported
- [RxAttachments](./rx-attachment.md) are currently not supported
- Doing non-RxDB writes on the MongoDB database is not supported. RxDB expects all writes to come from RxDB which update the required metadata. Doing non-RxDB writes can confuse the RxDatabase and lead to undefined behavior. But you can perform read-queries on the MongoDB storage from the outside at any time.
## Using the MongoDB RxStorage
### Install the mongodb package
```bash
npm install mongodb --save
```
### Setups the MongoDB RxStorage
To use the storage, you simply import the `getRxStorageMongoDB` method and use that when creating the [RxDatabase](./rx-database.md). The `connection` parameter contains the [MongoDB connection string](https://www.mongodb.com/docs/manual/reference/connection-string/).
```ts
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageMongoDB
} from 'rxdb/plugins/storage-mongodb';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageMongoDB({
/**
* MongoDB connection string
* @link https://www.mongodb.com/docs/manual/reference/connection-string/
*/
connection: 'mongodb://localhost:27017,localhost:27018,localhost:27019'
})
});
```
---
## DenoKV RxStorage
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB Database on top of Deno Key Value Store
With the DenoKV [RxStorage](./rx-storage.md) layer for [RxDB](https://rxdb.info), you can run a fully featured **NoSQL database** on top of the [DenoKV API](https://docs.deno.com/kv/manual).
This gives you the benefits and features of the RxDB JavaScript Database, combined with the global availability and distribution features of the DenoKV.
## What is DenoKV
[DenoKV](https://deno.com/kv) is a strongly consistent key-value storage, globally replicated for low-latency reads across 35 worldwide regions via [Deno Deploy](https://deno.com/deploy).
When you release your Deno application on Deno Deploy, it will start a instance on each of the [35 worldwide regions](https://docs.deno.com/deploy/manual/regions). This edge deployment guarantees minimal latency when serving requests to end users devices around the world. DenoKV is a shared storage which shares its state across all instances.
But, because DenoKV is "only" a **Key-Value storage**, it only supports basic CRUD operations on datasets and indexes. Complex features like queries, [encryption](./encryption.md), compression or client-server replication, are missing. Using RxDB on top of DenoKV fills this gap and makes it easy to build realtime [offline-first](./offline-first.md) application on top of Deno backend.
## Use cases
Using RxDB-DenoKV instead of plain DenoKV, can have a wide range of benefits depending on your use case.
- **Reduce vendor lock-in**: RxDB has a swappable [storage layer](./rx-storage.md) which allows you to swap out the underlying storage of your database. If you ever decide to move away from DenoDeploy or Deno at all, you do not have to refactor your whole application and instead just **swap the storage plugin**. For example if you decide migrate to Node.js, you can use the [FoundationDB RxStorage](./rx-storage-foundationdb.md) and store your data there. DenoKV is also implemented on top of FoundationDB so you can get similar performance. Alternatively RxDB supports a wide range of [storage plugins](./rx-storage.md) you can decide from.
- **Add reactiveness**: DenoKV is a plain request-response datastore. While it supports observation of single rows by id, it does not allow to observe row-ranges or events. This makes it hard to impossible to build realtime applications with it because polling would be the only way to watch ranges of key-value pairs. With RxDB on top of DenoKV, changes to the database are **shared between DenoDeploy instances** so when you **observe a [query](./rx-query.md)** you can be sure that it is always up to date, no matter which instance has changed the document. Internally RxDB uses the [Deno BroadcastChannel API](https://docs.deno.com/deploy/api/runtime-broadcast-channel) to share events between instances.
- **Reuse Client and Server Code**: When you use RxDB on the server and on the client side, many parts of your code can be reused on both sides which decreases development time significantly.
- **Replicate from DenoKV to a local RxDB state**: Instead of running all operations against the global DenoKV, you can run a [realtime-replication](./replication.md) between a DenoKV-RxDatabase and a [locally stored dataset](./rx-storage-filesystem-node.md) or maybe even an [in-memory](./rx-storage-memory.md) stored one. This improves **query performance** and can **reduce your Deno Deploy cloud costs** because less operations run against the DenoKV, they run only locally instead.
- **Replicate with other backends**: The RxDB [Sync Engine](./replication.md) is pretty simple and allows you to easily build a replication with any backend architecture. For example if you already have your data stored in a self-hosted MySQL server, you can use RxDB to do a realtime replication of that data into a DenoKV RxDatabase instance. RxDB also has many plugins for replication with backend/protocols like [GraphQL](./replication-graphql.md), [Websocket](./replication-websocket.md), [CouchDB](./replication-couchdb.md), [WebRTC](./replication-webrtc.md), [Firestore](./replication-firestore.md) and [NATS](./replication-nats.md).
## Using the DenoKV RxStorage
To use the DenoKV RxStorage with RxDB, you import the `getRxStorageDenoKV` function from the plugin and set it as storage when calling [createRxDatabase](./rx-database.md#creation)
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageDenoKV } from 'rxdb/plugins/storage-denokv';
const myRxDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageDenoKV({
/**
* Consistency level, either 'strong' or 'eventual'
* (Optional) default='strong'
*/
consistencyLevel: 'strong',
/**
* Path which is used in the first argument of Deno.openKv(settings.openKvPath)
* (Optional) default=''
*/
openKvPath: './foobar',
/**
* Some operations have to run in batches,
* you can test different batch sizes to improve performance.
* (Optional) default=100
*/
batchSize: 100
})
});
```
On top of that [RxDatabase](./rx-database.md) you can then create your collections and run operations. Follow the [quickstart](./quickstart.md) to learn more about how to use RxDB.
## Using non-DenoKV storages in Deno
When you use other storages than the DenoKV storage inside of a Deno app, make sure you set `multiInstance: false` when creating the database. Also you should only run one process per Deno-Deploy instance. This ensures your events are not mixed up by the [BroadcastChannel](https://docs.deno.com/deploy/api/runtime-broadcast-channel) across instances which would lead to wrong behavior.
```ts
// DenoKV based database
const db = await createRxDatabase({
name: 'denokvdatabase',
storage: getRxStorageDenoKV(),
/**
* Use multiInstance: true so that the Deno Broadcast Channel
* emits event across DenoDeploy instances
* (true is also the default, so you can skip this setting)
*/
multiInstance: true
});
// Non-DenoKV based database
const db = await createRxDatabase({
name: 'denokvdatabase',
storage: getRxStorageFilesystemNode(),
/**
* Use multiInstance: false so that it does not share events
* across instances because the stored data is anyway not shared
* between them.
*/
multiInstance: false
});
```
---
## RxDB on FoundationDB - Performance at Scale
# RxDB Database on top of FoundationDB
[FoundationDB](https://www.foundationdb.org/) is a distributed key-value store designed to handle large volumes of structured data across clusters of computers while maintaining high levels of performance, scalability, and fault tolerance. While FoundationDB itself only can store and query key-value pairs, it lacks more advanced features like complex queries, [encryption](./encryption.md) and [replication](./replication.md).
With the FoundationDB based [RxStorage](./rx-storage.md) of [RxDB](https://rxdb.info/) you can combine the benefits of FoundationDB while having a fully featured, high performance NoSQL database.
## Features of RxDB+FoundationDB
Using RxDB on top of FoundationDB, gives you many benefits compare to using the plain FoundationDB API:
- **Indexes**: In RxDB with a FoundationDB storage layer, indexes are used to optimize query performance, allowing for fast and efficient data retrieval even in large datasets. You can define single and compound indexes with the [RxDB schema](./rx-schema.md).
- **Schema Based Data Model**: Utilizing a [jsonschema](./rx-schema.md) based data model, the system offers a highly structured and versatile approach to organizing and [validating data](./schema-validation.md), ensuring consistency and clarity in database interactions.
- **Complex Queries**: The system supports complex [NoSQL queries](./rx-query.md), allowing for advanced data manipulation and retrieval, tailored to specific needs and intricate data relationships. For example you can do `$regex` or `$or` queries which is hardly possible with the plain key-value access of FoundationDB.
- **Observable Queries & Documents**: RxDB's observable queries and documents feature ensures real-time updates and synchronization, providing dynamic and responsive data interactions in applications.
- **Compression**: RxDB employs data [compression techniques](./key-compression.md) to reduce storage requirements and enhance transmission efficiency, making it more cost-effective and faster, especially for large volumes of data. You can compress the [NoSQL document](./key-compression.md) data, but also the [binary attachments](./rx-attachment.md#attachment-compression) data.
- **Attachments**: RxDB supports the storage and management of [attachments](./rx-attachment.md) which allowing for the seamless inclusion of binary data like images or documents alongside structured data within the database.
## Installation
- Install the [FoundationDB client cli](https://apple.github.io/foundationdb/getting-started-linux.html) which is used to communicate with the FoundationDB cluster.
- Install the [FoundationDB node bindings npm module](https://www.npmjs.com/package/foundationdb) via `npm install foundationdb`. This will install `v2.x.x`, which is only compatible with FoundationDB server and client `v7.3.x` (which is the only version currently maintained by the FoundationDB team). If you need to use an older version (e.g. `7.1.x` or `6.3.x`), you should run `npm install foundationdb@1.1.4` (though this might only work with `v6.3.x`).
- Due to an outstanding bug in node foundationdb, you will need to specify an `apiVersion` of `720` even though you are using `730`. When [this PR](https://github.com/josephg/node-foundationdb/pull/86) is merged, you will be able to use `730`.
## Usage
```typescript
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageFoundationDB
} from 'rxdb/plugins/storage-foundationdb';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageFoundationDB({
/**
* Version of the API of the FoundationDB cluster..
* FoundationDB is backwards compatible across a wide range of versions,
* so you have to specify the api version.
* If in doubt, set it to 720.
*/
apiVersion: 720,
/**
* Path to the FoundationDB cluster file.
* (optional)
* If in doubt, leave this empty to use the default location.
*/
clusterFile: '/path/to/fdb.cluster',
/**
* Amount of documents to be fetched in batch requests.
* You can change this to improve performance depending on
* your database access patterns.
* (optional)
* [default=50]
*/
batchSize: 50
})
});
```
## Multi Instance
Because FoundationDB does not offer a [changestream](https://forums.foundationdb.org/t/streaming-data-out-of-foundationdb/683/2), it is not possible to use the same cluster from more than one Node.js process at the same time. For example you cannot spin up multiple servers with RxDB databases that all use the same cluster. There might be workarounds to create something like a FoundationDB changestream and you can make a Pull Request if you need that feature.
## Running the FoundationDB Server with Docker
Instead of installing the FoundationDB server locally, you can run it in a Docker container. This is the recommended approach for local development and CI environments.
```bash
# Pull the Docker image
docker pull foundationdb/foundationdb:7.3.59
# Start the container with host networking
docker run -d \
--name rxdb-foundationdb \
--network host \
-e FDB_NETWORKING_MODE=host \
foundationdb/foundationdb:7.3.59
# Copy the cluster file from the container
sudo mkdir -p /etc/foundationdb
docker cp rxdb-foundationdb:/var/fdb/fdb.cluster /etc/foundationdb/fdb.cluster
# Configure the database
fdbcli --exec "configure new single memory" --timeout 30
```
You can also use the provided npm scripts:
```bash
npm run foundationdb:start # Starts the Docker container and configures the database
npm run foundationdb:stop # Stops and removes the Docker container
```
---
## Schema Validation
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_VALIDATION_INDEXEDDB, PERFORMANCE_DATA_VALIDATION_MEMORY } from '@site/src/components/performance-data';
import {Faq, FaqItem} from '@site/src/components/faq';
# Schema validation
RxDB has multiple validation implementations that can be used to ensure that your document data is always matching the provided JSON
schema of your [RxCollection](./rx-collection.md).
The schema validation is **not a plugin** but comes in as a wrapper around any other `RxStorage` and it will then validate all data that is written into that storage. This is required for multiple reasons:
- It allows us to run the validation inside of a [Worker RxStorage](./rx-storage-worker.md) instead of running it in the main JavaScript process.
- It allows us to configure which [RxDatabase](./rx-database.md) instance must use the validation and which does not. In production it often makes sense to validate user data, but you might not need the validation for data that is only replicated from the backend.
:::warning
Schema validation can be **CPU expensive** and increases your build size. You should always use a schema validation in development mode. For most use cases, you **should not** use a validation in production for better performance.
:::
When no validation is used, any document data can be saved but there might be **undefined behavior** when saving data that does not comply to the schema of a `RxCollection`.
RxDB has different implementations to validate data, each of them is based on a different [JSON Schema library](https://json-schema.org/tools). In this example we use the [LocalStorage RxStorage](./rx-storage-localstorage.md), but you can wrap the validation around **any other** [RxStorage](./rx-storage.md).
### validate-ajv
A validation-module that does the schema-validation. This one is using [ajv](https://github.com/epoberezkin/ajv) as validator which is a bit faster. Better compliant to the jsonschema-standard but also has a bigger build-size.
```javascript
import { wrappedValidateAjvStorage } from 'rxdb/plugins/validate-ajv';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// wrap the validation around the main RxStorage
const storage = wrappedValidateAjvStorage({
storage: getRxStorageLocalstorage()
});
const db = await createRxDatabase({
name: randomCouchString(10),
storage
});
```
### validate-z-schema
Both `is-my-json-valid` and `validate-ajv` use `eval()` to perform validation which might not be wanted when `'unsafe-eval'` is not allowed in Content Security Policies. This one is using [z-schema](https://github.com/zaggino/z-schema) as validator which doesn't use `eval`.
```javascript
import { wrappedValidateZSchemaStorage } from 'rxdb/plugins/validate-z-schema';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// wrap the validation around the main RxStorage
const storage = wrappedValidateZSchemaStorage({
storage: getRxStorageLocalstorage()
});
const db = await createRxDatabase({
name: randomCouchString(10),
storage
});
```
### validate-is-my-json-valid
**WARNING**: The `is-my-json-valid` validation is no longer supported until [this bug](https://github.com/mafintosh/is-my-json-valid/pull/192) is fixed.
The `validate-is-my-json-valid` plugin uses [is-my-json-valid](https://www.npmjs.com/package/is-my-json-valid) for schema validation.
```javascript
import {
wrappedValidateIsMyJsonValidStorage
} from 'rxdb/plugins/validate-is-my-json-valid';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// wrap the validation around the main RxStorage
const storage = wrappedValidateIsMyJsonValidStorage({
storage: getRxStorageLocalstorage()
});
const db = await createRxDatabase({
name: randomCouchString(10),
storage
});
```
## Custom Formats
The schema validators provide methods to add custom formats like a `email` format.
You have to add these formats **before** you create your database.
### Ajv Custom Format
```ts
import { getAjv } from 'rxdb/plugins/validate-ajv';
const ajv = getAjv();
ajv.addFormat('email', {
type: 'string',
validate: v => v.includes('@') // ensure email fields contain the @ symbol
});
```
### Z-Schema Custom Format
```ts
import { ZSchemaClass } from 'rxdb/plugins/validate-z-schema';
ZSchemaClass.registerFormat('email', function (v: string) {
return v.includes('@'); // ensure email fields contain the @ symbol
});
```
## Performance comparison of the validators
The RxDB team ran performance benchmarks using two storage options on an Ubuntu 24.04 machine with Chrome version `131.0.6778.85`. The testing machine has 32 core `13th Gen Intel(R) Core(TM) i9-13900HX` CPU.
IndexedDB Storage (based on the IndexedDB API in the browser):
Memory Storage: stores everything in memory for extremely fast reads and writes, with no persistence by default. Often used with the RxDB memory-mapped plugin that processes data in memory and later persists to disc in background:
Including a validator library also increases your JavaScript bundle size. Here's how it breaks down (minified + gzip):
| **Build Size** (minified+gzip) | Build Size (IndexedDB) | Build Size (memory) |
| ------------------------------ | :----------------: | ------------------: |
| no validator | 73103 B | 39976 B |
| ajv | 106135 B | 72773 B |
| z-schema | 125186 B | 91882 B |
## FAQ
Schema validation structurally guarantees that all document mutations strictly comply with the statically defined **[RxCollection](./rx-collection.md)** JSON Schema format before data is physically committed to the underlying `RxStorage`. Yes, both `ajv` (via `ajv-formats`) and `is-my-json-valid` rely natively on `eval()` or `new Function()` during compilation to aggressively optimize their validation runtimes. If your deployment environment enforces extremely strict `unsafe-eval` Content Security Policies (CSP), you must explicitly swap the validator wrapper to **`validate-z-schema`**, which strictly avoids `eval()` at the cost of marginally slower execution.
---
## Encryption
import {Steps} from '@site/src/components/steps';
import {PremiumBlock} from '@site/src/components/premium-block';
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_ENCRYPTION, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {IconEncryption} from '@site/src/components/icons/encryption';
import {Faq, FaqItem} from '@site/src/components/faq';
# }>Encrypted Local Storage with RxDB
The RxDB encryption plugin empowers developers to fortify their applications' data security. It seamlessly integrates with [RxDB](https://rxdb.info/), allowing for the secure storage and retrieval of documents by **encrypting them with a password**. With encryption and decryption processes handled internally, it ensures that sensitive data remains confidential, making it a valuable tool for building robust, privacy-conscious applications. The encryption works on all RxDB supported devices types like the **[browser](./articles/browser-database.md)**, **[ReactNative](./react-native-database.md)** or **[Node.js](./nodejs-database.md)**.
Encrypting client-side stored data in RxDB offers numerous advantages:
- **Enhanced Security**: In the unfortunate event of a user's device being stolen, the encrypted data remains safeguarded on the hard drive, inaccessible without the correct password.
- **Access Control**: You can retain control over stored data by revoking access at any time simply by withholding the password.
- **Tamper proof** Other applications on the device cannot read out the stored data when the password is only kept in the process-specific memory
## Querying encrypted data
RxDB handles the encryption and decryption of data internally. This means that when you work with a [RxDocument](./rx-document.md), you can access the properties of the document just like you would with normal, unencrypted data. RxDB automatically decrypts the data for you when you retrieve it, making it transparent to your application code.
This means the encryption works with all [RxStorage](./rx-storage.md) like **[SQLite](./rx-storage-sqlite.md)**, **[IndexedDB](./rx-storage-indexeddb.md)**, **[OPFS](./rx-storage-opfs.md)** and so on.
However, there's a limitation when it comes to querying encrypted fields. **Encrypted fields cannot be used as operators in queries**. This means you cannot perform queries like "find all documents where the encrypted field equals a certain value." RxDB does not expose the encrypted data in a way that allows direct querying based on the encrypted content. To filter or search for documents based on the contents of encrypted fields, you would need to first decrypt the data and then perform the query, which might not be efficient or practical in some cases.
You could however use the [memory mapped](./rx-storage-memory-mapped.md) RxStorage to replicate the encrypted documents into a non-encrypted in-memory storage and then query them like normal.
## Password handling
RxDB does not define how you should store or retrieve the encryption password. It only requires you to provide the password on database creation which grants you flexibility in how you manage encryption passwords.
You could ask the user on app-start to insert the password, or you can retrieve the password from your backend on app start (or revoke access by no longer providing the password).
## Asymmetric encryption
The encryption plugin itself uses **symmetric encryption** with a password to guarantee best performance when reading and storing data.
It is not able to do **Asymmetric encryption** by itself. If you need Asymmetric encryption with a private/publicKey, it is recommended to encrypted the password itself with the asymmetric keys and store the encrypted password beside the other data. On app-start you can decrypt the password with the private key and use the decrypted password in the RxDB encryption plugin
## Using the RxDB Encryption Plugins
RxDB currently has two plugins for encryption:
- The free `encryption-crypto-js` plugin that is based on the `AES` algorithm of the [crypto-js](https://www.npmjs.com/package/crypto-js) library
- `encryption-web-crypto` plugin that is based on the native [Web Crypto API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API) which makes it faster and more secure to use. Document inserts are about 10x faster compared to `crypto-js` and it has a smaller build size because it uses the browsers API instead of bundling an npm module.
An RxDB encryption plugin is a wrapper around any other [RxStorage](./rx-storage.md).
### Wrap your RxStorage with the encryption
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// wrap the normal storage with the encryption plugin
const encryptedStorage = wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageLocalstorage()
});
```
### Create a RxDatabase with the wrapped storage
Also you have to set a **password** when creating the database. The format of the password depends on which encryption plugin is used.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
// create an encrypted database
const db = await createRxDatabase({
name: 'mydatabase',
storage: encryptedStorage,
password: 'sudoLetMeIn'
});
```
### Create an RxCollection with an encrypted property
To define a field as being encrypted, you have to add it to the `encrypted` fields list in the schema.
```ts
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
secret: {
type: 'string'
},
},
required: ['id'],
encrypted: ['secret']
};
await db.addCollections({
myDocuments: {
schema
}
})
```
## Using the WebCrypto API
```ts
import {
wrappedKeyEncryptionWebCryptoStorage,
createPassword
} from 'rxdb-premium/plugins/encryption-web-crypto';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
// wrap the normal storage with the encryption plugin
const encryptedIndexedDbStorage = wrappedKeyEncryptionWebCryptoStorage({
storage: getRxStorageIndexedDB()
});
const myPasswordObject = {
// Algorithm can be oneOf: 'AES-CTR' | 'AES-CBC' | 'AES-GCM'
algorithm: 'AES-CTR',
password: 'myRandomPasswordWithMin8Length'
};
// create an encrypted database
const db = await createRxDatabase({
name: 'mydatabase',
storage: encryptedIndexedDbStorage,
password: myPasswordObject
});
/* ... */
```
## Changing the password
The password is set database specific and it is not possible to change the password of a database. Opening an existing database with a different password will throw an error. To change the password you can either:
- Use the [storage migration plugin](./migration-storage.md) to migrate the database state into a new database.
- Store a randomly created meta-password in a different RxDatabase as a value of a [local document](./rx-local-document.md). Encrypt the meta password with the actual user password and read it out before creating the actual database.
## Encrypted attachments
To store the [attachments](./rx-attachment.md) data encrypted, you have to set `encrypted: true` in the `attachments` property of the schema.
```ts
const mySchema = {
version: 0,
type: 'object',
properties: {
/* ... */
},
attachments: {
// if true, the attachment-data will be
// encrypted with the db-password
encrypted: true
}
};
```
## Encryption and workers
If you are using [Worker RxStorage](./rx-storage-worker.md) or [SharedWorker RxStorage](./rx-storage-shared-worker.md) with encryption, it's recommended to run encryption inside of the worker. Encryption can be very cpu intensive and would take away CPU-power from the main thread which is the main reason to use workers.
You do not need to worry about setting the password inside of the worker. The password will be set when calling createRxDatabase from the main thread, and will be passed internally to the storage in the worker automatically.
### Using encryption inside the worker with OPFS
When you wrap a storage like [OPFS](./rx-storage-opfs.md) with encryption inside of a worker, you have to set the `usesRxDatabaseInWorker` option on the OPFS storage. Without this option, the OPFS storage returns raw JSON strings instead of parsed objects as a performance optimization. The encryption wrapper cannot process these strings and will throw an error.
```ts
// inside of the worker.js file
import { getRxStorageOPFS } from 'rxdb-premium/plugins/storage-opfs';
import {
wrappedKeyEncryptionWebCryptoStorage
} from 'rxdb-premium/plugins/encryption-web-crypto';
const storage = wrappedKeyEncryptionWebCryptoStorage({
storage: getRxStorageOPFS({
// Required when wrapping OPFS with encryption inside a worker
usesRxDatabaseInWorker: true
})
});
```
## Encryption Performance
As shown in the chart, the WebCrypto based encryption plugins are generally **5 times faster** than the `crypto-js` plugin.
## FAQ
RxDB provides robust plugins for client side field encryption directly within your javascript database. You encrypt sensitive document properties transparently before they save to local storage. The `encryption-crypto-js` plugin utilizes AES algorithms for dependable security. The `encryption-web-crypto` plugin employs native browser APIs to achieve superior performance. You maintain data confidentiality across Web, React Native, and Node.js environments.
You can implement encryption in JavaScript by manually encrypting fields with the native `WebCrypto API` before storing them, but this breaks standard querying. Advanced databases like **[RxDB](./rx-database.md)** simplify this through schema-level encryption plugins (`encryption-web-crypto`). By flagging specific document fields as `encrypted: true` in your JSON Schema, RxDB automatically encrypts the data before writing to the storage engine (like IndexedDB or SQLite) and decrypts it instantly upon retrieval.
No, `chrome.storage.local` (and standard `IndexedDB` in the browser) is **not** encrypted at rest by default. Any user or potentially malicious extension with adequate local machine access can read the underlying data files. To properly secure sensitive data at rest in a browser extension or Web App, you must explicitly encrypt strings before saving them, a process seamlessly automated by using an encrypted [RxStorage](./rx-storage.md) wrapper. See [IndexedDB Encryption](./articles/indexeddb/indexeddb-encryption.md) for the details.
Yes, libraries like `crypto-js` or wrappers over the native WebCrypto API provide robust open-source encryption. For developers building native mobile apps (React Native, Expo, Ionic) or browser applications, utilizing a database that ships with native encryption wrappers like **[RxDB's Encryption Plugins](https://rxdb.info/encryption.html)** is the most reliable method. It ensures data is never written to disk in plain text while allowing you to effortlessly swap underlying storage layers without rewriting your cryptography logic.
No. When you encrypt a parent field, the entire object at that path is encrypted as a single string. You cannot also encrypt a child path of an already-encrypted parent. For example, if you encrypt `nested`, you must **not** also add `nested.secret` to the `encrypted` array. Doing so will throw an error in [dev-mode](./dev-mode.md).
```ts
// NOT ALLOWED - 'nested.secret' is a child of 'nested'
const schema = {
encrypted: ['nested', 'nested.secret']
};
// CORRECT - only encrypt the parent
const schema = {
encrypted: ['nested']
};
```
---
## Key Compression
import {Steps} from '@site/src/components/steps';
# Key Compression
With the key compression plugin, documents will be stored in a compressed format which saves up to 40% disc space.
For compression the npm module [jsonschema-key-compression](https://github.com/pubkey/jsonschema-key-compression) is used.
It compresses json-data based on its json-schema while still having valid json. It works by compressing long attribute-names into smaller ones and backwards.
The compression and decompression happens internally, so when you work with a [RxDocument](./rx-document.md), you can access any property like normal.
## Enable key compression
The key compression plugin is a wrapper around any other [RxStorage](./rx-storage.md).
### Wrap your RxStorage with the key compression plugin
```ts
import { wrappedKeyCompressionStorage } from 'rxdb/plugins/key-compression';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const storageWithKeyCompression = wrappedKeyCompressionStorage({
storage: getRxStorageLocalstorage()
});
```
### Create an RxDatabase
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
const db = await createRxDatabase({
name: 'mydatabase',
storage: storageWithKeyCompression
});
```
### Create a compressed RxCollection
```ts
const mySchema = {
keyCompression: true, // set this to true, to enable the keyCompression
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
/* ... */
}
};
await db.addCollections({
docs: {
schema: mySchema
}
});
```
---
## RxDB Logger Plugin - Track & Optimize
import {PremiumBlock} from '@site/src/components/premium-block';
import {Steps} from '@site/src/components/steps';
# RxDB Logger Plugin
With the logger plugin you can log all operations to the [storage layer](./rx-storage.md) of your [RxDatabase](./rx-database.md).
This is useful to debug performance problems and for monitoring with Application Performance Monitoring (APM) tools like **Bugsnag**, **Datadog**, **Elastic**, **Sentry** and others.
## Using the logger plugin
The logger is a wrapper that can be wrapped around any [RxStorage](./rx-storage.md). Once your storage is wrapped, you can create your database with the wrapped storage and the logging will automatically happen.
### Import Plugins
```ts
import {
wrappedLoggerStorage
} from 'rxdb-premium/plugins/logger';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
```
### Wrap Storage
```ts
// wrap a storage with the logger
const loggingStorage = wrappedLoggerStorage({
storage: getRxStorageIndexedDB({})
});
```
### Create Database
```ts
// create your database with the wrapped storage
const db = await createRxDatabase({
name: 'mydatabase',
storage: loggingStorage
});
// create collections etc...
```
## Specify what to be logged
By default, the plugin will log all operations and it will also run a `console.time()/console.timeEnd()` around each operation. You can specify what to log so that your logs are less noisy. For this you provide a settings object when calling `wrappedLoggerStorage()`.
```ts
const loggingStorage = wrappedLoggerStorage({
storage: getRxStorageIndexedDB({}),
settings: {
// can used to prefix all log strings, default=''
prefix: 'my-prefix',
/**
* Be default, all settings are true.
*/
// if true, it will log timings with console.time() and console.timeEnd()
times: true,
// if false, it will not log meta storage instances like used in replication
metaStorageInstances: true,
// operations
bulkWrite: true,
findDocumentsById: true,
query: true,
count: true,
info: true,
getAttachmentData: true,
getChangedDocumentsSince: true,
cleanup: true,
close: true,
remove: true
}
});
```
## Using custom logging functions
With the logger plugin you can also run custom log functions for all operations.
```ts
const loggingStorage = wrappedLoggerStorage({
storage: getRxStorageIndexedDB({}),
onOperationStart: (operationsName, logId, args) => void,
onOperationEnd: (operationsName, logId, args) => void,
onOperationError: (operationsName, logId, args, error) => void
});
```
---
## Remote RxStorage
The Remote [RxStorage](./rx-storage.md) is made to use a remote storage and communicate with it over an asynchronous message channel.
The remote part could be on another JavaScript process or even on a different host machine.
The remote storage plugin is used in many RxDB plugins like the [worker](./rx-storage-worker.md) or the [electron](./electron.md) plugin.
## Usage
The remote storage communicates over a message channel which has to implement the `messageChannelCreator` function which returns an object that has a `messages$` observable and a `send()` function on both sides and a `close()` function that closes the RemoteMessageChannel.
```ts
// on the client
import { getRxStorageRemote } from 'rxdb/plugins/storage-remote';
const storage = getRxStorageRemote({
identifier: 'my-id',
mode: 'storage',
messageChannelCreator: () => Promise.resolve({
messages$: new Subject(),
send(msg) {
// send to remote storage
}
})
});
const myDb = await createRxDatabase({
storage
});
// on the remote
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { exposeRxStorageRemote } from 'rxdb/plugins/storage-remote';
exposeRxStorageRemote({
storage: getRxStorageLocalstorage(),
messages$: new Subject(),
send(msg){
// send to other side
}
});
```
## Usage with a Websocket server
The remote storage plugin contains helper functions to create a remote storage over a WebSocket server.
This is often used in Node.js to give one microservice access to another services database **without** having to replicate the full database state.
```ts
// server.js
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
import {
startRxStorageRemoteWebsocketServer
} from 'rxdb/plugins/storage-remote-websocket';
// either you can create the server based on a RxDatabase
const serverBasedOnDatabase = await startRxStorageRemoteWebsocketServer({
port: 8080,
database: myRxDatabase
});
// or you can create the server based on a pure RxStorage
const serverBasedOn = await startRxStorageRemoteWebsocketServer({
port: 8080,
storage: getRxStorageMemory()
});
```
```ts
// client.js
import { getRxStorageRemoteWebsocket } from 'rxdb/plugins/storage-remote-websocket';
const myDb = await createRxDatabase({
storage: getRxStorageRemoteWebsocket({
url: 'ws://example.com:8080'
})
});
```
## Sending custom messages
The remote storage can also be used to send custom messages to and from the remote instance.
On the remote you have to define a `customRequestHandler` like:
```ts
const serverBasedOnDatabase = await startRxStorageRemoteWebsocketServer({
port: 8080,
database: myRxDatabase,
async customRequestHandler(msg){
// here you can return any JSON object as an 'answer'
return {
foo: 'bar'
};
}
});
```
On the client instance you can then call the `customRequest()` method:
```ts
const storage = getRxStorageRemoteWebsocket({
url: 'ws://example.com:8080'
});
const answer = await storage.customRequest({ bar: 'foo' });
console.dir(answer); // > { foo: 'bar' }
```
---
## Turbocharge RxDB with Worker RxStorage
import {PremiumBlock} from '@site/src/components/premium-block';
import {Faq, FaqItem} from '@site/src/components/faq';
# Worker RxStorage
With the worker plugin, you can put the [RxStorage](./rx-storage.md) of your database inside of a WebWorker (in browsers) or a Worker Thread (in node.js). By doing so, you can take CPU load from the main process and move it into the worker's process which can improve the perceived performance of your application. Notice that for browsers, it is recommended to use the [SharedWorker](./rx-storage-shared-worker.md) instead to get a better performance.
## On the worker process
```ts
// worker.ts
import { exposeWorkerRxStorage } from 'rxdb-premium/plugins/storage-worker';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
exposeWorkerRxStorage({
/**
* You can wrap any implementation of the RxStorage interface
* into a worker.
* Here we use the IndexedDB RxStorage.
*/
storage: getRxStorageIndexedDB()
});
```
## On the main process
```ts
import {
createRxDatabase
} from 'rxdb';
import { getRxStorageWorker } from 'rxdb-premium/plugins/storage-worker';
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageWorker(
{
/**
* Contains any value that can be used as parameter
* to the Worker constructor of thread.js
* Most likely you want to put the path to the worker.js file in here.
*
* @link https://developer.mozilla.org/en-US/docs/Web/API/Worker/Worker
*/
workerInput: 'path/to/worker.js',
/**
* (Optional) options
* for the worker.
*/
workerOptions: {
type: 'module',
credentials: 'omit'
}
}
)
});
```
## Pre-build workers
The `worker.js` must be a self containing JavaScript file that contains all dependencies in a bundle.
To make it easier for you, RxDB ships with pre-bundles worker files that are ready to use.
You can find them in the folder `node_modules/rxdb-premium/dist/workers` after you have installed the [RxDB Premium π Plugin](/premium/). From there you can copy them to a location where it can be served from the webserver and then use their path to create the `RxDatabase`.
Any valid `worker.js` JavaScript file can be used both, for normal Workers and SharedWorkers.
```ts
import {
createRxDatabase
} from 'rxdb';
import { getRxStorageWorker } from 'rxdb-premium/plugins/storage-worker';
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageWorker(
{
/**
* Path to where the copied file from node_modules/rxdb/dist/workers
* is reachable from the webserver.
*/
workerInput: '/indexeddb.worker.js'
}
)
});
```
## Building a custom worker
The easiest way to bundle a custom `worker.js` file is by using webpack. Here is the webpack-config that is also used for the prebuild workers:
```ts
// webpack.config.js
const path = require('path');
const TerserPlugin = require('terser-webpack-plugin');
const projectRootPath = path.resolve(
__dirname,
'../../' // path from webpack-config to the root folder of the repo
);
const babelConfig = require(path.join(projectRootPath, 'babel.config'));
const baseDir = './dist/workers/'; // output path
module.exports = {
target: 'webworker',
entry: {
'my-custom-worker': baseDir + 'my-custom-worker.js',
},
output: {
filename: '[name].js',
clean: true,
path: path.resolve(
projectRootPath,
'dist/workers'
),
},
mode: 'production',
module: {
rules: [
{
test: /\.tsx?$/,
exclude: /(node_modules)/,
use: {
loader: 'babel-loader',
options: babelConfig
}
}
],
},
resolve: {
extensions: ['.tsx', '.ts', '.js', '.mjs', '.mts']
},
optimization: {
moduleIds: 'deterministic',
minimize: true,
minimizer: [new TerserPlugin({
terserOptions: {
format: {
comments: false,
},
},
extractComments: false,
})],
}
};
```
## One worker per database
Each call to `getRxStorageWorker()` will create a different worker instance so that when you have more than one `RxDatabase`, each database will have its own JavaScript worker process.
To reuse the worker instance in more than one `RxDatabase`, you can store the output of `getRxStorageWorker()` into a variable and use that one. Reusing the worker can decrease the initial page load, but you might get slower database operations.
```ts
// Call getRxStorageWorker() exactly once
const workerStorage = getRxStorageWorker({
workerInput: 'path/to/worker.js'
});
// use the same storage for both databases.
const databaseOne = await createRxDatabase({
name: 'database-one',
storage: workerStorage
});
const databaseTwo = await createRxDatabase({
name: 'database-two',
storage: workerStorage
});
```
## Passing in a Worker instance
Instead of setting an url as `workerInput`, you can also specify a function that returns a new `Worker` instance when called.
```ts
getRxStorageWorker({
workerInput: () => new Worker('path/to/worker.js')
})
```
This can be helpful for environments where the worker is build dynamically by the bundler. For example in angular you would create a `my-custom.worker.ts` file that contains a custom build worker and then import it.
```ts
const storage = getRxStorageWorker({
workerInput: () => new Worker(new URL('./my-custom.worker', import.meta.url)),
});
```
```ts
//> my-custom.worker.ts
import { exposeWorkerRxStorage } from 'rxdb-premium/plugins/storage-worker';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
exposeWorkerRxStorage({
storage: getRxStorageIndexedDB()
});
```
## FAQ
WebWorkers (and Worker Threads in Node.js) execute in entirely separate, wholly isolated V8 JavaScript environments that do *not* share memory heaps with the main UI thread. Because they cannot pass memory pointers, transferring **[RxDB](./rx-database.md)** queries and JSON arrays between the UI and the Worker RxStorage requires structural cloning serialization over IPC channels. While this adds minor IPC latency, it guarantees the main thread's 60fps render loop remains utterly unblocked during extremely heavy database I/O workloads.
---
## Boost Performance with SharedWorker RxStorage
import {PremiumBlock} from '@site/src/components/premium-block';
import {Faq, FaqItem} from '@site/src/components/faq';
# SharedWorker RxStorage
The SharedWorker [RxStorage](./rx-storage.md) uses the [SharedWorker API](https://developer.mozilla.org/en-US/docs/Web/API/SharedWorker) to run the storage inside of a separate JavaScript process **in browsers**. Compared to a normal [WebWorker](./rx-storage-worker.md), the SharedWorker is created exactly once, even when there are multiple browser tabs opened. Because of having exactly one worker, multiple performance optimizations can be done because the storage itself does not have to handle multiple opened database connections.
## Usage
### On the SharedWorker process
In the worker process JavaScript file, you have to wrap the original RxStorage with `getRxStorageIndexedDB()`.
```ts
// shared-worker.ts
import { exposeWorkerRxStorage } from 'rxdb-premium/plugins/storage-worker';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
exposeWorkerRxStorage({
/**
* You can wrap any implementation of the RxStorage interface
* into a worker.
* Here we use the IndexedDB RxStorage.
*/
storage: getRxStorageIndexedDB()
});
```
### On the main process
```ts
import {
createRxDatabase
} from 'rxdb';
import { getRxStorageSharedWorker } from 'rxdb-premium/plugins/storage-worker';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageSharedWorker(
{
/**
* Contains any value that can be used as parameter
* to the SharedWorker constructor of thread.js
* Most likely you want to put the path to
* the shared-worker.js file in here.
*
* @link https://developer.mozilla.org/
* en-US/docs/Web/API/SharedWorker
*/
workerInput: 'path/to/shared-worker.js',
/**
* (Optional) options
* for the worker.
*/
workerOptions: {
type: 'module',
credentials: 'omit',
extendedLifetime: true
}
}
)
});
```
## Pre-build workers
The `shared-worker.js` must be a self containing JavaScript file that contains all dependencies in a bundle.
To make it easier for you, RxDB ships with pre-bundles worker files that are ready to use.
You can find them in the folder `node_modules/rxdb-premium/dist/workers` after you have installed the [RxDB Premium π Plugin](/premium/). From there you can copy them to a location where it can be served from the webserver and then use their path to create the `RxDatabase`
Any valid `worker.js` JavaScript file can be used both, for normal Workers and SharedWorkers.
```ts
import {
createRxDatabase
} from 'rxdb';
import { getRxStorageSharedWorker } from 'rxdb-premium/plugins/storage-worker';
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageSharedWorker(
{
/**
* Path to where the copied
* file from node_modules/rxdb-premium/dist/workers
* is reachable from the webserver.
*/
workerInput: '/indexeddb.shared-worker.js'
}
)
});
```
## Building a custom worker
To build a custom `worker.js` file, check out the webpack config at the [worker](./rx-storage-worker.md#building-a-custom-worker) documentation. Any worker file form the worker storage can also be used in a shared worker because `exposeWorkerRxStorage` detects where it runs and exposes the correct messaging endpoints.
## Passing in a SharedWorker instance
Instead of setting an url as `workerInput`, you can also specify a function that returns a new `SharedWorker` instance when called. This is mostly used when you have a custom worker file and dynamically import it.
This works equal to the [workerInput of the Worker Storage](./rx-storage-worker.md#passing-in-a-worker-instance)
## Set multiInstance: false
When you know that you only ever create your RxDatabase inside of the shared worker, you might want to set `multiInstance: false` to prevent sending change events across JavaScript realms and to improve performance. Do not set this when you also create the same storage on another realm, like when you have the same RxDatabase once inside the shared worker and once on the main thread.
## Replication with SharedWorker
When a SharedWorker RxStorage is used, it is recommended to run the [replication](./replication.md) **inside** of the worker. This is the best option for performance. You can do that by opening another [RxDatabase](./rx-database.md) inside of it and starting the replication there. If you are not concerned about performance, you can still start replication on the main thread instead. But you should never run replication on both the main thread **and** the worker.
```ts
// shared-worker.ts
import { exposeWorkerRxStorage } from 'rxdb-premium/plugins/storage-worker';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
import {
createRxDatabase,
addRxPlugin
} from 'rxdb';
import {
RxDBReplicationGraphQLPlugin
} from 'rxdb/plugins/replication-graphql';
addRxPlugin(RxDBReplicationGraphQLPlugin);
const baseStorage = getRxStorageIndexedDB();
// first expose the RxStorage to the outside
exposeWorkerRxStorage({
storage: baseStorage
});
/**
* Then create a normal RxDatabase and RxCollections
* and start the replication.
*/
const database = await createRxDatabase({
name: 'mydatabase',
storage: baseStorage
});
await db.addCollections({
humans: {/* ... */}
});
const replicationState = db.humans.syncGraphQL({/* ... */});
```
### Limitations
- The SharedWorker API is [not available in some mobile browser](https://caniuse.com/sharedworkers)
## FAQ
No. A Service Worker is not the same as a Shared Worker. While you can use RxDB inside of a ServiceWorker, you cannot use the ServiceWorker as a RxStorage that gets accessed by an outside RxDatabase instance.
The `SharedWorker` API spawns exactly one isolated JavaScript background thread that is shared globally across all open browser tabs targeting the same origin. When you attach RxDB to a Shared Worker, you eliminate redundant IndexedDB socket connections and expensive JSON serialization across individual tabs. Only the background worker executes resource-heavy database intensive CRUD operations, broadcasting the ultra-lightweight result differentials down to the passive UI tabs simultaneously.
---
## Blazing-Fast Memory Mapped RxStorage
import {PremiumBlock} from '@site/src/components/premium-block';
# Memory Mapped RxStorage
The memory mapped [RxStorage](./rx-storage.md) is a wrapper around any other RxStorage. The wrapper creates an in-memory storage that is used for query and write operations. This memory instance is kept persistent with a given underlying storage.
## Pros
- Improves read/write performance because these operations run against the in-memory storage.
- Decreases initial page load because it loads all data in a single bulk request. It even detects if the database is used for the first time and then it does not have to await the creation of the persistent storage.
- Can store encrypted data on disc while still being able to run queries on the non-encrypted in-memory state.
## Cons
- It does not support [attachments](./rx-attachment.md) because storing big attachments data in-memory should not be done.
- When the JavaScript process is killed ungracefully like when the browser crashes or the power of the PC is terminated, it might happen that some memory writes are not persisted to the parent storage. This can be prevented with the `awaitWritePersistence` flag.
- The memory-mapped storage can only be used if all data fits into the memory of the JavaScript process. This is normally not a problem because a browser has much memory these days and plain JSON document data is not that big.
- Because it has to await an initial data loading from the parent storage into the memory, initial page load time can increase when much data is already stored. This is likely not a problem when you store less than `10k` documents.
## Using the Memory-Mapped RxStorage
```ts
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
import {
getMemoryMappedRxStorage
} from 'rxdb-premium/plugins/storage-memory-mapped';
/**
* Here we use the IndexedDB RxStorage as persistence storage.
* Any other RxStorage can also be used.
*/
const parentStorage = getRxStorageIndexedDB();
// wrap the persistent storage with the memory-mapped storage.
const storage = getMemoryMappedRxStorage({
storage: parentStorage
});
// create the RxDatabase like you would do with any other RxStorage
const db = await createRxDatabase({
name: 'myDatabase',
storage,
});
/** ... **/
```
## Multi-Tab Support
By how the memory-mapped storage works, it is not possible to have the same storage open in multiple JavaScript processes. So when you use this in a browser application, you can not open multiple databases when the app is used in multiple browser tabs.
To solve this, use the [SharedWorker Plugin](./rx-storage-shared-worker.md) so that the memory-mapped storage runs inside of a SharedWorker exactly once and is then reused for all browser tabs.
If you have a single JavaScript process, like in a React Native app, you do not have to care about this and can just use the memory-mapped storage in the main process.
## Encryption of the persistent data
Normally RxDB is not capable of running queries on encrypted fields. But when you use the memory-mapped RxStorage, you can store the document data encrypted on disc, while being able to run queries on the not encrypted in-memory state. Make sure you use the [encryption](./encryption.md) storage wrapper around the persistent storage, **NOT** around the memory-mapped storage as a whole.
```ts
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
import {
getMemoryMappedRxStorage
} from 'rxdb-premium/plugins/storage-memory-mapped';
import {
wrappedKeyEncryptionWebCryptoStorage
} from 'rxdb-premium/plugins/encryption-web-crypto';
const storage = getMemoryMappedRxStorage({
storage: wrappedKeyEncryptionWebCryptoStorage({
storage: getRxStorageIndexedDB()
})
});
const db = await createRxDatabase({
name: 'myDatabase',
storage,
});
/** ... **/
```
## Await Write Persistence
Running operations on the memory-mapped storage by default returns directly when the operation has run on the in-memory state and then persist changes in the background.
Sometimes you might want to ensure write operations is persisted, you can do this by setting `awaitWritePersistence: true`.
```ts
const storage = getMemoryMappedRxStorage({
awaitWritePersistence: true,
storage: getRxStorageIndexedDB()
});
```
## Block Size Limit
During cleanup, the memory-mapped storage will merge many small write-blocks into single big blocks for better initial load performance.
The `blockSizeLimit` defines the maximum of how many documents get stored in a single block. The default is `10000`.
```ts
const storage = getMemoryMappedRxStorage({
blockSizeLimit: 1000,
storage: getRxStorageIndexedDB()
});
```
## Migrating from other Storages
When you switch from a "normal" persistent storage (like [IndexedDB](./rx-storage-indexeddb.md) or [SQLite](./rx-storage-sqlite.md)) to the memory-mapped storage, you **must** migrate the data using the [Storage Migrator](./migration-storage.md).
You cannot simply switch the storage adapter on an existing database because the memory-mapped storage uses a different internal data structure.
To provide the fast initial page load and low write latency, the memory-mapped storage saves data in a "blockchain-like" structure. Writes are appended in blocks rather than modifying the state in place. These blocks are lazily cleaned up and processed later when the CPU is idle (see [Idle Functions](./rx-database.md#requestidlepromise)).
---
## Instant Performance with Memory Synced RxStorage
import {PremiumBlock} from '@site/src/components/premium-block';
# Memory Synced RxStorage
The memory synced [RxStorage](./rx-storage.md) is a wrapper around any other RxStorage. The wrapper creates an in-memory storage that is used for query and write operations. This memory instance is replicated with the underlying storage for persistence.
The main reason to use this is to improve initial page load and query/write times. This is mostly useful in browser based applications.
## Pros
- Improves read/write performance because these operations run against the in-memory storage.
- Decreases initial page load because it load all data in a single bulk request. It even detects if the database is used for the first time and then it does not have to await the creation of the persistent storage.
## Cons
- It does not support [attachments](./rx-attachment.md).
- When the JavaScript process is killed ungracefully like when the browser crashes or the power of the PC is terminated, it might happen that some memory writes are not persisted to the parent storage. This can be prevented with the `awaitWritePersistence` flag.
- This can only be used if all data fits into the memory of the JavaScript process. This is normally not a problem because a browser has much memory these days and plain json document data is not that big.
- Because it has to await an initial [replication](./replication.md) from the parent storage into the memory, initial page load time can increase when much data is already stored. This is likely not a problem when you store less than `10k` documents.
- The memory-synced storage itself does not support replication and migration. Instead you have to replicate the underlying parent storage.
:::note The memory-synced RxStorage was removed in RxDB version 16
The `memory-synced` was removed in RxDB version 16. Instead consider using the newer and better [memory-mapped RxStorage](./rx-storage-memory-mapped.md) which has better trade-offs and is easier to configure.
:::
## Usage
```ts
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
import {
getMemorySyncedRxStorage
} from 'rxdb-premium/plugins/storage-memory-synced';
/**
* Here we use the IndexedDB RxStorage as persistence storage.
* Any other RxStorage can also be used.
*/
const parentStorage = getRxStorageIndexedDB();
// wrap the persistent storage with the memory synced one.
const storage = getMemorySyncedRxStorage({
storage: parentStorage
});
// create the RxDatabase like you would do with any other RxStorage
const db = await createRxDatabase({
name: 'myDatabase',
storage,
});
/** ... **/
```
## Options
Some options can be provided to fine tune the performance and behavior.
```ts
import {
requestIdlePromise
} from 'rxdb';
const storage = getMemorySyncedRxStorage({
storage: parentStorage,
/**
* Defines how many document
* get replicated in a single batch.
* [default=50]
*
* (optional)
*/
batchSize: 50,
/**
* By default, the parent storage will be created
* without indexes for a faster page load.
* Indexes are not needed because the queries
* will anyway run on the memory storage.
* You can disable this behavior by setting
* keepIndexesOnParent to true.
* If you use the same parent storage for multiple
* RxDatabase instances where one is not
* a asynced-memory storage, you will get the
* error: 'schema not equal to existing storage'
* if you do not set keepIndexesOnParent to true.
*
* (optional)
*/
keepIndexesOnParent: true,
/**
* If set to true, all write operations will resolve AFTER the writes
* have been persisted from the memory to the parentStorage.
* This ensures writes are not lost even if the JavaScript process exits
* between memory writes and the persistence interval.
* default=false
*/
awaitWritePersistence: true,
/**
* After a write, await until the return value of this method resolves
* before replicating with the master storage.
*
* By returning requestIdlePromise() we can ensure that the CPU is idle
* and no other, more important operation is running. By doing so we can be sure
* that the replication does not slow down any rendering of the browser process.
*
* (optional)
*/
waitBeforePersist: () => requestIdlePromise();
});
```
## Replication and Migration with the memory-synced storage
The memory-synced storage itself does not support replication and migration. Instead you have to replicate the underlying parent storage.
For example when you use it on top of an [IndexedDB storage](./rx-storage-indexeddb.md), you have to run replication on that storage instead by creating a different [RxDatabase](./rx-database.md).
```js
const parentStorage = getRxStorageIndexedDB();
const memorySyncedStorage = getMemorySyncedRxStorage({
storage: parentStorage,
keepIndexesOnParent: true
});
const databaseName = 'mydata';
/**
* Create a parent database with the same name+collections
* and use it for replication and migration.
* The parent database must be created BEFORE the memory-synced database
* to ensure migration has already been run.
*/
const parentDatabase = await createRxDatabase({
name: databaseName,
storage: parentStorage
});
await parentDatabase.addCollections(/* ... */);
replicateRxCollection({
collection: parentDatabase.myCollection,
/* ... */
});
/**
* Create an equal memory-synced database with the same name+collections
* and use it for writes and queries.
*/
const memoryDatabase = await createRxDatabase({
name: databaseName,
storage: memorySyncedStorage
});
await memoryDatabase.addCollections(/* ... */);
```
---
## Sharding RxStorage
import {PremiumBlock} from '@site/src/components/premium-block';
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_BROWSER_SHARDING_INDEXEDDB, PERFORMANCE_BROWSER_INDEXEDDB } from '@site/src/components/performance-data';
# Sharding RxStorage
With the sharding plugin, you can improve the write and query times of **some** `RxStorage` implementations.
For example on [slow IndexedDB](./slow-indexeddb.md), a performance gain of **30-50% on reads**, and **25% on writes** can be achieved by using multiple IndexedDB Stores instead of putting all documents into the same store.
The sharding plugin works as a wrapper around any other `RxStorage`. The sharding plugin will automatically create multiple shards per storage instance and it will merge and split read and write calls to it.
## Using the sharding plugin
```ts
import {
getRxStorageSharding
} from 'rxdb-premium/plugins/storage-sharding';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
/**
* First wrap the original RxStorage with the sharding RxStorage.
*/
const shardedRxStorage = getRxStorageSharding({
/**
* Here we use the localStorage RxStorage,
* it is also possible to use any other RxStorage instead.
*/
storage: getRxStorageLocalstorage()
});
/**
* Add the sharding options to your schema.
* Changing these options will require a data migration.
*/
const mySchema = {
/* ... */
sharding: {
/**
* Amount of shards per RxStorage instance.
* Depending on your data size and query
* patterns, the optimal shard amount may differ.
* Do a performance test to optimize that value.
* 10 Shards is a good value to start with.
*
* IMPORTANT: Changing the value of shards is
* not possible on an already existing
* database state,
* you will lose access to your data.
*/
shards: 10,
/**
* Sharding mode,
* you can either shard by collection or by database.
* For most cases you should use 'collection'
* which will shard on the collection level.
* For example with the IndexedDB RxStorage,
* it will then create multiple stores per
* IndexedDB database
* and not multiple IndexedDB databases, which would be slower.
*/
mode: 'collection'
}
/* ... */
}
/**
* Create the RxDatabase with the wrapped RxStorage.
*/
const database = await createRxDatabase({
name: 'mydatabase',
storage: shardedRxStorage
});
```
## Performance
The Sharding [RxStorage](./rx-storage.md) wrapper can improve performance, especially when using an underlying storage that has bottlenecks with large single stores like IndexedDB. Below is a comparison.
---
## Fastest RxDB Starts - Localstorage Meta Optimizer
import {PremiumBlock} from '@site/src/components/premium-block';
# RxStorage Localstorage Meta Optimizer
The [RxStorage](./rx-storage.md) Localstorage Meta Optimizer is a wrapper around any other RxStorage. The wrapper uses the original RxStorage for normal collection documents. But to optimize the initial page load time, it uses [localstorage](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage?retiredLocale=de) to store the plain key-value metadata that RxDB needs to create databases and collections. This plugin can only be used in browsers.
Depending on your database usage and the collection amount, this can save about 200 milliseconds on the initial pageload. It is recommended to use this when you create more than 4 [RxCollections](./rx-collection.md).
## Usage
The meta optimizer gets wrapped around any other RxStorage. It will then automatically detect if an RxDB internal storage instance is created, and replace that with a [localstorage](./articles/localstorage.md) based instance.
```ts
import {
getLocalstorageMetaOptimizerRxStorage
} from 'rxdb-premium/plugins/storage-localstorage-meta-optimizer';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
/**
* First wrap the original RxStorage with the optimizer.
*/
const optimizedRxStorage = getLocalstorageMetaOptimizerRxStorage({
/**
* Here we use the IndexedDB RxStorage,
* it is also possible to use any other RxStorage instead.
*/
storage: getRxStorageIndexedDB()
});
/**
* Create the RxDatabase with the wrapped RxStorage.
*/
const database = await createRxDatabase({
name: 'mydatabase',
storage: optimizedRxStorage
});
```
---
## Seamless Electron Storage with RxDB
import {Faq, FaqItem} from '@site/src/components/faq';
# Electron Plugin
## RxStorage Electron IpcRenderer & IpcMain
To use RxDB in [electron](./electron-database.md), it is recommended to run the RxStorage in the main process and the RxDatabase in the renderer processes. With the rxdb electron plugin you can create a [remote RxStorage](./rx-storage-remote.md) and consume it from the renderer process.
To do this in a convenient way, the RxDB electron plugin provides the helper functions `exposeIpcMainRxStorage` and `getRxStorageIpcRenderer`.
Similar to the [Worker RxStorage](./rx-storage-worker.md), these wrap any other [RxStorage](./rx-storage.md) once in the main process and once in each renderer process. In the renderer you can then use the storage to create a [RxDatabase](./rx-database.md) which communicates with the storage of the main process to store and query data.
:::note
`nodeIntegration` must be enabled in [Electron](https://www.electronjs.org/docs/latest/api/browser-window#new-browserwindowoptions).
:::
```ts
// main.js
const { exposeIpcMainRxStorage } = require('rxdb/plugins/electron');
const { getRxStorageMemory } = require('rxdb/plugins/storage-memory');
app.on('ready', async function () {
exposeIpcMainRxStorage({
key: 'main-storage',
storage: getRxStorageMemory(),
ipcMain: electron.ipcMain
});
});
```
```ts
// renderer.js
const { getRxStorageIpcRenderer } = require('rxdb/plugins/electron');
const { getRxStorageMemory } = require('rxdb/plugins/storage-memory');
const db = await createRxDatabase({
name,
storage: getRxStorageIpcRenderer({
key: 'main-storage',
ipcRenderer: electron.ipcRenderer
})
});
/* ... */
```
## FAQ
You securely create an offline-first Electron application by maintaining strict process isolation: the primary database connection runs securely within the hidden Node.js `main` process, while the vulnerable DOM execution runs in the heavily-restricted `renderer` process. **[RxDB](https://rxdb.info)** automates this architecture precisely via its dedicated Electron IPC plugin (`exposeIpcMainRxStorage`), enabling seamless, non-blocking data synchronization across the IPC boundary while mitigating direct local filesystem exposure to malicious client payloads.
## Related
- [Comparison of Electron Databases](./electron-database.md)
---
## RxDB realtime Sync Engine for Local-First Apps
import { IconGear } from '@site/src/components/icons/gear';
import { HeadlineWithIcon } from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
# }>RxDB's realtime Sync Engine for Local-First Apps
The RxDB Sync Engine provides the ability to sync the database state in **realtime** between the clients and the server.
The backend server does not have to be an RxDB instance; you can build a replication with **any infrastructure**.
For example you can replicate with a [custom GraphQL endpoint](./replication-graphql.md) or an [HTTP server](./replication-http.md) on top of a PostgreSQL or MongoDB database.
The replication is made to support the [Local-First](./articles/local-first-future.md) paradigm, so that when the client goes [offline](./offline-first.md), the RxDB [database](./rx-database.md) can still read and write [locally](./articles/local-database.md) and will continue the replication when the client goes online again.
## Design Decisions of the Sync Engine
In contrast to other (server-side) database replication protocols, the RxDB Sync Engine was designed with these goals in mind:
- **Easy to Understand**: The sync engine works in a simple "git-like" way that is easy to understand for an average developer. You only have to understand how three simple endpoints work.
- **Complex Parts are in RxDB, not in the Backend**: The complex parts of the Sync Engine, like [conflict handling](./transactions-conflicts-revisions.md) or offline-online switches, are implemented inside of RxDB itself. This makes creating a compatible backend very easy.
- **Compatible with any Backend**: Because the complex parts are in RxDB, the backend can be "dumb" which makes the protocol compatible to almost every backend. No matter if you use PostgreSQL, MongoDB or anything else.
- **Performance is optimized for Client Devices and Browsers**: By grouping updates and fetches into batches, it is faster to transfer and easier to compress. Client devices and browsers can also process this data faster, for example running `JSON.parse()` on a chunk of data is faster than calling it once per row. Same goes for how client side storage like [IndexedDB](./rx-storage-indexeddb.md) or [OPFS](./rx-storage-opfs.md) works where writing data in bulks is faster.
- **Offline-First Support**: By incorporating conflict handling at the client side, the protocol fully supports [offline-first apps](./offline-first.md). Users can continue making changes while offline, and those updates will sync seamlessly once a connection is reestablished - all without risking data loss or having undefined behavior.
- **Multi-Tab Support**: When RxDB is used in a browser and multiple tabs of the same application are opened, only exactly one runs the replication at any given time. This reduces client- and backend resources.
## The Sync Engine on the document level
On the [RxDocument](./rx-document.md) level, the replication works like git, where the fork/client contains all new writes and must be merged with the master/server before it can push its new state to the master/server.
```
A---B-----------D master/server state
\ /
B---C---D fork/client state
```
- The client pulls the latest state `B` from the master.
- The client does some changes `C+D`.
- The client pushes these changes to the master by sending the latest known master state `B` and the new client state `D` of the document.
- If the master state is equal to the latest master `B` state of the client, the new client state `D` is set as the latest master state.
- If the master also had changes and so the latest master change is different than the one that the client assumes, we have a conflict that has to be resolved on the client.
## The Sync Engine on the transfer level
When document states are transferred, all handlers use batches of documents for better performance.
The server **must** implement the following methods to be compatible with the replication:
- **pullHandler** Get the last checkpoint (or null) as input. Returns all documents that have been written **after** the given checkpoint. Also returns the checkpoint of the latest written returned document.
- **pushHandler** a method that can be called by the client to send client-side writes to the master. It gets an array with the `assumedMasterState` and the `newForkState` of each document write as input. It must return an array that contains the master document states of all conflicts. If there are no conflicts, it must return an empty array.
- **pullStream** an observable that emits batches of all master writes and the latest checkpoint of the write batches.
```
+--------+ +--------+
| | pullHandler() | |
| |---------------------> | |
| | | |
| | | |
| Client | pushHandler() | Server |
| |---------------------> | |
| | | |
| | pullStream$ | |
| | <-------------------------| |
+--------+ +--------+
```
The replication runs in two **different modes**:
### Checkpoint iteration
On first initial replication, or when the client comes online again, a checkpoint based iteration is used to catch up with the server state.
A checkpoint is a subset of the fields of the last pulled document. When the checkpoint is sent to the backend via `pullHandler()`, the backend must be able to respond with all documents that have been written **after** the given checkpoint.
For example if your documents contain an `id` and an `updatedAt` field, these two can be used as checkpoint.
When the checkpoint iteration reaches the last checkpoint, where the backend returns an empty array because there are no newer documents, the replication will automatically switch to the `event observation` mode.
### Event observation
While the client is connected to the backend, the events from the backend are observed via `pullStream$` and persisted to the client.
If your backend for any reason is not able to provide a full `pullStream$` that contains all events and the checkpoint, you can instead only emit `RESYNC` events that tell RxDB that anything unknown has changed on the server and it should run the pull replication via [checkpoint iteration](#checkpoint-iteration).
When the client goes offline and online again, it might happen that the `pullStream$` has missed out some events. Therefore the `pullStream$` should also emit a `RESYNC` event each time the client reconnects, so that the client can become in sync with the backend via the [checkpoint iteration](#checkpoint-iteration) mode.
## Data layout on the server
To use the replication you first have to ensure that:
- **documents are deterministically sortable by their last write time**
*deterministic* means that even if two documents have the same *last write time*, they have a predictable sort order.
This is most often ensured by using the *primaryKey* as second sort parameter as part of the checkpoint.
- **documents are never deleted, instead the `_deleted` field is set to `true`.**
This is needed so that the deletion state of a document exists in the database and can be replicated to other instances. If your backend uses a different field to mark deleted documents, you have to transform the data in the push/pull handlers or with the modifiers.
For example if your documents look like this:
```ts
const docData = {
"id": "foobar",
"name": "Alice",
"lastName": "Wilson",
/**
* Contains the last write timestamp
* so all document writes can be sorted by that value
* when they are fetched from the remote instance.
*/
"updatedAt": 1564483474,
/**
* Instead of physically deleting documents,
* a deleted document gets replicated.
*/
"_deleted": false
}
```
Then your data is always sortable by `updatedAt`. This ensures that when RxDB fetches 'new' changes via `pullHandler()`, it can send the latest `updatedAt+id` checkpoint to the remote endpoint and then receive all newer documents.
By default, the field is `_deleted`. If your remote endpoint uses a different field to mark deleted documents, you can set the `deletedField` in the replication options which will automatically map the field on all pull and push requests.
## Conflict handling
When multiple clients (or the server) modify the same document at the same time (or when they are offline), it can happen that a conflict arises during the replication.
```
A---B1---C1---X master/server state
\ /
B1---C2 fork/client state
```
In the case above, the client would tell the master to move the document state from `B1` to `C2` by calling `pushHandler()`. But because the actual master state is `C1` and not `B1`, the master would reject the write by sending back the actual master state `C1`.
**RxDB resolves all conflicts on the client** so it would call the conflict handler of the [RxCollection](./rx-collection.md) and create a new document state `D` that can then be written to the master.
```
A---B1---C1---X---D master/server state
\ / \ /
B1---C2---D fork/client state
```
The default conflict handler will always drop the fork state and use the master state. This ensures that clients that are offline for a very long time, do not accidentally overwrite other peoples changes when they go online again.
You can specify a custom conflict handler by setting the property `conflictHandler` when calling `addCollection()`.
Learn how to create a [custom conflict handler](./transactions-conflicts-revisions.md#custom-conflict-handler).
## replicateRxCollection()
You can start the replication of a single `RxCollection` by calling `replicateRxCollection()` like in the following:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
import {
lastOfArray
} from 'rxdb';
const replicationState = await replicateRxCollection({
collection: myRxCollection,
/**
* An id for the replication to identify it
* and so that RxDB is able to resume the replication on app reload.
* If you replicate with a remote server, it is recommended to put the
* server url into the replicationIdentifier.
*/
replicationIdentifier: 'my-rest-replication-to-https://example.com/api/sync',
/**
* By default it will do an ongoing realtime replication.
* By settings live: false the replication will run once until the local state
* is in sync with the remote state, then it will cancel itself.
* (optional), default is true.
*/
live: true,
/**
* Time in milliseconds after when a failed backend request
* has to be retried.
* This time will be skipped if a offline->online switch is detected
* via navigator.onLine
* (optional), default is 5 seconds.
*/
retryTime: 5 * 1000,
/**
* When multiInstance is true, like when you use RxDB in multiple browser tabs,
* the replication should always run in only one of the open browser tabs.
* If waitForLeadership is true, it will wait until
* the current instance is leader.
* If waitForLeadership is false, it will start
* replicating, even if it is not leader.
* [default=true]
*/
waitForLeadership: true,
/**
* If this is set to false,
* the replication will not start automatically
* but will wait for replicationState.start() being called.
* (optional), default is true
*/
autoStart: true,
/**
* Custom deleted field, the boolean property of the document data that
* marks a document as being deleted.
* If your backend uses a different field name
* than '_deleted', set the field name here.
* RxDB will still store the documents internally
* with '_deleted', setting this field
* only maps the data on the data layer.
*
* If a custom deleted field contains a non-boolean value, the deleted state
* of the documents depends on if the value is
* truthy or not. So instead of providing a boolean
* deleted value, you could also work with using a
* 'deletedAt' timestamp instead.
*
* [default='_deleted']
*/
deletedField: 'deleted',
/**
* Optional,
* only needed when you want to replicate local changes to the remote instance.
*/
push: {
/**
* Push handler
*/
async handler(docs) {
/**
* Push the local documents to a remote REST server.
*/
const rawResponse = await fetch('https://example.com/api/sync/push', {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({ docs })
});
/**
* Contains an array with all conflicts that appeared during this push.
* If there were no conflicts, return an empty array.
*/
const response = await rawResponse.json();
return response;
},
/**
* Batch size, optional
* Defines how many documents will be given to the push handler at once.
*/
batchSize: 5,
/**
* Modifies all documents before they are given to the push handler.
* Can be used to swap out a custom deleted
* flag instead of the '_deleted' field.
* If the push modifier return null, the
* document will be skipped and not sent to
* the remote.
* Notice that the modifier can be called
* multiple times and should not contain
* any side effects.
* (optional)
*/
modifier: d => d,
/**
* When a local write happens, the
* replication will normally start pushing
* immediately.
* By providing a function here that returns
* a promise, the replication waits for that
* promise to resolve before starting the
* next upstream persist cycle.
* This lets you batch writes from multiple
* rapid inserts into a single push call,
* or defer pushing until the CPU is idle
* (e.g. via requestIdleCallback).
* NOTE: The longer you wait, the higher
* the risk of losing writes if the
* replication closes unexpectedly.
* (optional)
*/
waitBeforePersist: () => new Promise(resolve => requestIdleCallback(resolve))
},
/**
* Optional,
* only needed when you want to replicate remote changes to the local state.
*/
pull: {
/**
* Pull handler
*/
async handler(lastCheckpoint, batchSize) {
const minTimestamp = lastCheckpoint ? lastCheckpoint.updatedAt : 0;
/**
* In this example we replicate with a remote REST server
*/
const response = await fetch(
`https://example.com/api/sync/` +
`?minUpdatedAt=${minTimestamp}` +
`&limit=${batchSize}`
);
const documentsFromRemote = await response.json();
return {
/**
* Contains the pulled documents from the remote.
* Not that if documentsFromRemote.length < batchSize,
* then RxDB assumes that there are no more un-replicated documents
* on the backend, so the replication
* will switch to 'Event observation'
* mode.
*/
documents: documentsFromRemote,
/**
* The last checkpoint of the returned documents.
* On the next call to the pull handler,
* this checkpoint will be passed as 'lastCheckpoint'
*/
checkpoint: documentsFromRemote.length === 0 ? lastCheckpoint : {
id: lastOfArray(documentsFromRemote).id,
updatedAt: lastOfArray(documentsFromRemote).updatedAt
}
};
},
batchSize: 10,
/**
* Modifies all documents after they have been pulled
* but before they are used by RxDB.
* Notice that the modifier can be called
* multiple times and should not contain
* any side effects.
* (optional)
*/
modifier: d => d,
/**
* Stream of the backend document writes.
* See below.
* You only need a stream$ when you have set live=true
*/
stream$: pullStream$.asObservable()
},
});
/**
* Creating the pull stream for realtime replication.
* Here we use a websocket but any other way of
* sending data to the client can be used,
* like long polling or server-sent events.
*/
const pullStream$ = new Subject>();
let firstOpen = true;
function connectSocket() {
const socket = new WebSocket('wss://example.com/api/sync/stream');
/**
* When the backend sends a new batch of documents+checkpoint,
* emit it into the stream$.
*
* event.data must look like this
* {
* documents: [
* {
* id: 'foobar',
* _deleted: false,
* updatedAt: 1234
* }
* ],
* checkpoint: {
* id: 'foobar',
* updatedAt: 1234
* }
* }
*/
socket.onmessage = event => pullStream$.next(event.data);
/**
* Automatically reconnect the socket on close and error.
*/
socket.onclose = () => connectSocket();
socket.onerror = () => socket.close();
socket.onopen = () => {
if(firstOpen) {
firstOpen = false;
} else {
/**
* When the client is offline and goes online again,
* it might have missed out events that happened on the server.
* So we have to emit a RESYNC so that the replication goes
* into 'Checkpoint iteration' mode until the client is in sync
* and then it will go back into 'Event observation' mode again.
*/
pullStream$.next('RESYNC');
}
}
}
```
## Multi Tab support
For better performance, the replication runs only in one instance when RxDB is used in multiple browser tabs or Node.js processes.
By setting `waitForLeadership: false` you can enforce that each tab runs its own replication cycles.
If used in a multi instance setting, so when at database creation `multiInstance: false` was not set,
you need to import the [leader election plugin](./leader-election.md) so that RxDB can know how many instances exist and which browser tab should run the replication.
## Error handling
When sending a document to the remote fails for any reason, RxDB will send it again in a later point in time.
This happens for **all** errors. The document write could have already reached the remote instance and be processed, while only the answering fails.
The remote instance must be designed to handle this properly and to not crash on duplicate data transmissions.
Depending on your use case, it might be ok to just write the duplicate document data again.
But for a more resilient error handling you could compare the last write timestamps or add a unique write id field to the document. This field can then be used to detect duplicates and ignore re-sent data.
Also the replication has an `.error$` stream that emits all [RxError](./errors.md) objects that arise during replication.
Notice that these errors contain an inner `.parameters.errors` field that contains the original error. Also they contain a `.parameters.direction` field that indicates if the error was thrown during `pull` or `push`. You can use these to properly handle errors. For example when the client is outdated, the server might respond with a `426 Upgrade Required` error code that can then be used to force a page reload.
```ts
replicationState.error$.subscribe((error) => {
if(
error.parameters.errors &&
error.parameters.errors[0] &&
error.parameters.errors[0].code === 426
) {
// client is outdated -> enforce a page reload
location.reload();
}
});
```
## Security
Be aware that client side clocks can never be trusted. When you have a client-backend replication, the backend should overwrite the `updatedAt` timestamp or use another field, when it receives the change from the client.
## RxReplicationState
The function `replicateRxCollection()` returns a `RxReplicationState` that can be used to manage and observe the replication.
### Observable
To observe the replication, the `RxReplicationState` has some `Observable` properties:
```ts
// emits each document that was received from the remote
myRxReplicationState.received$.subscribe(doc => console.dir(doc));
// emits each document that was sent to the remote
myRxReplicationState.sent$.subscribe(doc => console.dir(doc));
// emits all errors that happen when running the push- & pull-handlers.
myRxReplicationState.error$.subscribe(error => console.dir(error));
// emits true when the replication was canceled, false when not.
myRxReplicationState.canceled$.subscribe(bool => console.dir(bool));
// emits true when a replication cycle is running, false when not.
myRxReplicationState.active$.subscribe(bool => console.dir(bool));
// emits each conflict that was reported by the remote in the response
// of the push handler, together with the output of the conflictHandler
// that resolved it.
myRxReplicationState.conflict$.subscribe(conflict => console.dir(conflict));
```
### awaitInitialReplication()
With `awaitInitialReplication()` you can await the initial replication that is done when a full replication cycle was successfully finished for the first time. The returned promise will never resolve if you cancel the replication before the initial replication can be done.
```ts
await myRxReplicationState.awaitInitialReplication();
```
### awaitInSync()
Returns a `Promise` that resolves when:
- `awaitInitialReplication()` has emitted.
- All local data is replicated with the remote.
- No replication cycle is running or in retry-state.
:::warning
When `multiInstance: true` and `waitForLeadership: true` and another tab is already running the replication, `awaitInSync()` will not resolve until the other tab is closed and the replication starts in this tab.
```ts
await myRxReplicationState.awaitInSync();
```
:::
:::warning
#### `awaitInitialReplication()` and `awaitInSync()` should not be used to block the application
A common mistake in RxDB usage is when developers want to block the app usage until the application is in sync.
Often they just `await` the promise of `awaitInitialReplication()` or `awaitInSync()` and show a loading spinner until they resolve. This is dangerous and should not be done because:
- When `multiInstance: true` and `waitForLeadership: true (default)` and another tab is already running the replication, `awaitInitialReplication()` will not resolve until the other tab is closed and the replication starts in this tab.
- Your app can no longer be started when the device is offline because there `awaitInitialReplication()` will never resolve and the app cannot be used.
Instead you should store the last in-sync time in a [local document](./rx-local-document.md) and observe its value on all instances.
For example if you want to block clients from using the app if they have not been in sync for the last 24 hours, you could use this code:
```ts
// update last-in-sync-flag each time replication is in sync
// ensure flag exists
await myCollection.insertLocal(
'last-in-sync', { time: 0 }
).catch();
myReplicationState.active$.pipe(
mergeMap(async() => {
await myReplicationState.awaitInSync();
await myCollection.upsertLocal('last-in-sync', { time: Date.now() })
})
);
// observe the flag and toggle loading spinner
await showLoadingSpinner();
const oneDay = 1000 * 60 * 60 * 24;
await firstValueFrom(
myCollection.getLocal$('last-in-sync').pipe(
filter(d => d.get('time') > (Date.now() - oneDay))
)
);
await hideLoadingSpinner();
```
:::
### awaitDocumentPushed()
Returns a `Promise` that resolves when a specific `RxDocument` instance was successfully pushed to the server.
While `awaitInSync()` waits for the whole collection to be in sync, `awaitDocumentPushed()` only waits for a single document. This is useful when you have a sensitive write (like a financial transaction or a value with a uniqueness constraint) and you want to confirm that exactly this write reached the backend, without blocking on every other unrelated change in the collection.
You pass the `RxDocument` instance you got back from a write operation:
```ts
const doc = await myCollection.insert({ id: 'foobar', value: 10 });
await myReplicationState.awaitDocumentPushed(doc);
// here we know that the document state was pushed to the server
```
A `RxDocument` represents the state of a document at a given point in time, not the document in general. An older or a newer `RxDocument` instance of the same document is not the exact same `RxDocument`, because each instance carries the field values and the internal write time of that specific state. `awaitDocumentPushed()` therefore resolves based on the exact state of the instance you pass in.
It works by comparing the given document state with the last state that was written to the server, which RxDB stores in the replication meta data. Once the given state (or a newer one) was written to the server, the promise resolves.
If the document was overwritten by a newer local write before it could be pushed, the promise resolves as soon as a later state of that document has reached the server.
`awaitDocumentPushed()` does not set a timeout on purpose. If you need one, combine it with `Promise.race()`:
```ts
const doc = await myCollection.insert({ id: 'foobar', value: 10 });
await Promise.race([
myReplicationState.awaitDocumentPushed(doc),
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 5000))
]);
```
### reSync()
Triggers a `RESYNC` cycle where the replication goes into [checkpoint iteration](#checkpoint-iteration) until the client is in sync with the backend. Used in unit tests or when no proper `pull.stream$` can be implemented so that the client only knows that something has been changed but not what.
```ts
myRxReplicationState.reSync();
```
If your backend is not capable of sending events to the client at all, you could run `reSync()` in an interval so that the client will automatically fetch server changes after some time at least.
```ts
// trigger RESYNC each 10 seconds.
setInterval(() => myRxReplicationState.reSync(), 10 * 1000);
```
### cancel()
Cancels the replication. Returns a promise that resolves when everything has been cleaned up.
```ts
await myRxReplicationState.cancel();
```
### pause()
Pauses a running replication. The replication can later be resumed with `RxReplicationState.start()`.
```ts
await myRxReplicationState.pause();
await myRxReplicationState.start(); // restart
```
### remove()
Cancels the replication and deletes the metadata of the replication state. This can be used to restart the replication "from scratch". Calling `.remove()` will only delete the replication metadata, it will NOT delete the documents from the collection of the replication.
```ts
await myRxReplicationState.remove();
```
### isStopped()
Returns `true` if the replication is stopped. This can be if a non-live replication is finished or a replication got canceled.
```js
replicationState.isStopped(); // true/false
```
### isPaused()
Returns `true` if the replication is paused.
```js
replicationState.isPaused(); // true/false
```
### Setting a custom initialCheckpoint
By default, the push replication will start from the beginning of time and push all documents from there to the remote.
By setting a custom `push.initialCheckpoint`, you can tell the replication to only push writes that are newer than the given checkpoint.
```ts
// store the latest checkpoint of a collection
let lastLocalCheckpoint: any;
myCollection.checkpoint$.subscribe(checkpoint => lastLocalCheckpoint = checkpoint);
// start the replication but only push documents
// that are newer than the lastLocalCheckpoint
const replicationState = replicateRxCollection({
collection: myCollection,
replicationIdentifier: 'my-custom-replication-with-init-checkpoint',
/* ... */
push: {
handler: /* ... */,
initialCheckpoint: lastLocalCheckpoint
}
});
```
The same can be done for the other direction by setting a `pull.initialCheckpoint`. Notice that here we need the remote checkpoint from the backend instead of the one from the RxDB storage.
```ts
// get the last pull checkpoint from the server
const lastRemoteCheckpoint = await (
await fetch('http://example.com/pull-checkpoint')
).json();
// start the replication but only pull documents
// that are newer than the lastRemoteCheckpoint
const replicationState = replicateRxCollection({
collection: myCollection,
replicationIdentifier: 'my-custom-replication-with-init-checkpoint',
/* ... */
pull: {
handler: /* ... */,
initialCheckpoint: lastRemoteCheckpoint
}
});
```
### toggleOnDocumentVisible
Ensures replication continues running when the document is `visible`. This helps avoid situations where the leader-elected tab becomes stale or is hibernated by the browser to save battery.
When the tab becomes hidden, replication is automatically paused; when the tab becomes visible again (or the instance becomes leader), replication resumes.
**Default:** `true`
```ts
const replicationState = replicateRxCollection({
toggleOnDocumentVisible: true,
/* ... */
});
```
## Attachment replication
Attachment replication is supported in the RxDB Sync Engine itself. However not all replication plugins support it.
If you start the replication with a collection which has [enabled RxAttachments](./rx-attachment.md) attachment data will be added to all push- and write data.
The pushed documents will contain an `_attachments` object which contains:
- The attachment meta data (id, length, digest) of all non-attachments
- The full attachment data of all attachments that have been updated/added from the client.
- Deleted attachments are spared out in the pushed document.
With this data, the backend can decide onto which attachments must be deleted, added or overwritten.
Accordingly, the pulled document must contain the same data, if the backend has a new document state with updated attachments.
## Pull-Only Replication
With the replication protocol it is possible to do pull only replications where data is pulled from a backend but not pushed from the client. RxDB implements some performance optimizations for these like not storing server metadata on pull only streams.
## Partial Sync with RxDB
RxDB supports partial sync patterns where you dynamically manage multiple replication states for different data scopes. This keeps local storage lean and reduces network overhead. Learn more on the dedicated [Partial Sync](./partial-sync.md) page.
## FAQ
When you have infinite loops in your replication or random re-runs of http requests after some time, the reason is likely that your pull-handler
is crashing. To debug this, add a log to the error$ handler to debug it. `myRxReplicationState.error$.subscribe(err => console.log('error$', err))`.
---
## HTTP Replication
import {Steps} from '@site/src/components/steps';
import {Tabs} from '@site/src/components/tabs';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# HTTP Replication from a custom server to RxDB clients
While RxDB has a range of backend-specific replication plugins (like [GraphQL](./replication-graphql.md) or [Firestore](./replication-firestore.md)), the replication is built in a way to make it very easy to replicate data from a custom server to RxDB clients.
Using **HTTP** as a transport protocol makes it simple to create a compatible backend on top of your existing infrastructure. For events that must be sent from the server to the client, we can use [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events).
In this tutorial we will implement a HTTP replication between an RxDB client and a [MongoDB](./rx-storage-mongodb.md) express server. You can adapt this for any other backend database technology like PostgreSQL or even a non-Node.js server like go or java.
To create a compatible server for replication, we will start a server and implement the correct HTTP routes and replication handlers. We need a push-handler, a pull-handler and for the ongoing changes `pull.stream` we use **Server-Sent Events**.
## Setup
### Start the Replication on the RxDB Client
RxDB does not have a specific HTTP-replication plugin because the [replication primitives plugin](./replication.md) is simple enough to start a HTTP replication on top of it.
We import the `replicateRxCollection` function and start the replication from there for a single [RxCollection](./rx-collection.md).
```ts
// > client.ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: myRxCollection,
replicationIdentifier: 'my-http-replication',
push: { /* add settings from below */ },
pull: { /* add settings from below */ }
});
```
### Start a Node.js process with Express and MongoDB
On the server side, we start an express server that has a MongoDB connection and serves the HTTP requests of the client.
```ts
// > server.ts
import { MongoClient } from 'mongodb';
import express from 'express';
const mongoClient = new MongoClient('mongodb://localhost:27017/');
const mongoConnection = await mongoClient.connect();
const mongoDatabase = mongoConnection.db('myDatabase');
const mongoCollection = await mongoDatabase.collection('myDocs');
const app = express();
app.use(express.json());
/* ... add routes from below */
app.listen(80, () => {
console.log(`Example app listening on port 80`)
});
```
### Implement the Pull Endpoint
As first HTTP Endpoint, we need to implement the pull handler. This is used by the RxDB replication to fetch all documents writes that happened after a given `checkpoint`.
The `checkpoint` format is not determined by RxDB, instead the server can use any type of changepoint that can be used to iterate across document writes. Here we will just use a unix timestamp `updatedAt` and a string `id` which is the most common used format.
When the pull endpoint is called, the server responds with an array of document data based on the given checkpoint and a new checkpoint.
Also the server has to respect the batchSize so that RxDB knows when there are no more new documents and the server returns a non-full array.
```ts
// > server.ts
import { lastOfArray } from 'rxdb/plugins/core';
app.get('/pull', async (req, res) => {
const id = req.query.id;
const updatedAt = parseFloat(req.query.updatedAt);
const documents = await mongoCollection.find({
$or: [
/**
* Notice that we have to compare the updatedAt AND the id field
* because the updateAt field is not unique and when two documents
* have the same updateAt, we can still "sort" them by their id.
*/
{
updatedAt: { $gt: updatedAt }
},
{
updatedAt: { $eq: updatedAt },
id: { $gt: id }
}
]
})
.sort({updatedAt: 1, id: 1})
.limit(parseInt(req.query.batchSize, 10)).toArray();
const newCheckpoint = documents.length === 0 ? { id, updatedAt } : {
id: lastOfArray(documents).id,
updatedAt: lastOfArray(documents).updatedAt
};
res.setHeader('Content-Type', 'application/json');
res.end(JSON.stringify({ documents, checkpoint: newCheckpoint }));
});
```
### Implement the Pull Handler
On the client we add the `pull.handler` to the replication setting. The handler requests the correct server url and fetches the documents.
```ts
// > client.ts
const replicationState = await replicateRxCollection({
/* ... */
pull: {
async handler(checkpointOrNull, batchSize){
const updatedAt = checkpointOrNull ? checkpointOrNull.updatedAt : 0;
const id = checkpointOrNull ? checkpointOrNull.id : '';
const url = 'https://localhost/pull'
+ `?updatedAt=${updatedAt}`
+ `&id=${id}`
+ `&limit=${batchSize}`;
const response = await fetch(url);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
}
/* ... */
});
```
### Implement the Push Endpoint
To send client side writes to the server, we have to implement the `push.handler`. It gets an array of change rows as input and has to return only the conflicting documents that have not been written to the server. Each change row contains a `newDocumentState` and an optional `assumedMasterState`.
For [conflict detection](./transactions-conflicts-revisions.md), on the server we first have to detect if the `assumedMasterState` is correct for each row. If yes, we have to write the new document state to the database, otherwise we have to return the "real" master state in the conflict array.
The server also creates an `event` that is emitted to the `pullStream$` which is later used in the [pull.stream$](#implement-the-pullstream-endpoint).
```ts
// > server.ts
import { lastOfArray } from 'rxdb/plugins/core';
import { Subject } from 'rxjs';
// used in the pull.stream$ below
let lastEventId = 0;
const pullStream$ = new Subject();
app.get('/push', async (req, res) => {
const changeRows = req.body;
const conflicts = [];
const event = {
id: lastEventId++,
documents: [],
checkpoint: null
};
for(const changeRow of changeRows){
const realMasterState =
await mongoCollection.findOne(
{id: changeRow.newDocumentState.id}
);
if(
realMasterState && !changeRow.assumedMasterState ||
(
realMasterState && changeRow.assumedMasterState &&
/*
* For simplicity we detect conflicts
* on the server by only compare the
* updateAt value.
* In reality you might want to do a
* more complex check or do a
* deep-equal comparison.
*/
realMasterState.updatedAt !== changeRow.assumedMasterState.updatedAt
)
) {
// we have a conflict
conflicts.push(realMasterState);
} else {
// no conflict -> write the document
await mongoCollection.updateOne(
{id: changeRow.newDocumentState.id},
changeRow.newDocumentState
);
event.documents.push(changeRow.newDocumentState);
event.checkpoint = {
id: changeRow.newDocumentState.id,
updatedAt:
changeRow.newDocumentState.updatedAt
};
}
}
if(event.documents.length > 0){
myPullStream$.next(event);
}
res.setHeader('Content-Type', 'application/json');
res.end(JSON.stringify(conflicts));
});
```
:::note
For simplicity in this tutorial, we do not use transactions. In reality you should run the full push function inside of a MongoDB transaction to ensure that no other process can mix up the document state while the writes are processed. Also you should call batch operations on MongoDB instead of running the operations for each change row.
:::
### Implement the Push Handler
With the push endpoint in place, we can add a `push.handler` to the replication settings on the client.
```ts
// > client.ts
const replicationState = await replicateRxCollection({
/* ... */
push: {
async handler(changeRows){
const rawResponse = await fetch('https://localhost/push', {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify(changeRows)
});
const conflictsArray = await rawResponse.json();
return conflictsArray;
}
}
/* ... */
});
```
### Implement the pullStream$ Endpoint
While the normal pull handler is used when the replication is in [iteration mode](./replication.md#checkpoint-iteration), we also need a stream of ongoing changes when the replication is in [event observation mode](./replication.md#event-observation). This brings the realtime replication to RxDB where changes on the server or on a client will directly get propagated to the other instances.
On the server we have to implement the `pullStream` route and emit the events. We use the `pullStream$` observable from [above](#implement-the-push-endpoint) to fetch all ongoing events and respond them to the client. Here we use Server-Sent-Events (SSE) which is the most commonly used way to stream data from the server to the client. Other method also exist like [WebSockets or Long-Polling](./articles/websockets-sse-polling-webrtc-webtransport.md).
```ts
// > server.ts
app.get('/pullStream', (req, res) => {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache'
});
const subscription = pullStream$.subscribe(event => {
res.write('data: ' + JSON.stringify(event) + '\n\n');
});
req.on('close', () => subscription.unsubscribe());
});
```
:::note
How to build the `pullStream$` Observable is not part of this tutorial. This heavily depends on your backend and infrastructure. Likely you have to observe the MongoDB event stream.
:::
### Implement the pullStream$ Handler
From the client we can observe this endpoint and create a `pull.stream$` observable that emits all events that are sent from the server to the client.
The client connects to an url and receives server-sent-events that contain all ongoing writes.
```ts
// > client.ts
import { Subject } from 'rxjs';
const myPullStream$ = new Subject();
const eventSource = new EventSource(
'http://localhost/pullStream',
{ withCredentials: true }
);
eventSource.onmessage = event => {
const eventData = JSON.parse(event.data);
myPullStream$.next({
documents: eventData.documents,
checkpoint: eventData.checkpoint
});
};
const replicationState = await replicateRxCollection({
/* ... */
pull: {
/* ... */
stream$: myPullStream$.asObservable()
}
/* ... */
});
```
### pullStream$ RESYNC flag
In case the client loses the connection, the EventSource will automatically reconnect but there might have been some changes that have been missed out in the meantime. The replication has to be informed that it might have missed events by emitting a `RESYNC` flag from the `pull.stream$`.
The replication will then catch up by switching to the [iteration mode](./replication.md#checkpoint-iteration) until it is in sync with the server again.
```ts
// > client.ts
eventSource.onerror = () => myPullStream$.next('RESYNC');
```
The purpose of the `RESYNC` flag is to tell the client that "something might have changed" and then the client can react on that information without having to run operations in an interval.
If your backend is not capable of emitting the actual documents and checkpoint in the pull stream, you could just map all events to the `RESYNC` flag. This would make the replication work with a slight performance drawback:
```ts
// > client.ts
import { Subject } from 'rxjs';
const myPullStream$ = new Subject();
const eventSource = new EventSource(
'http://localhost/pullStream',
{ withCredentials: true }
);
eventSource.onmessage = () => myPullStream$.next('RESYNC');
const replicationState = await replicateRxCollection({
pull: {
stream$: myPullStream$.asObservable()
}
});
```
## Missing implementation details
In this tutorial we only covered the basics of doing a HTTP replication between RxDB clients and a server. We did not cover the following aspects of the implementation:
- Authentication: To authenticate the client on the server, you might want to send authentication headers with the HTTP requests
- Skip events on the `pull.stream$` for the client that caused the changes to improve performance.
- Version upgrades: You should add a version-flag to the endpoint urls. If you then update the version of your endpoints in any way, your old endpoints should emit a `Code 426` to outdated clients so that they can update their client version.
---
## RxDB Server Replication
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# RxDB Server Replication
The *Server Replication Plugin* connects to the [replication](./replication.md) endpoint of an [RxDB Server Replication Endpoint](./rx-server.md#replication-endpoint) and replicates data between the client and the server.
## Usage
The replication server plugin is imported from the `rxdb-server` npm package. Then you start the replication with a given collection and endpoint url by calling `replicateServer()`.
```ts
import { replicateServer } from 'rxdb-server/plugins/replication-server';
const replicationState = await replicateServer({
collection: usersCollection,
replicationIdentifier: 'my-server-replication',
// endpoint url with the servers collection
// schema version at the end
url: 'http://localhost:80/users/0',
headers: {
Authorization: 'Bearer S0VLU0UhI...'
},
push: {},
pull: {},
live: true
});
```
## outdatedClient$
When you update your schema at the server and run a migration, you end up with a different replication url that has a new schema version number at the end.
Your clients might still be running an old version of your application that will no longer be compatible with the endpoint. Therefore when the client tries to call a server endpoint with an outdated schema version, the `outdatedClient$` observable emits to tell your client that the application must be updated. With that event you can tell the client to update the application.
On browser application you might want to just reload the page on that event:
```ts
replicationState.outdatedClient$.subscribe(() => {
location.reload();
});
```
## unauthorized$
When you clients auth data is not valid (or no longer valid), the server will no longer accept any requests from you client and inform the client that the auth headers must be updated.
The `unauthorized$` observable will emit and expects you to update the headers accordingly so that following requests will be accepted again.
```ts
replicationState.unauthorized$.subscribe(() => {
replicationState.setHeaders({
Authorization: 'Bearer S0VLU0UhI...'
});
});
```
## forbidden$
When you client behaves wrong in any case, like update non-allowed values or changing documents that it is not allowed to,
the server will drop the connection and the replication state will emit on the `forbidden$` observable.
It will also automatically stop the replication so that your client does not accidentally DOS attack the server.
```ts
replicationState.forbidden$.subscribe(() => {
console.log('Client is behaving wrong');
});
```
## Custom EventSource implementation
For the server send events, the [eventsource](https://github.com/EventSource/eventsource) npm package is used instead of the native `EventSource` API. We need this because the native browser API does not support sending headers with the request which is required by the server to parse the auth data.
If the eventsource package does not work for you, you can set an own implementation when creating the replication.
```ts
const replicationState = await replicateServer({
/* ... */
eventSource: MyEventSourceConstructor
/* ... */
});
```
---
## GraphQL Replication
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# Replication with GraphQL
The GraphQL replication provides handlers for GraphQL to run [replication](./replication.md) with GraphQL as the transport layer.
The GraphQL replication is mostly used when you already have a backend that exposes a GraphQL API that can be adjusted to serve as a replication endpoint. If you do not already have a GraphQL endpoint, using the [HTTP replication](./replication-http.md) is an easier solution.
:::note
To play around, check out the full example of the RxDB [GraphQL replication with server and client](https://github.com/pubkey/rxdb/tree/master/examples/graphql)
:::
## Usage
Before you use the GraphQL replication, make sure you've learned how the [RxDB replication](./replication.md) works.
### Creating a compatible GraphQL Server
At the server-side, there must exist an endpoint which returns newer rows when the last `checkpoint` is used as input. For example lets say you create a `Query` `pullHuman` which returns a list of document writes that happened after the given checkpoint.
For the push-replication, you also need a `Mutation` `pushHuman` which lets RxDB update data of documents by sending the previous document state and the new client document state.
Also for being able to stream all ongoing events, we need a `Subscription` called `streamHuman`.
```graphql
input HumanInput {
id: ID!,
name: String!,
lastName: String!,
updatedAt: Float!,
deleted: Boolean!
}
type Human {
id: ID!,
name: String!,
lastName: String!,
updatedAt: Float!,
deleted: Boolean!
}
input Checkpoint {
id: String!,
updatedAt: Float!
}
type HumanPullBulk {
documents: [Human]!
checkpoint: Checkpoint
}
type Query {
pullHuman(checkpoint: Checkpoint, limit: Int!): HumanPullBulk!
}
input HumanInputPushRow {
assumedMasterState: HeroInputPushRowT0AssumedMasterStateT0
newDocumentState: HeroInputPushRowT0NewDocumentStateT0!
}
type Mutation {
# Returns a list of all conflicts
# If no document write caused a conflict, return an empty list.
pushHuman(rows: [HumanInputPushRow!]): [Human]
}
# headers are used to authenticate the subscriptions
# over websockets.
input Headers {
AUTH_TOKEN: String!;
}
type Subscription {
streamHuman(headers: Headers): HumanPullBulk!
}
```
The GraphQL resolver for the `pullHuman` would then look like:
```js
const rootValue = {
pullHuman: args => {
const minId = args.checkpoint ? args.checkpoint.id : '';
const minUpdatedAt = args.checkpoint ? args.checkpoint.updatedAt : 0;
// sorted by updatedAt first and the id as second
const sortedDocuments = documents.sort((a, b) => {
if (a.updatedAt > b.updatedAt) return 1;
if (a.updatedAt < b.updatedAt) return -1;
if (a.updatedAt === b.updatedAt) {
if (a.id > b.id) return 1;
if (a.id < b.id) return -1;
else return 0;
}
});
// only return documents newer than the input document
const filterForMinUpdatedAtAndId = sortedDocuments.filter(doc => {
if (doc.updatedAt < minUpdatedAt) return false;
if (doc.updatedAt > minUpdatedAt) return true;
if (doc.updatedAt === minUpdatedAt) {
// if updatedAt is equal, compare by id
if (doc.id > minId) return true;
else return false;
}
});
// only return some documents in one batch
const limitedDocs = filterForMinUpdatedAtAndId.slice(0, args.limit);
// use the last document for the checkpoint
const lastDoc = limitedDocs[limitedDocs.length - 1];
const retCheckpoint = lastDoc ? {
id: lastDoc.id,
updatedAt: lastDoc.updatedAt
} : args.checkpoint;
return {
documents: limitedDocs,
checkpoint: retCheckpoint
};
}
};
```
For examples for the other resolvers, consult the [GraphQL Example Project](https://github.com/pubkey/rxdb/blob/master/examples/graphql/server/index.js).
### RxDB Client
#### Pull replication
For the pull-replication, you first need a `pullQueryBuilder`. This is a function that gets the last replication `checkpoint` and a `limit` as input and returns an object with a GraphQL-query and its variables (or a promise that resolves to the same object). RxDB will use the query builder to construct what is later sent to the GraphQL endpoint.
```js
const pullQueryBuilder = (checkpoint, limit) => {
/**
* The first pull does not have a checkpoint
* so we fill it up with defaults
*/
if (!checkpoint) {
checkpoint = {
id: '',
updatedAt: 0
};
}
const query = `query PullHuman($checkpoint: CheckpointInput, $limit: Int!) {
pullHuman(checkpoint: $checkpoint, limit: $limit) {
documents {
id
name
age
updatedAt
deleted
}
checkpoint {
id
updatedAt
}
}
}`;
return {
query,
operationName: 'PullHuman',
variables: {
checkpoint,
limit
}
};
};
```
With the queryBuilder, you can then setup the pull-replication.
```js
import { replicateGraphQL } from 'rxdb/plugins/replication-graphql';
const replicationState = replicateGraphQL(
{
collection: myRxCollection,
// urls to the GraphQL endpoints
url: {
http: 'http://example.com/graphql'
},
pull: {
queryBuilder: pullQueryBuilder, // the queryBuilder from above
// (optional) modifies all pulled documents
// before they are handled by RxDB
modifier: doc => doc,
// (optional) specifies the object path to
// access the document(s). Otherwise, the
// first result of the response data
// is used.
dataPath: undefined,
/**
* Amount of documents that the remote will send in one request.
* If the response contains less than [batchSize] documents,
* RxDB will assume there are no more changes on the backend
* that are not replicated.
* This value is the same as the limit in the pullHuman() schema.
* [default=100]
*/
batchSize: 50
},
// headers which will be used in http requests against the server.
headers: {
Authorization: 'Bearer abcde...'
},
/**
* Options that have been inherited from the RxReplication
*/
deletedField: 'deleted',
live: true,
retryTime: 1000 * 5,
waitForLeadership: true,
autoStart: true,
}
);
```
#### Push replication
For the push-replication, you also need a `queryBuilder`. Here, the builder receives a changed document as input which has to be sent to the server. It also returns a GraphQL-Query and its data.
```js
const pushQueryBuilder = rows => {
const query = `
mutation PushHuman($writeRows: [HumanInputPushRow!]) {
pushHuman(writeRows: $writeRows) {
id
name
age
updatedAt
deleted
}
}
`;
const variables = {
writeRows: rows
};
return {
query,
operationName: 'PushHuman',
variables
};
};
```
With the queryBuilder, you can then setup the push-replication.
```js
const replicationState = replicateGraphQL(
{
collection: myRxCollection,
// urls to the GraphQL endpoints
url: {
http: 'http://example.com/graphql'
},
push: {
queryBuilder: pushQueryBuilder, // the queryBuilder from above
/**
* batchSize (optional)
* Amount of document that will be pushed
* to the server in a single request.
*/
batchSize: 5,
/**
* modifier (optional)
* Modifies all pushed documents before
* they are sent to the GraphQL endpoint.
* Returning null will skip the document.
*/
modifier: doc => doc
},
headers: {
Authorization: 'Bearer abcde...'
},
pull: {
/* ... */
},
/* ... */
}
);
```
#### Pull Stream
To create a **realtime** replication, you need to create a pull stream that pulls ongoing writes from the server.
The pull stream gets the `headers` of the `RxReplicationState` as input, so that it can be authenticated on the backend.
```js
const pullStreamQueryBuilder = (headers) => {
const query = `subscription onStream($headers: Headers) {
streamHero(headers: $headers) {
documents {
id,
name,
age,
updatedAt,
deleted
},
checkpoint {
id
updatedAt
}
}
}`;
return {
query,
variables: {
headers
}
};
};
```
With the `pullStreamQueryBuilder` you can then start a realtime replication.
```js
const replicationState = replicateGraphQL(
{
collection: myRxCollection,
// urls to the GraphQL endpoints
url: {
http: 'http://example.com/graphql',
// The websocket has to use a different url.
ws: 'ws://example.com/subscriptions'
},
push: {
batchSize: 100,
queryBuilder: pushQueryBuilder
},
headers: {
Authorization: 'Bearer abcde...'
},
pull: {
batchSize: 100,
queryBuilder: pullQueryBuilder,
streamQueryBuilder: pullStreamQueryBuilder,
// Includes headers as connection
// parameter to Websocket.
includeWsHeaders: false,
// Websocket options that can be passed
// as a parameter to initialize the
// subscription
// Can be applied anything from the
// graphql-ws ClientOptions:
// https://the-guild.dev/graphql/ws/docs/client/interfaces/ClientOptions
// Except these parameters: 'url',
// 'shouldRetry', 'webSocketImpl' -
// locked for internal usage
// Note: if you provide connectionParams
// as a wsOption, make sure it returns any
// necessary headers (e.g. authorization)
// because providing your own
// connectionParams prevents headers from
// being included automatically
wsOptions: {
retryAttempts: 10,
}
},
deletedField: 'deleted'
}
);
```
:::note
If it is not possible to create a websocket server on your backend, you can use any other method to pull out the ongoing events from the backend and then you can send them into `RxReplicationState.emitEvent()`.
:::
### Transforming null to undefined in optional fields
GraphQL fills up non-existent optional values with `null` while RxDB required them to be `undefined`.
Therefore, if your schema contains optional properties, you have to transform the pulled data to switch out `null` to `undefined`
```js
const replicationState: RxGraphQLReplicationState = replicateGraphQL(
{
collection: myRxCollection,
url: {/* ... */},
headers: {/* ... */},
push: {/* ... */},
pull: {
queryBuilder: pullQueryBuilder,
modifier: (doc => {
// We have to remove optional non-existent field values
// they are set as null by GraphQL but should be undefined
Object.entries(doc).forEach(([k, v]) => {
if (v === null) {
delete doc[k];
}
});
return doc;
})
},
/* ... */
}
);
```
### pull.responseModifier
With the `pull.responseModifier` you can modify the whole response from the GraphQL endpoint **before** it is processed by RxDB.
For example if your endpoint is not capable of returning a valid checkpoint, but instead only returns the plain document array, you can use the `responseModifier` to aggregate the checkpoint from the returned documents.
```ts
import {
} from 'rxdb';
const replicationState: RxGraphQLReplicationState = replicateGraphQL(
{
collection: myRxCollection,
url: {/* ... */},
headers: {/* ... */},
push: {/* ... */},
pull: {
responseModifier: async function(
plainResponse, // the exact response that was returned from the server
// either 'handler' if plainResponse
// came from the pull.handler,
// or 'stream' if it came from
// the pull.stream
origin,
// if origin==='handler', the
// requestCheckpoint contains the
// checkpoint that was sent to
// the backend
requestCheckpoint
) {
/**
* In this example we aggregate the
* checkpoint from the documents array
* that was returned from the graphql endpoint.
*/
const docs = plainResponse;
return {
documents: docs,
checkpoint: docs.length === 0 ? requestCheckpoint : {
name: lastOfArray(docs).name,
updatedAt: lastOfArray(docs).updatedAt
}
};
}
},
/* ... */
}
);
```
### push.responseModifier
It's also possible to modify the response of a push mutation. For example if your server returns more than just the conflicting docs:
```graphql
type PushResponse {
conflicts: [Human]
conflictMessages: [ReplicationConflictMessage]
}
type Mutation {
# Returns a PushResponse type that contains
# the conflicts along with other information
pushHuman(rows: [HumanInputPushRow!]): PushResponse!
}
```
```ts
import {} from "rxdb";
const replicationState: RxGraphQLReplicationState = replicateGraphQL(
{
collection: myRxCollection,
url: {/* ... */},
headers: {/* ... */},
push: {
responseModifier: async function (plainResponse) {
/**
* In this example we aggregate the
* conflicting documents from a
* response object
*/
return plainResponse.conflicts;
},
},
pull: {/* ... */},
/* ... */
}
);
```
#### Helper Functions
RxDB provides the helper functions `graphQLSchemaFromRxSchema()`, `pullQueryBuilderFromRxSchema()`, `pullStreamBuilderFromRxSchema()` and `pushQueryBuilderFromRxSchema()` that can be used to generate handlers and schemas from the [RxJsonSchema](./rx-schema.md). To learn how to use them, please inspect the [GraphQL Example](https://github.com/pubkey/rxdb/tree/master/examples/graphql).
### RxGraphQLReplicationState
When you call `myCollection.syncGraphQL()` it returns a `RxGraphQLReplicationState` which can be used to subscribe to events, for debugging or other functions. It extends the [RxReplicationState](./replication.md) with some GraphQL specific methods.
#### .setHeaders()
Changes the headers for the replication after it has been set up.
```js
replicationState.setHeaders({
Authorization: `...`
});
```
#### Sending Cookies
The underlying fetch framework uses a `same-origin` policy for credentials by default. That means, cookies and session data is only shared if you backend and frontend run on the same domain and port. Pass the credential parameter to `include` cookies in requests to servers from different origins via:
```js
replicationState.setCredentials('include');
```
or directly pass it in the `replicateGraphQL` function:
```js
replicateGraphQL(
{
collection: myRxCollection,
/* ... */
credentials: 'include',
/* ... */
}
);
```
See [the fetch spec](https://fetch.spec.whatwg.org/#concept-request-credentials-mode) for more information about available options.
:::note
To play around, check out the full example of the RxDB [GraphQL replication with server and client](https://github.com/pubkey/rxdb/tree/master/examples/graphql)
:::
---
## Websocket Replication
import {Faq, FaqItem} from '@site/src/components/faq';
# Websocket Replication
With the websocket replication plugin, you can spawn a websocket server from a RxDB database in Node.js and replicate with it.
:::note
The websocket replication plugin does not have any concept for authentication or permission handling. It is designed to create an easy **server-to-server** replication. It is **not** made for client-server replication. Make a pull request if you need that feature.
:::
## Starting the Websocket Server
```ts
import { createRxDatabase } from 'rxdb';
import {
startWebsocketServer
} from 'rxdb/plugins/replication-websocket';
// create a RxDatabase like normal
const myDatabase = await createRxDatabase({/* ... */});
// start a websocket server
const serverState = await startWebsocketServer({
database: myDatabase,
port: 1337,
path: '/socket'
});
// stop the server
await serverState.close();
```
## Connect to the Websocket Server
The replication has to be started once for each collection that you want to replicate.
```ts
import {
replicateWithWebsocketServer
} from 'rxdb/plugins/replication-websocket';
// start the replication
const replicationState = await replicateWithWebsocketServer({
/**
* To make the replication work,
* the client collection name must be equal
* to the server collection name.
*/
collection: myRxCollection,
url: 'ws://localhost:1337/socket'
});
// stop the replication
await replicationState.cancel();
```
## Customize
We use the [ws](https://www.npmjs.com/package/ws) npm library, so you can use all optional configuration provided by it.
This is especially important to improve performance by opting in of some optional settings.
## FAQ
The WebSocket protocol operates over a persistent, full-duplex TCP connection, fundamentally differing from standard HTTP's stateless, unidirectional request-response model. While traditional HTTP requires the client to initiate every exchange and constantly re-send heavy headers, WebSockets remain open indefinitely, allowing the server to push real-time data events down to the client with mere bytes of overhead, which is vital for high-throughput database [Replication](./replication.md).
WebSockets should be used when your application demands heavy, bidirectional communication such as live chat, multiplayer gaming, or synchronous database multi-master replication. You should opt for [Server-Sent Events (SSE)](./articles/websockets-sse-polling-webrtc-webtransport.md) if your communication is strictly unidirectional (server-to-client) like delivering live sports scores or stock tickers, as SSE works natively over traditional HTTP/1.1 connections without encountering common corporate firewall and proxy blocks.
OpenAI and ChatGPT rely heavily on [Server-Sent Events (SSE)](./articles/websockets-sse-polling-webrtc-webtransport.md) rather than WebSockets to stream their generative text responses. Because LLM generation is inherently a unidirectional operation (the server generating and streaming tokens down to the client after an initial prompt), SSE is the perfect fit. It drastically reduces server overhead by utilizing standard HTTP/1.1 connections and avoids the bidirectional complexities, strict statefulness, and proxy-blocking issues commonly associated with persistent WebSocket streams.
Traditional polling requests data at fixed intervals regardless of state changes, wasting bandwidth and battery. Long polling holds the HTTP connection open until the server has new data or a timeout occurs, simulating a push. Unlike WebSockets, which establish a single, continuous, bidirectional TCP stream, both polling methods are strictly unidirectional and incur the overhead of re-establishing TCP handshakes and heavy HTTP headers for every discrete data event.
Scaling WebSockets requires load balancers that support persistent TCP connections and protocol upgrades (like HAProxy, NGINX, or AWS ALB). Because WebSockets are heavily stateful, you must configure sticky sessions (Session Affinity) to ensure a client's continuous stream routes to the exact same backend node. For distributing real-time replication events globally across multiple horizontal backend nodes, integrating a secondary Pub/Sub mechanism (like Redis Pub/Sub) is highly recommended.
No, Service Worker `fetch` events do not intercept WebSocket connections. The `fetch` event handler in a Service Worker is strictly designed to intercept standard HTTP/HTTPS requests. If you require offline caching or Request interception for your real-time data stream, you must intercept the initial REST calls or implement a custom sync queue inside the Service Worker, which is an architectural pattern databases like **[RxDB](./rx-database.md)** handle inherently on the client side without relying on Service Workers for WS persistence.
WebSockets are significantly faster (lower latency) because they eliminate the HTTP request/response header overhead for every message, pushing data instantly down an open TCP pipe. However, they are more "expensive" on the server side in terms of memory utilization; each open WebSocket connection consumes a dedicated file descriptor and RAM on the server indefinitely, making massive horizontal scaling more complex and costly compared to completely stateless HTTP polling endpoints.
According to the HTTP/1.1 specification (RFC 2616), browsers traditionally limit clients to a maximum of 6 concurrent connections per domain. This limit is rigidly shared across all open tabs. If a user opens 7 tabs connecting to the same WebSocket or SSE endpoint, the 7th tab will stall indefinitely. To bypass this, sophisticated real-time architectures (like **[RxDB](./rx-database.md)**) utilize [Leader Election](./leader-election.md) via the BroadcastChannel API to designate a single tab to maintain the active Socket connection, sharing the data pipeline across all other passive tabs locally.
---
## RxDB's CouchDB Replication Plugin
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# Replication with CouchDB
A plugin to replicate between a [RxCollection](./rx-collection.md) and a CouchDB server.
This plugin uses the RxDB [Sync Engine](./replication.md) to replicate with a CouchDB endpoint. This plugin **does NOT** use the official [CouchDB replication protocol](https://docs.couchdb.org/en/stable/replication/protocol.html) because the CouchDB protocol was optimized for server-to-server replication and is not suitable for fast client side applications, mostly because it has to run many HTTP-requests (at least one per document) and also it has to store the whole revision tree of the documents at the client. This makes initial replication and querying very slow.
Because the way RxDB handles revisions and documents is very similar to CouchDB, using the RxDB replication with a CouchDB endpoint is pretty straightforward.
## Pros
- Faster initial replication.
- Works with any [RxStorage](./rx-storage.md), not just [PouchDB](./rx-storage-pouchdb.md).
- Easier [conflict handling](./transactions-conflicts-revisions.md) because conflicts are handled during replication and not afterwards.
- Does not have to store all document revisions on the client, only stores the newest version.
## Cons
- Does not support the replication of [attachments](./rx-attachment.md).
- Like all CouchDB replication plugins, this one is also limited to replicating 6 collections in parallel. [Read this for workarounds](./replication-couchdb.md#limitations)
## Usage
Start the replication via `replicateCouchDB()`.
```ts
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
// url to the CouchDB endpoint (required)
url: 'http://example.com/db/humans',
/**
* true for live replication,
* false for a one-time replication.
* [default=true]
*/
live: true,
/**
* A custom fetch() method can be provided
* to add authentication or credentials.
* Can be swapped out dynamically
* by running 'replicationState.fetch = newFetchMethod;'.
* (optional)
*/
fetch: myCustomFetchMethod,
pull: {
/**
* Amount of documents to be fetched in one HTTP request
* (optional)
*/
batchSize: 60,
/**
* Custom modifier to mutate pulled documents
* before storing them in RxDB.
* (optional)
*/
modifier: docData => {/* ... */},
/**
* Heartbeat time in milliseconds
* for the long polling of the changestream.
* @link https://docs.couchdb.org/en/3.2.2-docs/api/database/changes.html
* (optional, default=60000)
*/
heartbeat: 60000
},
push: {
/**
* How many local changes to process at once.
* (optional)
*/
batchSize: 60,
/**
* Custom modifier to mutate documents
* before sending them to the CouchDB endpoint.
* (optional)
*/
modifier: docData => {/* ... */}
}
}
);
```
When you call `replicateCouchDB()` it returns a `RxCouchDBReplicationState` which can be used to subscribe to events, for debugging or other functions. It extends the [RxReplicationState](./replication.md) so any other method that can be used there can also be used on the CouchDB replication state.
## Conflict handling
When conflicts appear during replication, the `conflictHandler` of the `RxCollection` is used, equal to the other replication plugins. Read more about conflict handling [here](./replication.md#conflict-handling).
## Auth example
Lets say for authentication you need to add a [bearer token](https://swagger.io/docs/specification/authentication/bearer-authentication/) as HTTP header to each request. You can achieve that by crafting a custom `fetch()` method that adds the header field.
```ts
const myCustomFetch = (url, options) => {
// flat clone the given options to not mutate the input
const optionsWithAuth = Object.assign({}, options);
// ensure the headers property exists
if(!optionsWithAuth.headers) {
optionsWithAuth.headers = {};
}
// add bearer token to headers
optionsWithAuth.headers['Authorization'] ='Basic S0VLU0UhIExFQ0...';
// call the original fetch function with our custom options.
return fetch(
url,
optionsWithAuth
);
};
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
url: 'http://example.com/db/humans',
/**
* Add the custom fetch function here.
*/
fetch: myCustomFetch,
pull: {},
push: {}
}
);
```
Also when your bearer token changes over time, you can set a new custom `fetch` method while the replication is running:
```ts
replicationState.fetch = newCustomFetchMethod;
```
Also there is a helper method `getFetchWithCouchDBAuthorization()` to create a fetch handler with authorization:
```ts
import {
replicateCouchDB,
getFetchWithCouchDBAuthorization
} from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
url: 'http://example.com/db/humans',
/**
* Add the custom fetch function here.
*/
fetch: getFetchWithCouchDBAuthorization('myUsername', 'myPassword'),
pull: {},
push: {}
}
);
```
## Limitations
Since CouchDB only allows synchronization through HTTP/1.1 long polling requests there is a limitation of 6 active synchronization connections before the browser prevents sending any further request. This limitation is at the level of browser per tab per domain (some browser, especially older ones, might have a different limit, [see here](https://docs.pushtechnology.com/cloud/latest/manual/html/designguide/solution/support/connection_limitations.html)).
Since this limitation is at the **browser** level there are several solutions:
- Use only a single database for all entities and set a "type" field for each of the documents
- Create multiple subdomains for CouchDB and use a max of 6 active synchronizations (or less) for each
- Use a proxy (ex: HAProxy) between the browser and CouchDB and configure it to use HTTP/2.0, since HTTP/2.0 multiplexes requests.
If you use nginx in front of your CouchDB, you can use these settings to enable http2-proxying to prevent the connection limit problem:
```
server {
http2 on;
location /db {
rewrite /db/(.*) /$1 break;
proxy_pass http://172.0.0.1:5984;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded;
proxy_set_header Connection "keep_alive";
}
}
```
## Known problems
### Database missing
In contrast to PouchDB, this plugin **does NOT** automatically create missing CouchDB databases.
If your CouchDB server does not have a database yet, you have to create it by yourself by running a `PUT` request to the database `name` url:
```ts
// create a 'humans' CouchDB database on the server
const remoteDatabaseName = 'humans';
await fetch(
'http://example.com/db/' + remoteDatabaseName,
{
method: 'PUT'
}
);
```
## React Native
React Native does not have a global `fetch` method. You have to import fetch method with the [cross-fetch](https://www.npmjs.com/package/cross-fetch) package:
```ts
import crossFetch from 'cross-fetch';
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
url: 'http://example.com/db/humans',
fetch: crossFetch,
pull: {},
push: {}
}
);
```
---
## WebRTC P2P Replication with RxDB - Sync Browsers and Devices
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# P2P WebRTC Replication with RxDB
WebRTC P2P data connections are revolutionizing real-time web and mobile development by **eliminating central servers** in scenarios where clients can communicate directly. With the **RxDB** [Sync Engine](./replication.md), you can sync your local database state across multiple browsers or devices via **WebRTC P2P (Peer-to-Peer)** connections, ensuring scalable, secure, and **low-latency** data flows without traditional server bottlenecks.
## What is WebRTC?
[WebRTC](https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API) stands for Web [Real-Time](./articles/realtime-database.md) Communication. It is an open standard that enables browsers and native apps to exchange audio, video, or **arbitrary data** directly between peers, bypassing a central server after the initial connection is established. WebRTC uses NAT traversal techniques like [ICE](https://developer.liveswitch.io/liveswitch-server/guides/what-are-stun-turn-and-ice.html) (Interactive Connectivity Establishment) to punch through firewalls and establish direct links. This peer-to-peer nature drastically reduces latency while maintaining **high security** and **end-to-end encryption** capabilities.
For a deeper look at comparing WebRTC with **WebSockets** and **WebTransport**, you can read our [comprehensive overview](./articles/websockets-sse-polling-webrtc-webtransport.md). While WebSockets or WebTransport often work in client-server contexts, WebRTC offers direct peer-to-peer connections ideal for fully decentralized data flows.
## Benefits of P2P Sync with WebRTC Compared to Client-Server Architecture
1. **Reduced Latency** - By skipping a central server hop, data travels directly from one client to another, minimizing round-trip times and improving responsiveness.
2. **Scalability** - New peers can join without overloading a central infrastructure. The sync overhead increases linearly with the number of connections rather than requiring a massive server cluster.
3. **Privacy & Ownership** - Data stays within the userβs devices, avoiding risks tied to storing data on third-party servers. This design aligns well with [local-first](./articles/local-first-future.md) or "[zero-latency](./articles/zero-latency-local-first.md)" apps.
4. **Resilience** - In some scenarios, if the central server is unreachable, P2P connections remain operational (assuming a functioning signaling path). Apps can still replicate data among local networks like when they are in the same Wifi or LAN.
5. **Cost Savings** - Reducing the reliance on a high-bandwidth server can cut hosting and bandwidth expenses, particularly in high-traffic or IoT-style use cases.
## Peer-to-Peer (P2P) WebRTC Replication with the RxDB JavaScript Database
Traditionally, real-time data synchronization depends on **centralized servers** to manage and distribute updates. In contrast, RxDBβs WebRTC P2P replication allows data to flow **directly** among clients, removing the server as a data store. This approach is **live** and **fully decentralized**, requiring only a [signaling server](#signaling-server) for initial discovery:
- **No master-slave** concept - each peer hosts its own local RxDB.
- Clients ([browsers](./articles/browser-database.md), devices) connect to each other via WebRTC data channels.
- The [RxDB replication protocol](./replication.md) then handles pushing/pulling document changes across peers.
Because RxDB is a NoSQL database and the replication protocol is straightforward, setting up robust P2P sync is far **easier** than orchestrating a complex client-server database architecture.
## Using RxDB with the WebRTC Replication Plugin
Before you use this plugin, make sure that you understand how [WebRTC works](https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API). Here we build a todo-app that replicates todo-entries between clients:
You can find a fully build example of this at the [RxDB Quickstart Repository](https://github.com/pubkey/rxdb-quickstart) which you can also [try out online](https://pubkey.github.io/rxdb-quickstart/).
First you create the [database](./rx-database.md) and then you can configure the replication:
### Create the Database and Collection
Here we create a database with the [localstorage](./rx-storage-localstorage.md) based storage that stores data inside of the [LocalStorage API](./articles/localstorage.md) in a browser. RxDB has a wide [range of storages](./rx-storage.md) for other JavaScript runtimes.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'myTodoDB',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean', default: false },
created: { type: 'string', format: 'date-time' }
},
required: ['id', 'title', 'done']
}
}
});
// insert an example document
await db.todos.insert({
id: 'todo-1',
title: 'P2P demo task',
done: false,
created: new Date().toISOString()
});
```
### Import the WebRTC replication plugin
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
```
### Start the P2P replication
To start the replication you have to call `replicateWebRTC` on the [collection](./rx-collection.md).
As options you have to provide a `topic` and a connection handler function that implements the `P2PConnectionHandlerCreator` interface. As default you should start with the `getConnectionHandlerSimplePeer` method which uses the [simple-peer](https://github.com/feross/simple-peer) library and comes shipped with RxDB.
```ts
const replicationPool = await replicateWebRTC(
{
// Start the replication for a single collection
collection: db.todos,
// The topic is like a 'room-name'. All clients with the same topic
// will replicate with each other. In most cases you want to use
// a different topic string per user. Also you should prefix the topic with
// a unique identifier for your app, to ensure
// you do not let your users connect
// with other apps that also use the RxDB P2P Replication.
topic: 'my-users-pool',
/**
* You need a collection handler to be able to create WebRTC connections.
* Here we use the simple peer handler which
* uses the 'simple-peer' npm library.
* To learn how to create a custom connection handler, read the source code,
* it is pretty simple.
*/
connectionHandlerCreator: getConnectionHandlerSimplePeer({
// Set the signaling server url.
// You can use the server provided by RxDB for tryouts,
// but in production you should use your own server instead.
signalingServerUrl: 'wss://signaling.rxdb.info/',
// only in Node.js, we need the wrtc library
// because Node.js does not have WebRTC.
// Wrap with createSimplePeerWrtc().
wrtc: createSimplePeerWrtc(
require('node-datachannel/polyfill')
),
// only in Node.js, we need the WebSocket library
// because Node.js does not contain the WebSocket API.
webSocketConstructor: require('ws').WebSocket
}),
pull: {},
push: {}
}
);
```
Notice that in difference to the other [replication plugins](./replication.md), the WebRTC replication returns a `replicationPool` instead of a single `RxReplicationState`. The `replicationPool` contains all replication states of the connected peers in the P2P network.
### Observe Errors
To ensure we log out potential errors, observe the `error$` observable of the pool.
```ts
replicationPool.error$.subscribe(err => console.error('WebRTC Error:', err));
```
### Stop the Replication
You can also dynamically stop the replication.
```ts
replicationPool.cancel();
```
## Live replications
The WebRTC replication is **always live** because there can not be a one-time sync when it is always possible to have new Peers that join the connection pool. Therefore you cannot set the `live: false` option like in the other replication plugins.
## Signaling Server
For P2P replication to work with the RxDB WebRTC Replication Plugin, a [signaling server](https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Signaling_and_video_calling) is required. The signaling server helps peers discover each other and establish connections.
RxDB ships with a default signaling server that can be used with the simple-peer connection handler. This server is made for demonstration purposes and tryouts. It is not reliable and might be offline at any time.
In production you must always use your own signaling server instead!
Creating a basic signaling server is straightforward. The provided example uses 'socket.io' for WebSocket communication. However, in production, you'd want to create a more robust signaling server with authentication and additional logic to suit your application's needs.
Here is a quick example implementation of a signaling server that can be used with the connection handler from `getConnectionHandlerSimplePeer()`:
```ts
import {
startSignalingServerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
const serverState = await startSignalingServerSimplePeer({
port: 8080 // <- port
});
```
For custom signaling servers with more complex logic, you can check the [source code of the default one](https://github.com/pubkey/rxdb/blob/master/src/plugins/replication-webrtc/signaling-server.ts).
## Peer Validation
By default the replication will replicate with every peer the signaling server tells them about.
You can prevent invalid peers from replication by passing a custom `isPeerValid()` function that either returns `true` on valid peers and `false` on invalid peers.
```ts
const replicationPool = await replicateWebRTC(
{
/* ... */
isPeerValid: async (peer) => {
return true;
}
pull: {},
push: {}
/* ... */
}
);
```
## Conflict detection in WebRTC replication
RxDB's conflict handling works by detecting and resolving conflicts that may arise when multiple clients in a decentralized database system attempt to modify the same data concurrently.
A **custom conflict handler** can be set up, which is a plain JavaScript function. The conflict handler is run on each replicated document write and resolves the conflict if required. [Find out more about RxDB conflict handling here](https://rxdb.info/transactions-conflicts-revisions.html)
## Known problems
### SimplePeer requires to have `process.nextTick()`
In the browser you might not have a process variable or process.nextTick() method. But the [simple peer](https://github.com/feross/simple-peer) uses that so you have to polyfill it.
In webpack you can use the `process/browser` package to polyfill it:
```js
const plugins = [
/* ... */
new webpack.ProvidePlugin({
process: 'process/browser',
})
/* ... */
];
```
In angular or other libraries you can add the polyfill manually:
```js
window.process = {
nextTick: (fn, ...args) => setTimeout(() => fn(...args)),
};
```
### Polyfill the WebSocket and WebRTC API in Node.js
While all modern browsers support the WebRTC and WebSocket APIs, they is missing in Node.js which will throw the error `No WebRTC support: Specify opts.wrtc option in this environment`. Therefore you have to polyfill it with a compatible WebRTC and WebSocket polyfill. It is recommended to use the [node-datachannel package](https://github.com/murat-dogan/node-datachannel/tree/master/src/polyfill) for WebRTC which **does not** come with RxDB but has to be installed before via `npm install node-datachannel --save`.
For the Websocket API use the `ws` package that is included into RxDB.
Because the `node-datachannel/polyfill` has read-only properties on `RTCSessionDescription` that are incompatible with `simple-peer`, you must use the `createSimplePeerWrtc()` wrapper:
```ts
import nodeDatachannelPolyfill from 'node-datachannel/polyfill';
import { WebSocket } from 'ws';
import { createSimplePeerWrtc } from 'rxdb/plugins/replication-webrtc';
const replicationPool = await replicateWebRTC(
{
/* ... */
connectionHandlerCreator: getConnectionHandlerSimplePeer({
signalingServerUrl: 'wss://example.com:8080',
wrtc: createSimplePeerWrtc(nodeDatachannelPolyfill),
webSocketConstructor: WebSocket
}),
pull: {},
push: {}
/* ... */
}
);
```
## Storing replicated data encrypted on client device
Storing replicated data encrypted on client devices using the RxDB Encryption Plugin is a pivotal step towards bolstering **data security** and **user privacy**.
The WebRTC replication plugin seamlessly integrates with the [RxDB encryption plugins](./encryption.md), providing a robust solution for encrypting sensitive information before it's stored locally. By doing so, it ensures that even if unauthorized access to the device occurs, the data remains protected and unintelligible without the encryption key (or password). This approach is particularly vital in scenarios where user-generated content or confidential data is replicated across devices, as it empowers users with control over their own data while adhering to stringent security standards. [Read more about the encryption plugins here](./encryption.md).
## FAQ
WebRTC enables true peer-to-peer (P2P) communication by establishing direct UDP/TCP data channels between browsers, completely bypassing centralized database architectures. Because the WebRTC connection requires initial IP discovery, clients must briefly connect to a centralized WebSocket Signaling Server to exchange SDP offers and ICE candidates. Once peered, the **[RxDB WebRTC Replication](./replication.md)** plugin streams NoSQL document diffs and [CRDT](./crdt.md) operations instantly across the channel, providing decentralized real-time sync with absolute zero cloud latency.
RxDB offers comprehensive peer discovery and sync plugins for distributed applications. The WebRTC replication plugin facilitates direct peer-to-peer data synchronization. A signaling server handles initial peer discovery and connection establishment. You connect browsers and mobile apps without a central database server. The sync engine automatically replicates local changes across all discovered peers.
Very few databases support true decentralized peer-to-peer (P2P) synchronization. **[RxDB](./rx-database.md)** is one of the leading options for this architecture, offering a dedicated WebRTC replication plugin that allows direct, client-to-client data synchronization via [WebRTC data channels](./replication-webrtc.md) without routing through a central cloud database. Other notable decentralized tools include **Ditto**, **GunDB**, and CRDT-based libraries like **Yjs** or **Automerge** (though these are often data structure libraries, not fully queryable databases).
## Follow Up
- **Check out the [RxDB Quickstart](./quickstart.md)** to see how to set up your first RxDB database.
- **Explore advanced features** like [Custom Conflict Handling](./transactions-conflicts-revisions.md) or [Offline-First Performance](./rx-storage-performance.md).
- **Try an example** at [RxDB Quickstart GitHub](https://github.com/pubkey/rxdb-quickstart) to see a working P2P Sync setup.
- **Join the RxDB Community** on [GitHub](/code/) or [Discord](/chat/) if you have questions or want to share your P2P WebRTC experiences.
---
## Smooth Firestore Sync for Offline Apps
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# Replication with Firestore from Firebase
With the `replication-firestore` plugin you can do a two-way realtime replication
between your client side [RxDB](./) Database and a [Cloud Firestore](https://firebase.google.com/docs/firestore) database that is hosted on the Firebase platform. It will use the [RxDB Sync Engine](./replication.md) to manage the replication streams, error- and [conflict handling](./transactions-conflicts-revisions.md).
Replicating your Firestore state to RxDB can bring multiple benefits compared to using the Firestore directly:
- It can reduce your cloud fees because your queries run against the local state of the documents without touching a server and writes can be batched up locally and send to the backend in bulks. This is mostly the case for read heavy applications.
- You can run complex [NoSQL queries](./why-nosql.md) on your documents because you are not bound to the [Firestore Query](https://firebase.google.com/docs/firestore/query-data/queries) handling. You can also use local indexes, [compression](./key-compression.md) and [encryption](./encryption.md) and do things like [fulltext search](./fulltext-search.md), fully locally.
- Your application can be truly [Offline-First](./offline-first.md) because your data is stored in a client side database. In contrast Firestore by itself only provides options to support [offline also](https://cloud.google.com/firestore/docs/manage-data/enable-offline) which more works like a cache and requires the user to be online at application start to run authentication.
- It reduces the vendor lock in because you can switch out the backend server afterwards without having to rebuild big parts of the application. RxDB supports replication plugins with multiple technologies and it is even easy to set up with your [custom backend](./replication.md).
- You can use sophisticated [conflict resolution strategies](./replication.md#conflict-handling) so you are not bound to the Firestore [last-write-wins](https://stackoverflow.com/a/47781502/3443137) strategy which is not suitable for many applications.
- The initial load time of your application can be decreased because it will do an incremental replication on restarts.
## Usage
### Install the firebase package
```bash
npm install firebase
```
### Initialize your Firestore Database
```ts
import * as firebase from 'firebase/app';
import {
getFirestore,
collection
} from 'firebase/firestore';
const projectId = 'my-project-id';
const app = firebase.initializeApp({
projectId,
databaseURL: 'http://localhost:8080?ns=' + projectId,
/* ... */
});
const firestoreDatabase = getFirestore(app);
const firestoreCollection = collection(firestoreDatabase, 'my-collection-name');
```
### Start the Replication
Start the replication by calling `replicateFirestore()` on your [RxCollection](./rx-collection.md).
```ts
const replicationState = replicateFirestore({
replicationIdentifier: `https://firestore.googleapis.com/${projectId}`,
collection: myRxCollection,
firestore: {
projectId,
database: firestoreDatabase,
collection: firestoreCollection
},
/**
* (required) Enable push and pull replication with firestore by
* providing an object with optional filter
* for each type of replication desired.
* [default=disabled]
*/
pull: {},
push: {},
/**
* Either do a live or a one-time replication
* [default=true]
*/
live: true,
/**
* (optional) likely you should just use the default.
*
* In firestore it is not possible to read out
* the internally used write timestamp of a document.
* Even if we could read it out, it is not indexed which
* is required for fetch 'changes-since-x'.
* So instead we have to rely on a custom user defined field
* that contains the server time
* which is set by firestore via serverTimestamp()
* Notice that the serverTimestampField MUST NOT be
* part of the collections RxJsonSchema!
* [default='serverTimestamp']
*/
serverTimestampField: 'serverTimestamp'
});
```
To observe and cancel the replication, you can use any other methods from the [ReplicationState](./replication.md) like `error$`, `cancel()` and `awaitInitialReplication()`.
## Handling deletes
RxDB requires you to never [fully delete documents](./replication.md#data-layout-on-the-server). This is needed to be able to replicate the deletion state of a document to other instances. The firestore replication will set a boolean `_deleted` field to all documents to indicate the deletion state. You can change this by setting a different `deletedField` in the sync options.
## Do not set `enableIndexedDbPersistence()`
Firestore has the `enableIndexedDbPersistence()` feature which caches document states locally to [IndexedDB](./rx-storage-indexeddb.md). This is not needed when you replicate your Firestore with RxDB because RxDB itself will store the data locally already.
## Using the replication with an already existing Firestore Database State
If you have not used RxDB before and you already have documents inside of your Firestore database, you have
to manually set the `_deleted` field to `false` and the `serverTimestamp` to all existing documents.
```ts
import {
getDocs,
query,
where,
serverTimestamp
} from 'firebase/firestore';
const allDocsResult = await getDocs(query(firestoreCollection));
allDocsResult.forEach(doc => {
doc.update({
_deleted: false,
serverTimestamp: serverTimestamp()
})
});
```
Also notice that if you do writes from non-RxDB applications, you have to keep these fields in sync. It is recommended to use the [Firestore triggers](https://firebase.google.com/docs/functions/firestore-events) to ensure that.
## Filtered Replication
You might need to replicate only a subset of your collection, either to or from Firestore. You can achieve this using `push.filter` and `pull.filter` options.
```ts
const replicationState = replicateFirestore(
{
collection: myRxCollection,
firestore: {
projectId,
database: firestoreDatabase,
collection: firestoreCollection
},
pull: {
filter: [
where('ownerId', '==', userId)
]
},
push: {
filter: (item) => item.syncEnabled === true
}
}
);
```
Keep in mind that you can not use inequality operators `(<, <=, !=, not-in, >, or >=)` in `pull.filter` since that would cause a conflict with ordering by `serverTimestamp`.
---
## MongoDB Realtime Sync Engine for Local-First Apps
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {RxdbMongoDiagramPlain} from '@site/src/components/mongodb-sync';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# MongoDB Replication Plugin
The [MongoDB](https://www.mongodb.com/) Replication Plugin for RxDB delivers seamless, two-way synchronization between [MongoDB](./rx-storage-mongodb.md) and RxDB, enabling [real-time](./articles/realtime-database.md) updates and [offline-first](./offline-first.md) functionality for your applications. Built on **MongoDB Change Streams**, it supports both Atlas and self-hosted deployments, ensuring your data stays consistent across every device and service.
Behind the scenes, the plugin is powered by the RxDB [Sync Engine](./replication.md), which manages the complexities of real-world data replication for you. It automatically handles [conflict detection and resolution](./transactions-conflicts-revisions.md), maintains precise checkpoints for incremental updates, and gracefully manages transitions between offline and online states. This means you don't need to manually implement retry logic, reconcile divergent changes, or worry about data loss during connectivity drops, the Sync Engine ensures consistency and reliability in every sync cycle.
## Key Features
- **Two-way replication** between MongoDB and RxDB collections
- **Offline-first support** with automatic incremental re-sync
- **Incremental updates** via MongoDB Change Streams
- **Conflict resolution** handled by the RxDB Sync Engine
- **Atlas and self-hosted support** for replica sets and sharded clusters
## Architecture Overview
The plugin operates in a three-tier architecture: Clients connect to [RxServer](./rx-server.md), which in turn connects to MongoDB. RxServer streams changes from MongoDB to connected clients and pushes client-side updates back to MongoDB.
For the client side, RxServer exposes a [replication endpoint](./rx-server.md#replication-endpoint) over WebSocket or HTTP, which your RxDB-powered applications can consume.
The following diagram illustrates the flow of updates between clients, RxServer, and MongoDB in a live synchronization setup:
:::note
The MongoDB Replication Plugin is optimized for Node.js environments (e.g., when RxDB runs within RxServer or other backend services). Direct connections from browsers or mobile apps to MongoDB are not supported because MongoDB does not use HTTP as its wire protocol and requires a driver-level connection to a replica set or sharded cluster.
:::
## Setting up the Client-RxServer-MongoDB Sync
### Install the Client Dependencies
In your JavaScript project, install the RxDB libraries and the MongoDB node.js driver:
```npm install rxdb rxdb-server mongodb --save```
### Set up a MongoDB Server
As first step, you need access to a running
MongoDB Server. This can be done by either
running a server locally or using the Atlas
Cloud. Notice that we need to have a
[replica set](https://www.mongodb.com/docs/manual/tutorial/deploy-replica-set/)
because only on these, the MongoDB changestream
can be used.
### Shell
If you have installed MongoDB locally, you can start the server with this command:
```mongod --replSet rs0 --bind_ip_all```
### Docker
If you have docker installed, you can start a container that runs the MongoDB server:
```docker run -p 27017:27017 -p 27018:27018 -p 27019:27019 --rm --name rxdb-mongodb mongo:8.0.4 mongod --replSet rs0 --bind_ip_all```
### MongoDB Atlas
Learn here how to create a MongoDB atlas
account and how to start a MongoDB cluster
that runs in the cloud:
After this step you should have a valid
connection string that points to a running
MongoDB Server like
`mongodb://localhost:27017/`.
### Create a MongoDB Database and Collection
On your MongoDB server, make sure to create a database and a collection.
```ts
//> server.ts
import { MongoClient } from 'mongodb';
const mongoClient = new MongoClient(
'mongodb://localhost:27017/?directConnection=true'
);
const mongoDatabase = mongoClient.db('my-database');
await mongoDatabase.createCollection('my-collection', {
changeStreamPreAndPostImages: { enabled: true }
});
```
:::note
To observe document deletions on the
changestream, `changeStreamPreAndPostImages`
must be enabled. This is not required if you
have an insert/update-only collection where
no documents are deleted ever.
:::
### Create a RxDB Database and Collection
Now we create an RxDB
[database](./rx-database.md) and a
[collection](./rx-collection.md). In this
example the
[memory storage](./rx-storage-memory.md),
in production you would use a
[persistent storage](./rx-storage.md) instead.
```ts
//> server.ts
import { createRxDatabase, addRxPlugin } from 'rxdb';
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
// Create server-side RxDB instance
const db = await createRxDatabase({
name: 'serverdb',
storage: getRxStorageMemory()
});
// Add your collection schema
await db.addCollections({
humans: {
schema: {
version: 0,
primaryKey: 'passportId',
type: 'object',
properties: {
passportId: { type: 'string', maxLength: 100 },
firstName: { type: 'string' },
lastName: { type: 'string' }
},
required: ['passportId', 'firstName', 'lastName']
}
}
});
```
### Sync the Collection with the MongoDB Server
Now we can start a
[replication](./replication.md) that does a
two-way replication between the RxDB
Collection and the MongoDB Collection.
```ts
//> server.ts
import { replicateMongoDB } from 'rxdb/plugins/replication-mongodb';
const replicationState = replicateMongoDB({
mongodb: {
collectionName: 'my-collection',
connection: 'mongodb://localhost:27017',
databaseName: 'my-database'
},
collection: db.humans,
replicationIdentifier: 'humans-mongodb-sync',
pull: { batchSize: 50 },
push: { batchSize: 50 },
live: true
});
```
:::note You can do many things with the replication state
The `RxMongoDBReplicationState` which is
returned from `replicateMongoDB()` allows
you to run all functionality of the normal
[RxReplicationState](./replication.md) like
observing errors or doing start/stop
operations.
:::
### Start a RxServer
Now that we have a RxDatabase and Collection
that is replicated with MongoDB, we can
spawn a [RxServer](./rx-server.md) on top
of it. This server can then be used by
client devices to connect.
```ts
//> server.ts
import { createRxServer } from 'rxdb-server/plugins/server';
import { RxServerAdapterExpress } from 'rxdb-server/plugins/adapter-express';
const server = await createRxServer({
database: db,
adapter: RxServerAdapterExpress,
port: 8080,
cors: '*'
});
const endpoint = server.addReplicationEndpoint({
name: 'humans',
collection: db.humans
});
console.log('Replication endpoint:', `http://localhost:8080/${endpoint.urlPath}`);
// do not forget to start the server!
await server.start();
```
### Sync a Client with the RxServer
On the client-side we create the exact same
RxDatabase and collection and then replicate
it with the replication endpoint of the
RxServer.
```ts
//> client.ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import { replicateServer } from 'rxdb-server/plugins/replication-server';
const db = await createRxDatabase({
name: 'mydb-client',
storage: getRxStorageDexie()
});
await db.addCollections({
humans: {
schema: {
version: 0,
primaryKey: 'passportId',
type: 'object',
properties: {
passportId: { type: 'string', maxLength: 100 },
firstName: { type: 'string' },
lastName: { type: 'string' }
},
required: ['passportId', 'firstName', 'lastName']
}
}
});
// Start replication to the RxServer endpoint printed by the server:
// e.g. http://localhost:8080/humans/0
const replicationState = replicateServer({
replicationIdentifier: 'humans-rxserver',
collection: db.humans,
url: 'http://localhost:8080/humans/0',
live: true,
pull: { batchSize: 50 },
push: { batchSize: 50 }
});
```
## Follow Up
- Try it out with the RxDB-MongoDB
[example repository](https://github.com/pubkey/rxdb-mongodb-sync-example)
- Read [From Local to Global: Scalable
Edge Apps with RxDB + MongoDB][1]
[1]: https://www.mongodb.com/company/blog/innovation/from-local-global-scalable-edge-apps-rxdb
- [Replication API Reference](./replication.md)
- [RxServer Documentation](./rx-server.md)
- Join our [Discord Forum](./chat) for questions and feedback
---
## Supabase Replication Plugin for RxDB - Real-Time, Offline-First Sync
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {RxdbMongoDiagramPlain} from '@site/src/components/mongodb-sync';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
# Supabase Replication Plugin
The **Supabase Replication Plugin** for RxDB delivers seamless, two-way synchronization between your RxDB collections and a Supabase (Postgres) table. It uses **PostgREST** for pull/push and **Supabase Realtime** (logical replication) to stream live updates, so your data stays consistent across devices with first-class [local-first](./articles/local-first-future.md), offline-ready support.
Under the hood, the plugin is powered by the RxDB [Sync Engine](./replication.md). It handles checkpointed incremental pulls, robust retry logic, and [conflict detection/resolution](./transactions-conflicts-revisions.md) for you. You focus on features, and RxDB takes care of sync.
## Key Features of the RxDB-Supabase Plugin
- **Cloud Only Backend**: No self-hosted server required. Client devices directly sync with the Supabase Servers.
- **Two-way replication** between Supabase tables and RxDB [collections](./rx-collection.md)
- **Offline-first** with resumable, incremental sync
- **Live updates** via Supabase Realtime channels
- **Conflict resolution** handled by the [RxDB Sync Engine](./replication.md)
- **Works in browsers and Node.js** with `@supabase/supabase-js`
## Architecture Overview
Clients connect **directly to Supabase** using the official JS client. The plugin:
- **Pulls** documents over PostgREST using a checkpoint `(modified, id)` and deterministic ordering.
- **Pushes** inserts/updates using optimistic concurrency guards.
- **Streams** new changes using Supabase Realtime so live replication stays up to date.
:::note
Because Supabase exposes Postgres over **HTTP/WebSocket**, you can safely replicate from browsers and mobile apps. Protect your data with **Row Level Security (RLS)** policies; use the **anon** key on clients and the **service role** key only on trusted servers.
:::
## Setting up RxDB β Supabase Sync
### Install Dependencies
```bash
npm install rxdb @supabase/supabase-js
```
### Create a Supabase Project & Table
In your supabase project, create a new table. Ensure that:
- The primary key must have the type text (Primary keys must always be strings in RxDB)
- You have an modified field which stores the last modification timestamp of a row (default is `_modified`)
- You have a boolean field which stores if a row should is "deleted". You should not hard-delete rows in Supabase, because clients would miss the deletion if they haven't been online at the deletion time. Instead, use a deleted `boolean` to mark rows as deleted. This way all clients can still pull the deletion, and RxDB will hide the complexity on the client side.
- Enable the realtime observation of writes to the table.
Here is an example for a "human" table:
```sql
create extension if not exists moddatetime schema extensions;
create table "public"."humans" (
"passportId" text primary key,
"firstName" text not null,
"lastName" text not null,
"age" integer,
"_deleted" boolean DEFAULT false NOT NULL,
"_modified" timestamp with time zone DEFAULT now() NOT NULL
);
-- auto-update the _modified timestamp
CREATE TRIGGER update_modified_datetime BEFORE UPDATE ON public.humans FOR EACH ROW
EXECUTE FUNCTION extensions.moddatetime('_modified');
-- add a table to the publication so we can subscribe to changes
alter publication supabase_realtime add table "public"."humans";
```
### Create an RxDB Database & Collection
Create a normal RxDB database, then add a collection whose **schema mirrors your Supabase table**. The **primary key must match** (same column name and type), and fields should be **top-level simple types** (string/number/boolean). You donβt need to model server internals: the plugin maps the serverβs \_deleted flag to doc.\_deleted automatically, and \_modified is optional in your schema (the plugin strips it on push and will include it on pull only if you define it). For browsers use a persistent storage like Localstorage or IndexedDB. For tests you can use the [in-memory storage](./rx-storage-memory.md).
```ts
// client
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
export const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
humans: {
schema: {
version: 0,
primaryKey: 'passportId',
type: 'object',
properties: {
passportId: { type: 'string', maxLength: 100 },
firstName: { type: 'string' },
lastName: { type: 'string' },
age: { type: 'number' }
},
required: ['passportId', 'firstName', 'lastName']
}
}
});
```
### Create the Supabase Client
Make a single Supabase client and reuse it across your app. In the browser, use the anon key (RLS-protected). On trusted servers you may use the service role key, but never ship that to clients.
#### Production
```ts
//> client
import { createClient } from '@supabase/supabase-js';
export const supabase = createClient(
'https://xyzcompany.supabase.co',
'eyJhbGciOi...'
);
```
#### Vite
```ts
//> client
import { createClient } from '@supabase/supabase-js';
export const supabase = createClient(
import.meta.env.VITE_SUPABASE_URL!, // e.g. https://xyzcompany.supabase.co
import.meta.env.VITE_SUPABASE_ANON_KEY! // anon key for browsers
// optional options object here
);
```
#### Local Development
```ts
//> client
import { createClient } from '@supabase/supabase-js';
export const supabase = createClient(
'http://127.0.0.1:54321',
'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...'
);
```
### Start Replication
Connect your RxDB collection to the Supabase table to start the replication.
```ts
//> client
import { replicateSupabase } from 'rxdb/plugins/replication-supabase';
const replication = replicateSupabase({
tableName: 'humans',
client: supabase,
collection: db.humans,
replicationIdentifier: 'humans-supabase',
live: true,
pull: {
batchSize: 50,
// optional: shape incoming docs
modifier: (doc) => {
// map nullable age-field
if (!doc.age) delete doc.age;
return doc;
}
// optional: customize the pull query before fetching
queryBuilder: ({ query }) => {
// Add filters, joins, or other PostgREST query modifiers
// This runs before checkpoint filtering and ordering
return query.eq("status", "active");
},
},
push: {
batchSize: 50
},
// optional overrides if your column names differ:
// modifiedField: '_modified',
// deletedField: '_deleted'
});
// (optional) observe errors and wait for the first sync barrier
replication.error$.subscribe(err => console.error('[replication]', err));
await replication.awaitInitialReplication();
```
:::note Nullable values must be mapped
Supabase returns `null` for nullable columns, but in RxDB you often model those fields as optional (i.e., they can be undefined/missing). To avoid schema errors, map `null` β `undefined` in the `pull.modifier` (usually by deleting the key).
:::
## Using Joins
You can use the `pull.queryBuilder` to use joins and also pull data from related tables.
To do that, you have to create a **new** query object in the `pull.queryBuilder` with the `.select()` method and return it.
```ts
const replication = replicateSupabase({
pull: {
queryBuilder: (/* ignore the passed query instance from here */) => {
/**
* Create a totally new query instance
* and return that.
*/
return supabase.from('humans').select('*, pets(*), toys(*)');
}
}
});
```
### Do other things with the replication state
The `RxSupabaseReplicationState` which is returned from `replicateSupabase()` allows you to run all functionality of the normal [RxReplicationState](./replication.md).
## FAQ
Supabase and RxDB offer the best of both worlds for Node.js and TypeScript applications. Supabase provides a powerful PostgreSQL backend. You can use PostgreSQL as a document database by storing JSON data. RxDB provides a reactive local document store. You achieve real-time synchronization between the local document store and the Supabase backend. You benefit from strong TypeScript typing on the client and robust SQL querying on the server.
You connect an anonymous key to a Supabase project by initializing the official `@supabase/supabase-js` client utilizing your project's `SUPABASE_URL` and `SUPABASE_ANON_KEY`. In frontend applications interacting with the **[RxDB Supabase Replication](./replication.md)** plugin, you must inject the `anon` key, while simultaneously configuring strict Row Level Security (RLS) policies within your Supabase PostgreSQL backend to prevent unauthorized data manipulation.
Natively, the Supabase JavaScript client does not support advanced [offline-first](./offline-first.md) synchronization pipelines or complex Conflict-free Replicated Data Type (CRDT) architectures. To implement full offline sync capable of continuous background disconnected writes, you must attach the **[RxDB](./rx-database.md)** Supabase Replication Plugin. RxDB acts as the offline-first local CRDT-like cache, deferring all local mutations into a unified outbound queue until the Supabase TCP connection is restored.
Yes, Row Level Security (RLS) is strictly mandatory whenever you expose a Supabase database directly to the frontend. Without RLS, the anonymous `anon` key used by the **[RxDB](./rx-database.md)** client grants full read and write access to your entire PostgreSQL cluster. You must configure RLS policies that enforce `auth.uid() = user_id` checks to guarantee clients only replicate and mutate their own specific documents.
Supabase Realtime acts as an Elixir-based WebSocket broadcasting server that taps directly into PostgreSQL's logical replication stream. When a row changes on the database, the Realtime server parses the WAL (Write-Ahead Log) and pushes the event down to subscribed clients. The **[RxDB Supabase Replication](./replication.md)** plugin leverages this WebSocket channel strictly for live change detection, triggering rapid localized pulls over PostgREST to guarantee no data is dropped during connection turbulence.
## Follow Up
- **Replication API Reference:** Learn the core concepts and lifecycle hooks - [Replication](./replication.md)
- **Offline-First Guide:** Caching, retries, and conflict strategies - [Local-First](./articles/local-first-future.md)
- **Supabase Crash Course:** Build a React Native app with RxDB - [Webinar Video](https://www.youtube.com/watch?v=F051fX1z6lE)
- Row Level Security (RLS) - https://supabase.com/docs/guides/auth/row-level-security
- Realtime - https://supabase.com/docs/guides/realtime
- Local dev with the Supabase CLI - https://supabase.com/docs/guides/cli
- **Community:** Questions or feedback? Join our Discord - [Chat](./chat)
---
## Google Drive Sync
import {Steps} from '@site/src/components/steps';
import {BetaBlock} from '@site/src/components/beta-block';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
# Replication with Google Drive
The `replication-google-drive` plugin allows you to replicate your client-side [RxDB](./) database to a folder in the user's Google Drive. This enables cross-device [sync](./replication.md) for single users without requiring any backend server.
## Overview
The replication uses the Google Drive API v3 and v2.
- **[Offline-First](./offline-first.md):** Users can work offline. Changes are synced when they go online.
- **No Backend Required:** You don't need to host your own database server.
- **Cross-Device:** Users can access their data from multiple devices by signing into the same Google account.
- **Realtime Sync:** Uses [WebRTC](./replication-webrtc.md) for peer-to-peer signaling to achieve near real-time updates. Uses the same google-drive folder instead of a signaling-server.
## Usage
### Enable Google Drive API
You need to enable the Google Drive API in the [Google Cloud Console](https://console.cloud.google.com/) and create credentials (OAuth 2.0 Client ID) for your application.
### Authenticate the User
Your application must handle the OAuth flow to get an `accessToken` from Google. You can use libraries like [`@react-oauth/google`](https://www.npmjs.com/package/@react-oauth/google) or the Google Identity Services SDK.
### Start Replication
Once you have the `accessToken`, you can start the replication.
```ts
import { replicateGoogleDrive } from 'rxdb/plugins/replication-google-drive';
const replicationState = await replicateGoogleDrive({
replicationIdentifier: 'my-app-drive-sync',
collection: myRxCollection, // RxCollection
googleDrive: {
oauthClientId: 'YOUR_GOOGLE_CLIENT_ID',
authToken: 'USER_ACCESS_TOKEN',
folderPath: 'my-app-data/user-1'
},
live: true,
pull: {
batchSize: 60,
modifier: doc => doc // (optional) modify invalid data
},
push: {
batchSize: 60,
modifier: doc => doc // (optional) modify before sending
}
});
// Observe replication states
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
replicationState.awaitInitialReplication().then(() => {
console.log('Initial replication done');
});
```
## Signaling & WebRTC
Google Drive does not provide real-time events for file changes. If a user changes data on **User Device A**, **User Device B** would not know about it until it periodically polls the Drive API.
To achieve real-time updates, this plugin uses **WebRTC** to signal changes between connected devices.
1. Devices create "signal files" in a `signaling` subfolder on Google Drive.
2. Other devices detect these files, read the WebRTC connection data, and establish a direct P2P connection with each other.
3. When a device makes a write, it sends a "RESYNC" signal via WebRTC to all connected peers to notify them about the change.
### Polyfill for Node.js
WebRTC is native in browsers but requires a polyfill in Node.js. Use `createSimplePeerWrtc()` to wrap the polyfill for compatibility with `simple-peer`:
```ts
import nodeDatachannelPolyfill from 'node-datachannel/polyfill';
import { createSimplePeerWrtc } from 'rxdb/plugins/replication-webrtc';
// ...
const replicationState = await replicateGoogleDrive({
// ...
signalingOptions: {
wrtc: createSimplePeerWrtc(nodeDatachannelPolyfill)
}
});
```
## Options
### googleDrive
- **oauthClientId** `string`: The OAuth 2.0 Client ID of your application.
- **authToken** `string`: The valid access token associated with the user.
- **folderPath** `string`: The path to the folder in Google Drive where data should be stored.
- The plugin will ensure this folder exists.
- For the default `drive` space it must **not** be the root folder.
- For the `appDataFolder` space it is optional and interpreted relative to the appDataFolder root.
- It creates subfolders `docs` (for data) and `signaling` (for WebRTC).
- **space** `'drive' | 'appDataFolder'` (optional): Defaults to `'drive'`. See the section below.
- **apiEndpoint** `string` (optional): Defaults to `https://www.googleapis.com`. Useful for mocking or proxies.
- **transactionTimeout** `number` (optional): Default `10000` (10s). The plugin uses a `transaction` file in Drive to ensure data integrity during writes. This is the timeout after which a lock is considered stale.
### Using the appDataFolder
By default the plugin stores data in the user visible "My Drive". Set `space: 'appDataFolder'` to store data in Google Drive's hidden, per-application data folder instead. This is useful on Android and other clients where you want app state synced across the user's devices without cluttering their Drive UI.
```ts
const replicationState = await replicateGoogleDrive({
replicationIdentifier: 'my-app-drive-sync',
collection: myRxCollection,
googleDrive: {
oauthClientId: 'YOUR_GOOGLE_CLIENT_ID',
authToken: 'USER_ACCESS_TOKEN',
space: 'appDataFolder'
// folderPath is optional here
},
live: true,
pull: {},
push: {}
});
```
When using `appDataFolder`:
- Request the `https://www.googleapis.com/auth/drive.appdata` OAuth scope when you authenticate the user. The regular Drive scopes do not grant access to this space.
- The data is hidden from the user's Drive UI and cannot be browsed manually.
- The folder is isolated per OAuth client id. A debug build and a release build with different client ids will not see each other's data.
- `folderPath` is optional. If omitted, the `docs` and `signaling` subfolders are created directly in the appDataFolder root.
### attachments
Controls whether binary [attachment](./rx-attachment.md) data is replicated along with documents.
- **Default**: enabled automatically when the collection schema has `attachments: {}` defined.
- Set `attachments: false` to disable attachment replication (only document fields are synced).
```ts
const replicationState = await replicateGoogleDrive({
// ...
attachments: false, // disable attachment replication
pull: {},
push: {}
});
```
When attachment replication is enabled, attachment binary data is encoded as base64 and stored in a separate `_attachments_data` field inside the document's JSON file on Drive. The standard `_attachments` field only contains metadata stubs (`digest`, `length`, `type`). On pull, the base64 data is decoded back to `Blob` instances and written to local storage.
### pull & push
Standard RxDB [Replication Options](./replication.md) for batch size, modifiers, etc.
## Technical Details
### File Mapping
- Each RxDB document corresponds to **one JSON file** in the `docs` subfolder.
- The filename is `[primaryKey].json`.
- This simple mapping makes it easy to inspect or backup data manually.
### Checkpointing
- The replication relies on the `modifiedTime` of files in Google Drive.
### Conflict Resolution
- Conflicts are handled using the standard RxDB [conflict handling](./replication.md#conflict-handling) strategies.
- The plugin assumes a master-slave replication pattern where the client (RxDB) merges changes.
- If the `transaction` file is locked by another device, the write retries until the lock is released or times out.
## Limitations
- **Rate Limits:** Google Drive API has strict rate limits. The plugin attempts to handle 429 errors with exponential backoff, but heavy concurrent writes might hit these limits.
- **Latency:** Changes take time to propagate and appear in listings (eventual consistency), which the plugin handles internally.
- **Signaling Delay:** The initial WebRTC handshake requires writing and reading files from Drive, which can take a few seconds. Once connected, signaling is instant.
## Testing
For testing, it is recommended to use [google-drive-mock](https://github.com/pubkey/google-drive-mock). It simulates the Google Drive API so you can run tests without real credentials.
## FAQ
Google Drive API rate limiting is based on **quota units**, not a flat request count. According to the official Google Drive API usage limits documentation, the default limits are:
- **1,000,000 quota units per minute per project**
- **325,000 quota units per minute per user (within a project)**
So the HTTP requests/minute depends on the endpoint cost:
- If an endpoint costs **1 quota unit/request**, the theoretical maximum is **1,000,000 req/min/project** and **325,000 req/min/user**.
- If an endpoint costs **100 quota units/request** (for example `files.list` in many setups), that is about **10,000 req/min/project** and **3,250 req/min/user**.
When limits are exceeded, Google can return `403` (`User rate limit exceeded`) or `429` (`Rate limit exceeded`) responses.
The replication plugin already retries with exponential backoff, but for high-traffic apps you should monitor quota usage and tune your sync frequency/batch sizes.
- Official limits: https://developers.google.com/workspace/drive/api/guides/limits
- Error handling guidance: https://developers.google.com/workspace/drive/api/guides/handle-errors
---
## Microsoft OneDrive Sync
import {Steps} from '@site/src/components/steps';
import {BetaBlock} from '@site/src/components/beta-block';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# Replication with Microsoft OneDrive
The `replication-microsoft-onedrive` plugin allows you to replicate your client-side [RxDB](./) database to a folder in the user's Microsoft OneDrive. This enables cross-device [sync](./replication.md) for single users without requiring any backend server.
## Overview
The replication uses the Microsoft Graph API.
- **[Offline-First](./offline-first.md):** Users can work offline. Changes are synced when they go online.
- **No Backend Required:** You don't need to host your own database server.
- **Cross-Device:** Users can access their data from multiple devices by signing into the same Microsoft account.
- **Realtime Sync:** Uses [WebRTC](./replication-webrtc.md) for peer-to-peer signaling to achieve near real-time updates. Uses the same onedrive folder instead of a signaling-server.
## Usage
### Enable Microsoft Graph API
You need to register your application in the [Azure portal](https://portal.azure.com/) and create credentials (OAuth 2.0 Client ID) with `Files.ReadWrite` permissions for your application.
### Authenticate the User
Your application must handle the OAuth flow to get an `accessToken` from Microsoft. You can use libraries like `@azure/msal-browser` or `@azure/msal-react`.
### Start Replication
Once you have the `accessToken`, you can start the replication.
```ts
import {
replicateMicrosoftOneDrive
} from 'rxdb/plugins/replication-microsoft-onedrive';
const replicationState = await replicateMicrosoftOneDrive({
replicationIdentifier: 'my-app-onedrive-sync',
collection: myRxCollection, // RxCollection
oneDrive: {
authToken: 'USER_ACCESS_TOKEN',
folderPath: 'my-app-data/user-1'
},
live: true,
pull: {
batchSize: 60,
modifier: doc => doc // (optional) modify invalid data
},
push: {
batchSize: 60,
modifier: doc => doc // (optional) modify before sending
}
});
// Observe replication states
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
replicationState.awaitInitialReplication().then(() => {
console.log('Initial replication done');
});
```
## Signaling & WebRTC
Microsoft OneDrive does not provide real-time events for file changes that a client can easily subscribe to in the browser. If a user changes data on **User Device A**, **User Device B** would not know about it until it periodically polls the API.
To achieve real-time updates, this plugin uses **WebRTC** to signal changes between connected devices.
1. Devices create "signal files" in a `signaling` subfolder on OneDrive.
2. Other devices detect these files, read the WebRTC connection data, and establish a direct P2P connection with each other.
3. When a device makes a write, it sends a "RESYNC" signal via WebRTC to all connected peers to notify them about the change.
### Polyfill for Node.js
WebRTC is native in browsers but requires a polyfill in Node.js. Use `createSimplePeerWrtc()` to wrap the polyfill for compatibility with `simple-peer`:
```ts
import nodeDatachannelPolyfill from 'node-datachannel/polyfill';
import { createSimplePeerWrtc } from 'rxdb/plugins/replication-webrtc';
// ...
const replicationState = await replicateMicrosoftOneDrive({
// ...
signalingOptions: {
wrtc: createSimplePeerWrtc(nodeDatachannelPolyfill)
}
});
```
## Options
### oneDrive
- **authToken** `string`: The valid access token associated with the user.
- **folderPath** `string`: The path to the folder in Microsoft OneDrive where data should be stored.
- The plugin will ensure this folder exists.
- It must **not** be the root folder.
- It creates subfolders `docs` (for data) and `signaling` (for WebRTC).
- **apiEndpoint** `string` (optional): Defaults to `https://graph.microsoft.com/v1.0/me/drive`. Useful for mocking or proxies.
- **transactionTimeout** `number` (optional): Default `10000` (10s). The plugin uses a `transaction.json` file in OneDrive to ensure data integrity during writes. This is the timeout after which a lock is considered stale.
### pull & push
Standard RxDB [Replication Options](./replication.md) for batch size, modifiers, etc.
## Technical Details
### File Mapping
- Each RxDB document corresponds to **one JSON file** in the `docs` subfolder.
- The filename is `[primaryKey].json`.
- This simple mapping makes it easy to inspect or backup data manually.
### Checkpointing
- The replication relies on the `lastModifiedDateTime` of files in Microsoft OneDrive.
### Conflict Resolution
- Conflicts are handled using the standard RxDB [conflict handling](./replication.md#conflict-handling) strategies.
- The plugin assumes a master-slave replication pattern where the client (RxDB) merges changes.
- If the `transaction.json` file is locked by another device, the write retries until the lock is released or times out.
## Limitations
- **Rate Limits:** Microsoft Graph API has strict rate limits. The plugin attempts to handle 429 errors with exponential backoff, but heavy concurrent writes might hit these limits.
- **Latency:** Changes take time to propagate and appear in listings (eventual consistency), which the plugin handles internally.
- **Signaling Delay:** The initial WebRTC handshake requires writing and reading files from OneDrive, which can take a few seconds. Once connected, signaling is instant.
## Testing
For testing, it is recommended to use [microsoft-onedrive-mock](https://github.com/pubkey/microsoft-onedrive-mock). It simulates the Microsoft Graph API so you can run tests without real credentials.
---
## RxDB & NATS - Realtime Sync
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
# Replication with NATS
With this RxDB plugin you can run a two-way realtime replication with a [NATS](https://nats.io/) server.
The replication itself uses the [RxDB Sync Engine](./replication.md) which handles conflicts, errors and retries.
On the client side the official [NATS npm package](https://www.npmjs.com/package/nats) is used to connect to the NATS server.
NATS is a messaging system that by itself does not have a validation or granulary access control build in.
Therefore it is not recommended to directly replicate the NATS server with an untrusted RxDB client application. Instead you should replicated from NATS to your Node.js server side RxDB database.
## Precondition
For the replication endpoint the NATS cluster must have enabled [JetStream](https://docs.nats.io/nats-concepts/jetstream) and store all message data as [structured JSON](https://docs.nats.io/using-nats/developer/sending/structure).
The easiest way to start a compatible NATS server is to use the official docker image:
```docker run --rm --name rxdb-nats -p 4222:4222 nats:2.9.17 -js```
## Usage
### Install the nats package
```bash
npm install nats --save
```
### Start the Replication
To start the replication, import the
`replicateNats()` method from the RxDB
plugin and call it with the collection
that must be replicated.
The replication runs
*per [RxCollection](./rx-collection.md)*,
you can replicate multiple RxCollections by
starting a new replication for each of them.
```typescript
import {
replicateNats
} from 'rxdb/plugins/replication-nats';
const replicationState = replicateNats({
collection: myRxCollection,
replicationIdentifier: 'my-nats-replication-collection-A',
// in NATS, each stream need a name
streamName: 'stream-for-replication-A',
/**
* The subject prefix determines how the documents are stored in NATS.
* For example the document with id 'alice'
* will have the subject 'foobar.alice'
*/
subjectPrefix: 'foobar',
connection: { servers: 'localhost:4222' },
live: true,
pull: {
batchSize: 30
},
push: {
batchSize: 30
}
});
```
## Handling deletes
RxDB requires you to never
[fully delete documents](./replication.md#data-layout-on-the-server).
This is needed to be able to replicate the
deletion state of a document to other
instances. The NATS replication will set a
boolean `_deleted` field to all documents to
indicate the deletion state. You can change
this by setting a different `deletedField`
in the sync options.
---
## Appwrite Realtime Sync for Local-First Apps
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {RxdbMongoDiagramPlain} from '@site/src/components/mongodb-sync';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
# RxDB Appwrite Replication
This replication plugin allows you to synchronize documents between RxDB and an Appwrite server. It supports both push and pull replication, live updates via Appwrite's real-time subscriptions, [offline-capability](./offline-first.md) and [conflict resolution](./transactions-conflicts-revisions.md).
## Why you should use RxDB with Appwrite?
**Appwrite** is a secure, open-source backend server that simplifies backend tasks like user authentication, storage, database management, and real-time APIs.
**[RxDatabase](./rx-database.md)** is a reactive database for the frontend that offers offline-first capabilities and rich client-side data handling.
Combining the two provides several benefits:
1. [Offline-First](./offline-first.md): RxDB keeps all data locally, so your application remains fully functional even when the network is unavailable. When connectivity returns, the RxDB β Appwrite replication automatically resolves and synchronizes changes.
2. **Real-Time Sync**: With Appwriteβs real-time subscriptions and RxDBβs live replication, you can build collaborative features that update across all clients instantaneously.
3. [Conflict Handling](./transactions-conflicts-revisions.md): RxDB offers flexible conflict resolution strategies, making it simpler to handle concurrent edits across multiple users or devices.
4. **Scalable & Secure**: Appwrite is built to handle production loads with granular access controls, while RxDB easily scales across various storage backends on the client side.
5. **Simplicity & Modularity**: RxDBβs plugin-based architecture, combined with Appwriteβs Cloud offering makes it one of the easiest way to build local-first [realtime apps](./articles/realtime-database.md) that scale.
## Preparing the Appwrite Server
You can either use the appwrite cloud or self-host the Appwrite server. In this tutorial we use the Cloud which is recommended for beginners because it is way easier to set up. You can later decide to self-host if needed.
### Set up an Appwrite Endpoint and Project
#### Self-Hosted
##### Docker
Ensure docker and docker-compose is installed and your version are up to date:
```bash
docker-compose -v
```
##### Run the installation script
The installation script runs inside of a docker container. It will create a docker-compose file and an `.env` file.
```bash
docker run -it --rm \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \
--entrypoint="install" \
appwrite/appwrite:1.6.1
```
##### Start/Stop
After the installation is done, you can manually stop and start the appwrite instance with docker compose:
```bash
# stop
docker-compose down
# start
docker-compose up
```
#### Appwrite Cloud
##### Create a Cloud Account
Got to the Appwrite Console, create an account and login.
#### Create a Project
At the console click the `+ Create Project` button to create a new project. Remember the `project-id` which will be used later.
### Create an Appwrite Database and Collection
After creating an Appwrite project you have to create an Appwrite Database and a collection, you can either do this in code with the node-appwrite SDK or in the Appwrite Console as shown in this video:
### Add your documents attributes
In the appwrite collection, create all attributes of your documents. You have to define all the fields that your document in your [RxDB schema](./rx-schema.md) knows about. Notice that Appwrite does not allow for nested attributes. So when you use RxDB with Appwrite, you should also not have nested attributes in your RxDB schema.
### Add a `deleted` attribute
Appwrite (natively) hard-deletes documents. But for offline-handling RxDB needs soft-deleted documents on the server so that the deletion state can be replicated with other clients.
In RxDB, `_deleted` indicates that a document is removed locally and you need a similar field in your Appwrite collection on the Server: You must define a deletedField with any name to mark documents as "deleted" in the remote collection. Mostly you would use a boolean field named `deleted` (set it to `required`). The plugin will treat any document with `{ [deletedField]: true }` as deleted and replicate that state to local RxDB.
### Set the Permission on the Appwrite Collection
Appwrite uses permissions to control data access on the collection level. Make sure that in the Console at `Collection -> Settings -> Permissions` you have set the permission according to what you want to allow your clients to do. For testing, just enable all of them (Create, Read, Update and Delete).
## Setting up the RxDB - Appwrite Replication
Now that we have set up the Appwrite server, we can go to the client side code and set up RxDB and the replication:
### Install the Appwrite SDK and RxDB:
```bash
npm install appwrite rxdb
```
### Import the Appwrite SDK and RxDB
```ts
import {
replicateAppwrite
} from 'rxdb/plugins/replication-appwrite';
import {
createRxDatabase,
addRxPlugin,
RxCollection
} from 'rxdb/plugins/core';
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
import { Client } from 'appwrite';
```
### Create a Database with a Collection
```ts
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage()
});
const mySchema = {
title: 'my schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
}
},
required: ['id', 'name']
};
await db.addCollections({
humans: {
schema: mySchema
}
});
const collection = db.humans;
```
### Configure the Appwrite Client
#### Appwrite Cloud
```ts
const client = new Client();
client.setEndpoint('https://cloud.appwrite.io/v1');
client.setProject('YOUR_APPWRITE_PROJECT_ID');
```
#### Self-Hosted
```ts
const client = new Client();
client.setEndpoint('http://localhost/v1');
client.setProject('YOUR_APPWRITE_PROJECT_ID');
```
### Start the Replication
```ts
const replicationState = replicateAppwrite({
replicationIdentifier: 'my-appwrite-replication',
client,
databaseId: 'YOUR_APPWRITE_DATABASE_ID',
collectionId: 'YOUR_APPWRITE_COLLECTION_ID',
deletedField: 'deleted', // Field that represents deletion in Appwrite
collection,
pull: {
batchSize: 10,
},
push: {
batchSize: 10
},
/*
* ...
* You can set all other options for RxDB replication states
* like 'live' or 'retryTime'
* ...
*/
});
```
### Do other things with the replication state
The `RxAppwriteReplicationState` which is returned from `replicateAppwrite()` allows you to run all functionality of the normal [RxReplicationState](./replication.md).
## FAQ
Yes, Appwrite supports creating multiple top-level databases within a single project, which cleanly partition collections. However, Appwrite is a rigid NoSQL document store that does *not* support nested subcollections (unlike Firebase). When utilizing the **[RxDB Appwrite Replication](./replication.md)** plugin, your local RxDB schema must mirror this flat topology precisely, keeping all documents completely devoid of complex nested relationships.
Appwrite uses MariaDB (a highly performant MySQL fork) as its core backing database driver. To offer developers a flat NoSQL experience, Appwrite abstracts the MariaDB relational complexity behind a unified Document API. This architectural mapping allows **[RxDB](./rx-database.md)** to replicate data effortlessly into Appwrite via standard REST endpoints without ever dealing with strict SQL table mappings or migrations.
Appwrite natively provides robust WebSocket subscriptions allowing clients to receive real-time document events while the network is active. However, Appwrite does *not* feature a built-in offline-first caching or background-sync engine. To achieve true offline capabilities, you must mount the **[RxDB Appwrite Replication](./replication.md)** plugin on the client. RxDB handles all local caching, queues offline writes securely, and automatically pushes local mutations to Appwrite when connectivity returns.
## Limitations of the Appwrite Replication Plugin
- Appwrite primary keys only allow for the characters `a-z`, `A-Z`, `0-9`, and underscore `_` (They cannot start with a leading underscore). Also the primary key has a max length of 36 characters.
- The Appwrite replication **only works on browsers**. This is because the Appwrite SDK does not support subscriptions in Node.js.
- Appwrite does not allow for bulk write operations so on push one HTTP request will be made per document. Reads run in bulk so this is mostly not a problem.
- Appwrite does not allow for transactions or "update-if" calls which can lead to overwriting documents instead of properly handling [conflicts](./transactions-conflicts-revisions.md#conflicts) when multiple clients edit the same document in parallel. This is not a problem for inserts because "insert-if-not" calls are made.
- Nested attributes in Appwrite collections are only possible via experimental relationship attributes, and compatibility with RxDB is not tested. Users opting to use these experimental relationship attributes with RxDB do so at their own risk.
---
## RxDB Server - Deploy Your Data
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
# RxDB Server
The RxDB Server Plugin makes it possible to spawn a server on top of a RxDB database that offers multiple types of endpoints for various usages. It can spawn basic CRUD REST endpoints or even realtime replication endpoints that can be used by the client devices to replicate data. The RxServer plugin is designed to be used in Node.js but you can also use it in Deno, Bun or the [Electron](./electron-database.md) "main" process. You can use it either as a **standalone server** or add it on top of an **existing http server** (like express) in nodejs.
## Starting a RxServer
To create an `RxServer`, you have to install the `rxdb-server` package with `npm install rxdb-server --save` and then you can import the `createRxServer()` function and create a server on a given [RxDatabase](./rx-database.md) and adapter.
After adding the endpoints to the server, do not forget to call `myServer.start()` to start the actually http-server.
```ts
import { createRxServer } from 'rxdb-server/plugins/server';
/**
* We use the express adapter which is the one that comes with RxDB core
* Make sure you have express installed in the correct version!
* @see https://github.com/pubkey/rxdb-server/blob/master/package.json
*/
import { RxServerAdapterExpress } from 'rxdb-server/plugins/adapter-express';
const myServer = await createRxServer({
database: myRxDatabase,
adapter: RxServerAdapterExpress,
port: 443
});
// add endpoints here (see below)
// after adding the endpoints, start the server
await myServer.start();
```
### Using RxServer with Fastify
There is also a [RxDB Premium π](/premium/) adapter to use the RxServer with [Fastify](https://fastify.dev/) instead of express. Fastify has shown to have better performance and in general is more modern.
```ts
import { createRxServer } from 'rxdb-server/plugins/server';
import { RxServerAdapterFastify } from 'rxdb-premium/plugins/server-adapter-fastify';
const myServer = await createRxServer({
database: myRxDatabase,
adapter: RxServerAdapterFastify,
port: 443
});
await myServer.start();
```
### Using RxServer with Koa
There is also a [RxDB Premium π](/premium/) adapter to use the RxServer with [Koa](https://koajs.com/) instead of express. Koa has shown to have better performance compared to express.
```ts
import { createRxServer } from 'rxdb-server/plugins/server';
import { RxServerAdapterKoa } from 'rxdb-premium/plugins/server-adapter-koa';
const myServer = await createRxServer({
database: myRxDatabase,
adapter: RxServerAdapterKoa,
port: 443
});
await myServer.start();
```
## RxServer Endpoints
On top of the RxServer you can add different types of **endpoints**. An endpoint is always connected to exactly one [RxCollection](./rx-collection.md) and it only serves data from that single collection.
For now there are only two endpoints implemented, the [replication endpoint](#replication-endpoint) and the [REST endpoint](#rest-endpoint). Others will be added in the future.
An endpoint is added to the server by calling the add endpoint method like `myRxServer.addReplicationEndpoint()`. Each needs a different `name` string as input which will define the resulting endpoint url.
The endpoint urls is a combination of the given `name` and schema `version` of the collection, like `/my-endpoint/0`.
```ts
const myEndpoint = server.addReplicationEndpoint({
name: 'my-endpoint',
collection: myServerCollection
});
console.log(myEndpoint.urlPath) // > 'my-endpoint/0'
```
Notice that it is **not required** that the server side schema version is equal to the client side schema version. You might want to change server schemas more often and then only do a [migration](./migration-schema.md) on the server, not on the clients.
## Replication Endpoint
The replication endpoint allows clients that connect to it to replicate data with the server via the [RxDB Sync Engine](./replication.md). There is also the [Replication Server](./replication-server.md) plugin that is used on the client side to connect to the endpoint.
The endpoint is added to the server with the `addReplicationEndpoint()` method. It requires a specific collection and the endpoint will only provided replication for documents inside of that collection.
```ts
// > server.ts
const endpoint = server.addReplicationEndpoint({
name: 'my-endpoint',
collection: myServerCollection
});
```
Then you can start the [Server Replication](./replication-server.md) on the client:
```ts
// > client.ts
const replicationState = await replicateServer({
collection: usersCollection,
replicationIdentifier: 'my-server-replication',
url: 'http://localhost:80/my-endpoint/0',
push: {},
pull: {}
});
```
## REST endpoint
The REST endpoint exposes various methods to access the data from the RxServer with non-RxDB tools via plain HTTP operations. You can use it to connect apps that are programmed in different programming languages than JavaScript or to access data from other third party tools.
Creating a REST endpoint on a RxServer:
```ts
const endpoint = await server.addRestEndpoint({
name: 'my-endpoint',
collection: myServerCollection
});
```
```ts
// plain http request with fetch
const request = await fetch('http://localhost:80/' + endpoint.urlPath + '/query', {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({ selector: {} })
});
const response = await request.json();
```
There is also the `client-rest` plugin that provides type-save interactions with the REST endpoint:
```ts
// using the client (optional)
import { createRestClient } from 'rxdb-server/plugins/client-rest';
const client = createRestClient(
'http://localhost:80/' + endpoint.urlPath,
{/* headers */}
);
const response = await client.query({ selector: {} });
```
The REST endpoint exposes the following paths:
- **query [POST]**: Fetch the results of a NoSQL query.
- **query/observe [GET]**: Observe a query's results via [Server Send Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events).
- **get [POST]**: Fetch multiple documents by their primary key.
- **set [POST]**: Write multiple documents at once.
- **delete [POST]**: Delete multiple documents by their primary key.
## CORS
When creating a server or adding endpoints, you can specify a [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) string.
Endpoint cors always overwrite server cors. The default is the wildcard `*` which allows all requests.
```ts
const myServer = await startRxServer({
database: myRxDatabase,
cors: 'http://example.com'
port: 443
});
const endpoint = await server.addReplicationEndpoint({
name: 'my-endpoint',
collection: myServerCollection,
cors: 'http://example.com'
});
```
## Auth handler
To authenticate users and to make user-specific data available on server requests, an `authHandler` must be provided that parses the headers and returns the actual auth data that is used to authenticate the client and in the [queryModifier](#query-modifier) and [changeValidator](#change-validator).
An auth handler gets the given headers object as input and returns the auth data in the format `{ data: {}, validUntil: 1706579817126}`.
The `data` field can contain any data that can be used afterwards in the queryModifier and changeValidator.
The `validUntil` field contains the unix timestamp in milliseconds at which the authentication is no longer valid and the client will get disconnected.
For example your authHandler could get the `Authorization` header and parse the [JSON web token](https://jwt.io/) to identify the user and store the user id in the `data` field for later use.
## Query modifier
The query modifier is a JavaScript function that is used to restrict which documents a client can fetch or replicate from the server.
It gets the auth data and the actual NoSQL query as input parameter and returns a modified NoSQL query that is then used internally by the server.
You can pass a different query modifier to each endpoint so that you can have different endpoints for different use cases on the same server.
For example you could use a query modifier that get the `userId` from the auth data and then restricts the query to only return documents that have the same `userId` set.
```ts
function myQueryModifier(authData, query) {
query.selector.userId = { $eq: authData.data.userid };
return query;
}
const endpoint = await server.addReplicationEndpoint({
name: 'my-endpoint',
collection: myServerCollection,
queryModifier: myQueryModifier
});
```
The RxServer will use the queryModifier at many places internally to determine which queries to run or if a document is allowed to be seen/edited by a client.
:::note
For performance reasons the `queryModifier` and `changeValidator` **MUST NOT** be `async` and return a promise. If you need async data to run them, you should gather that data in the `RxServerAuthHandler` and store it in the auth data to access it later.
:::
## Change validator
The change validator is a JavaScript function that is used to restrict which document writes are allowed to be done by a client.
For example you could restrict clients to only change specific document fields or to not do any document writes at all.
It can also be used to validate change document data before storing it at the server.
In this example we restrict clients from doing inserts and only allow updates. For that we check if the change contains an `assumedMasterState` property and return false to block the write.
```ts
function myChangeValidator(authData, change) {
if(change.assumedMasterState) {
return false;
} else {
return true;
}
}
const endpoint = await server.addReplicationEndpoint({
name: 'my-endpoint',
collection: myServerCollection,
changeValidator: myChangeValidator
});
```
## Server-only indexes
Normal RxDB schema indexes get the `_deleted` field prepended because all [RxQueries](./rx-query.md) automatically only search for documents with `_deleted=false`.
When you use RxDB on a server, this might not be optimal because there can be the need to query for documents where the value of `_deleted` does not matter. Mostly this is required in the [pull.stream$](./replication.md#checkpoint-iteration) of a replication when a [queryModifier](#query-modifier) is used to add an additional field to the query.
To set indexes without `_deleted`, you can use the `internalIndexes` field of the schema like the following:
```json
{
"version": 0,
"primaryKey": "id",
"type": "object",
"properties": {
"id": {
"type": "string",
"maxLength": 100
},
"name": {
"type": "string",
"maxLength": 100
}
},
"internalIndexes": [
["name", "id"]
]
}
```
:::note
Indexes come with a performance burden. You should only use the indexes you need and make sure you **do not** accidentally set the `internalIndexes` in your client side [RxCollections](./rx-collection.md).
:::
## Server-only fields
All endpoints can be created with the `serverOnlyFields` set which defines some fields to only exist on the server, not on the clients. Clients will not see that fields and cannot do writes where one of the `serverOnlyFields` is set.
Notice that when you use `serverOnlyFields` you likely need to have a different schema on the server than the schema that is used on the clients.
```ts
const endpoint = await server.addReplicationEndpoint({
name: 'my-endpoint',
collection: col,
// here the field 'my-secretss' is defined to be server-only
serverOnlyFields: ['my-secrets']
});
```
:::note
For performance reasons, only top-level fields can be used as `serverOnlyFields`. Otherwise the server would have to deep-clone all document data which is too expensive.
:::
## Readonly fields
When you have fields that should only be modified by the server, but not by the client, you can ensure that by comparing the fields value in the [changeValidator](#change-validator).
```ts
const myChangeValidator = function(authData, change){
if(
change.newDocumentState.myReadonlyField !==
change.assumedMasterState.myReadonlyField
){
throw new Error('myReadonlyField is readonly');
}
}
```
## $regex queries not allowed
`$regex` queries are not allowed to run at the server to prevent [ReDos Attacks](https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS).
## Conflict handling
To [detect and handle conflicts](./replication.md#conflict-handling), the conflict handler from the endpoints RxCollection is used.
## FAQ
The RxServer and its other plugins are in a different github repository because:
It has too many dependencies that you do not want to install if you only use RxDB at the client side
It has a different license (SSPL) to prevent large cloud vendors from "stealing" the revenue, similar to MongoDB's license.
After `RxServer.start()` is called, you can no longer add endpoints. This is because many of the supported
server libraries do not allow dynamic routing for performance and security reasons.
---
## RxServer Scaling - Vertical or Horizontal
# Scaling the RxServer
The [RxDB Server](./rx-server.md) run in JavaScript and JavaScript runs on a single process on the operating system. This can make the CPU performance limit to be the main bottleneck when serving requests to your users. To mitigate that problem, there are a wide range of methods to scale up the server so that it can serve more requests at the same time faster.
## Vertical Scaling
Vertical Scaling aka "scaling up" has the goal to get more power out of a single server by utilizing more of the servers compute. Vertical scaling should be the first step when you decide it is time to scale.
### Run multiple JavaScript processes
To utilize more compute power of your server, the first step is to scale vertically by running the RxDB server on **multiple processes** in parallel.
RxDB itself is already build to support multiInstance-usage on the client, like when the user has opened multiple browser tabs at once. The same method works also on the server side in Node.js. You can spawn multiple JavaScript processes that use the same [RxDatabase](./rx-database.md) and the instances will automatically communicate with each other and distribute their data and events with the [BroadcastChannel](https://github.com/pubkey/broadcast-channel).
By default the [multiInstance param](./rx-database.md#multiinstance) is set to `true` when calling `createRxDatabase()`, so you do not have to change anything. To make all processes accessible through the same endpoint, you can put a load-balancer like [nginx](https://nginx.org/en/docs/http/load_balancing.html) in front of them.
### Using workers to split up the load
Another way to increases the server capacity is to put the storage into a [Worker thread](./rx-storage-worker.md) so that the "main" thread with the webserver can handle more requests. This might be easier to set up compared to using multiple JavaScript processes and a load balancer.
### Use an in-memory storage at the user facing level
Another way to serve more requests to your end users, is to use an [in-memory](./rx-storage-memory.md) storage that has the [best](./rx-storage-performance.md) read- and write performance. It outperforms persistent storages by a factor of 10x.
So instead of directly serving requests from the persistence layer, you add an in-memory layer on top of that. You could either do a [replication](./replication.md) from your memory database to the persistent one, or you use the [memory mapped](./rx-storage-memory-mapped.md) storage which has this build in.
```ts
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
import { replicateRxCollection } from 'rxdb/plugins/replication';
import {
getRxStorageFilesystemNode
} from 'rxdb-premium/plugins/storage-filesystem-node';
import { getMemoryMappedRxStorage } from 'rxdb-premium/plugins/storage-memory-mapped';
const myRxDatabase = await createRxDatabase({
name: 'mydb',
storage: getMemoryMappedRxStorage({
storage: getRxStorageFilesystemNode({
basePath: path.join(__dirname, 'my-database-folder')
})
})
});
await myDatabase.addCollections({/* ... */});
const myServer = await startRxServer({
database: myRxDatabase,
port: 443
});
```
But notice that you have to check your persistence requirements. When a write happens to the memory layer and the server crashes while it has not persisted, in rare cases the write operation might get lost. You can remove that risk by setting `awaitWritePersistence: true` on the [memory mapped storage](./rx-storage-memory-mapped.md) settings.
## Horizontal Scaling
To scale the RxDB Server above a single physical hardware unit, there are different solutions where the decision depends on the exact use case.
### Single Datastore with multiple branches
The most common way to use multiple servers with RxDB is to split up the server into a tree with a root "datastore" and multiple "branches". The datastore contains the persisted data and only servers as a replication endpoint for the branches. The branches themself will replicate data to and from the datastore and server requests to the end users.
This is mostly useful on read-heavy applications because reads will directly run on the branches without ever reaching the main datastore and you can always add more branches to **scale up**. Even adding additional layers of "datastores" is possible so the tree can grow (or shrink) with the demand.
### Moving the branches to "the edge"
Instead of running the "branches" of the tree on the same physical location as the datastore, it often makes sense to move the branches into a datacenter near the end users. Because the RxDB [replication algorithm](./replication.md) is made to work with slow and even partially offline users, using it for physically separated servers will work the same way. Latency is not that important because writes and reads will not decrease performance by blocking each other and the replication can run in the background without blocking other servers during transaction.
### Replicate Databases for Microservices
If your application is build with a [microservice architecture](https://en.wikipedia.org/wiki/Microservices) and your microservices are also build in Node.js, you can scale the database horizontally by moving the database into the microservices and use the [RxDB replication](./replication.md) to do a realtime sync between the microservices and a main "datastore" server. The "datastore" server would then only handle the replication requests or do some additional things like logging or [backups](./backup.md). The compute for reads and writes will then mainly be done on the microservices themself. This simplifies setting up more and more microservices without decreasing the performance of the whole system.
### Use a self-scaling RxStorage
An alternative to scaling up the RxDB servers themself, you can also switch to a [RxStorage](./rx-storage.md) which scales up internally. For example the [FoundationDB storage](./rx-storage-foundationdb.md) or [MongoDB](./rx-storage-mongodb.md) can work on top of a cluster that can increase load by adding more servers to itself. With that you can always add more Node.js RxDB processes that connect to the same cluster and server requests from it.
---
## Transactions, Conflicts and Revisions
In contrast to most SQL databases, RxDB does not have the concept of relational ACID transactions. Instead, RxDB has to apply different techniques that better suit the offline-first, client-side world where it is not possible to create a transaction between multiple maybe-offline client devices.
## Why RxDB does not have transactions
When talking about transactions, we mean [ACID transactions](https://en.wikipedia.org/wiki/ACID) that guarantee the properties of atomicity, consistency, isolation and durability.
With an ACID transaction you can mutate data dependent on the current state of the database. It is ensured that no other database operations happen in between your transaction and after the transaction has finished, it is guaranteed that the new data is actually written to the disk.
To implement ACID transactions on a **single server**, the database has to keep track on who is running transactions and then schedule these transactions so that they can run in isolation.
As soon as you have to split your database on **multiple servers**, transaction handling becomes way more difficult. The servers have to communicate with each other to find a consensus about which transaction can run and which has to wait. Network connections might break, or one server might complete its part of the transaction and then be required to roll back its changes because of an error on another server.
But with RxDB you have **multiple clients** that can go randomly online or offline. The users can have different devices and the clock of these devices can go off by any time. To support ACID transactions here, RxDB would have to make the whole world stand still for all clients, while one client is doing a write operation. And even that can only work when all clients are online. Implementing that might be possible, but at the cost of an unpredictable amount of performance loss and not being able to support [offline-first](./offline-first.md).
> A single write operation to a document is the only atomic thing you can do in [RxDatabase](./rx-database.md).
The benefits of not having to support transactions:
- Clients can read and write data without blocking each other.
- Clients can write data while being **offline** and then replicate with a server when they are **online** again, called [offline-first](./offline-first.md).
- Creating a compatible backend for the replication is easy so that RxDB can replicate with any existing infrastructure.
- Optimizations like [Sharding](./rx-storage-sharding.md) can be used.
## Revisions
Working without transactions leads to having undefined state when doing multiple database operations at the same time. Most client-side databases rely on a last-write-wins strategy on write operations. This might be a viable solution for some cases, but often this leads to strange problems that are hard to debug.
Instead, to ensure that the behavior of RxDB is **always predictable**, RxDB relies on **revisions** for version control. Revisions work similar to [Lamport Clocks](https://martinfowler.com/articles/patterns-of-distributed-systems/lamport-clock.html).
Each document is stored together with its revision string, that looks like `1-9dcca3b8e1a` and consists of:
- The revision height, a number that starts with `1` and is increased with each write to that document.
- The database instance token.
An operation to the RxDB data layer does not only contain the new document data, but also the previous document data with its revision string. If the previous revision matches the revision that is currently stored in the database, the write operation can succeed. If the previous revision is **different** than the revision that is currently stored in the database, the operation will throw a `409 CONFLICT` error.
## Conflicts
There are two types of conflicts in RxDB, the **local conflict** and the **replication conflict**.
### Local conflicts
A local conflict can happen when a write operation assumes a different previous document state, than what is currently stored in the database. This can happen when multiple parts of your application do simultaneous writes to the same document. This can happen on a single browser tab, or when multiple tabs write at once or when a write appears while the document gets replicated from a remote server replication.
When a local conflict appears, RxDB will throw a `409 CONFLICT` error. The calling code must then handle the error properly, depending on the application logic.
Instead of handling local conflicts, in most cases it is easier to ensure that they cannot happen, by using `incremental` database operations like [incrementalModify()](./rx-document.md), [incrementalPatch()](./rx-document.md) or [incrementalUpsert()](./rx-collection.md). These write operations have a built-in way to handle conflicts by re-applying the mutation functions to the conflicting document state.
## Replication conflicts
A replication conflict appears when multiple clients write to the same documents at once and these documents are then replicated to the backend server.
When you replicate with the [GraphQL replication](./replication-graphql.md) and the [replication primitives](./replication.md), RxDB assumes that conflicts are **detected** and **resolved** at the client side.
When a document is sent to the backend and the backend detected a conflict (by comparing revisions or other properties), the backend will respond with the actual document state so that the client can compare this with the local document state and create a new, resolved document state that is then pushed to the server again. You can read more about the replication conflicts [here](./replication.md#conflict-handling).
## Custom conflict handler
A conflict handler is an object with two JavaScript functions:
- Detect if two document states are equal
- Solve existing conflicts
Because the conflict handler also is used for conflict detection, it will run many times on pull-, push- and write operations of RxDB. Most of the time it will detect that there is no conflict and then return.
Lets have a look at the [default conflict handler](https://github.com/pubkey/rxdb/blob/master/src/replication-protocol/default-conflict-handler.ts) of RxDB to learn how to create a custom one:
```ts
import { deepEqual } from 'rxdb/plugins/utils';
export const defaultConflictHandler: RxConflictHandler = {
isEqual(a, b) {
/**
* isEqual() is used to detect conflicts or to detect if a
* document has to be pushed to the remote.
* If the documents are deep equal,
* we have no conflict.
* Because deepEqual is CPU expensive, on your
* custom conflict handler you might only
* check some properties, like the updatedAt time or revisions
* for better performance.
*/
return deepEqual(a, b);
},
resolve(i) {
/**
* The default conflict handler will always
* drop the fork state and use the master state instead.
*
* In your custom conflict handler you likely want to merge properties
* of the realMasterState and the newDocumentState instead.
*/
return i.realMasterState;
}
};
```
To overwrite the default conflict handler, you have to specify a custom `conflictHandler` property when creating a collection with `addCollections()`.
```js
const myCollections = await myDatabase.addCollections({
// key = collectionName
humans: {
schema: mySchema,
conflictHandler: myCustomConflictHandler
}
});
```
---
## Efficient RxDB Queries via Query Cache
# QueryCache
RxDB uses a `QueryCache` which optimizes the reuse of queries at runtime. This makes sense especially when RxDB is used in UI-applications where people move for- and backwards on different routes or pages and the same queries are used many times. Because of the [event-reduce algorithm](https://github.com/pubkey/event-reduce) cached queries are even valuable for optimization, when changes to the database occur between now and the last execution.
## Cache Replacement Policy
To not let RxDB fill up all the memory, a `cache replacement policy` is defined that clears up the cached queries. This is implemented as a function which runs regularly, depending on when queries are created and the database is idle. The default policy should be good enough for most use cases but defining custom ones can also make sense.
## The default policy
The default policy starts cleaning up queries depending on how much queries are in the cache and how much document data they contain.
* It will never uncache queries that have subscribers to their results
* It tries to always have less than 100 queries without subscriptions in the cache.
* It prefers to uncache queries that have never executed and are older than 30 seconds
* It prefers to uncache queries that have not been used for longer time
## Other references to queries
With JavaScript, it is not possible to count references to variables. Therefore it might happen that an uncached `RxQuery` is still referenced by the users code and used to get results. This should never be a problem, uncached queries must still work. Creating the same query again however, will result in having two `RxQuery` instances instead of one.
## Using a custom policy
A cache replacement policy is a normal JavaScript function according to the type `RxCacheReplacementPolicy`.
It gets the `RxCollection` as first parameter and the `QueryCache` as second. Then it iterates over the cached `RxQuery` instances and uncaches the desired ones with `uncacheRxQuery(rxQuery)`. When you create your custom policy, you should have a look at the [default](https://github.com/pubkey/rxdb/blob/master/src/query-cache.ts).
To apply a custom policy to a [RxCollection](./rx-collection.md), add the function as attribute `cacheReplacementPolicy`.
```ts
const collection = await myDatabase.addCollections({
humans: {
schema: mySchema,
cacheReplacementPolicy: function(){ /* ... */ }
}
});
```
---
## Creating Plugins
Creating your own plugin is very simple. A plugin is basically a javascript-object which overwrites or extends RxDB's internal classes, prototypes, and hooks.
A basic plugin:
```javascript
const myPlugin = {
rxdb: true, // this must be true so rxdb knows that this is a rxdb-plugin
/**
* (optional) init() method
* that is called when the plugin is added to RxDB for the first time.
*/
init() {
// import other plugins or initialize stuff
},
/**
* every value in this object can manipulate
* the prototype of the keynames class
* You can manipulate every prototype in this list:
* @link https://github.com/pubkey/rxdb/blob/master/src/plugin.ts#L22
*/
prototypes: {
/**
* add a function to RxCollection so you can call 'myCollection.hello()'
*
* @param {object} prototype of RxCollection
*/
RxCollection: (proto) => {
proto.hello = function() {
return 'world';
};
}
},
/**
* some methods are static and can be overwritten in the overwritable-object
*/
overwritable: {
validatePassword: function(password) {
if (password && typeof password !== 'string' || password.length < 10)
throw new TypeError('password is not valid');
}
},
/**
* you can add hooks to the hook-list
*/
hooks: {
/**
* add a `foo` property to each document.
* You can then call myDocument.foo (='bar')
*/
createRxDocument: {
/**
* You can either add the hook running 'before' or 'after'
* the hooks of other plugins.
*/
after: function(doc) {
doc.foo = 'bar';
}
}
}
};
// now you can import the plugin into rxdb
addRxPlugin(myPlugin);
```
# Properties
## rxdb
The `rxdb`-property signals that this plugin is an rxdb-plugin. The value should always be `true`.
## prototypes
The `prototypes`-property contains a function for each of RxDB's internal prototype that you want to manipulate. Each function gets the prototype-object of the corresponding class as parameter and then can modify it. You can see a list of all available prototypes [here](https://github.com/pubkey/rxdb/blob/master/src/plugin.ts)
## overwritable
Some of RxDB's functions are not inside of a class-prototype but are static. You can set and overwrite them with the `overwritable`-object. You can see a list of all overwritables [here](https://github.com/pubkey/rxdb/blob/master/src/overwritable.ts).
# hooks
Sometimes you don't want to overwrite an existing RxDB-method, but extend it. You can do this by adding hooks which will be called each time the code jumps into the hooks corresponding call. You can find a list of all hooks [here](https://github.com/pubkey/rxdb/blob/master/src/hooks.ts).
# options
[RxDatabase](./rx-database.md) and [RxCollection](./rx-collection.md) have an additional options-parameter, which can be filled with any data required be the plugin.
```javascript
const collection = myDatabase.addCollections({
foo: {
schema: mySchema,
options: { // anything can be passed into the options
foo: ()=>'bar'
}
}
})
// Afterwards you can use these options in your plugin.
collection.options.foo(); // 'bar'
```
---
## Error Messages
# RxDB Error Messages
When RxDB has an error, an `RxError` object is thrown instead of a normal JavaScript `Error`. This `RxError` contains additional properties such as a `code` field and `parameters`. By default the full human readable error messages are not included into the RxDB build. This is because error messages have a high entropy and cannot be compressed well. Therefore only an error message with the correct error-code and parameters is thrown but without the full text.
When you enable the [DevMode Plugin](./dev-mode.md) the full error messages are added to the `RxError`. This should only be done in development, not in production builds to keep a small build size.
## All RxDB error messages
import { ErrorMessages } from '@site/src/components/error-messages';
---
## Testing
Writing tests for your RxDB application is crucial to ensure reliability. Because RxDB runs in many different environments (Browser, [Node.js](nodejs-database.md), [React Native](react-native-database.md), [Electron](electron-database.md), ...), testing strategies might vary. However, there are some common patterns that make testing easier and faster.
## Use the `memory` RxStorage
For unit tests, you should generally use the [`memory` RxStorage](rx-storage-memory.md). It keeps data only in memory, which has several advantages:
- **Speed**: It is much faster than writing to disc.
- **Isolation**: Each test run starts with a clean state; you don't have to delete database files between tests.
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
const db = await createRxDatabase({
name: 'test-db',
storage: getRxStorageMemory()
});
```
## The `using` Keyword
RxDB supports the [Explicit Resource Management](https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-2.html#using-declarations-and-explicit-resource-management) `using` keyword (available in TypeScript 5.2+). This automatically closes the database when the variable goes out of scope, which is perfect for tests.
Instead of manually calling `await db.close()`, you can do:
```ts
describe('my test suite', () => {
it('should insert a document', async () => {
// Did you know?
// using 'using' ensures db.close() is called automatically
// at the end of the test function.
await using db = await createRxDatabase({
name: 'test-db',
storage: getRxStorageMemory()
});
await db.addCollections({ ... });
// ... run your tests
});
});
```
## Cleanup
When running many tests, it is important to ensure that all databases are cleaned up after your tests run. Having non-closed `RxDatabase` instances after some tests can significantly decrease performance because background tasks and event listeners are still active.
A good practice is to verify that no database instances or connections are left open. You can check internal RxDB states to ensure everything is closed.
```ts
import { dbCount } from 'rxdb/plugins/core';
import assert from 'assert';
describe('cleanup', () => {
it('ensure every db is cleaned up', () => {
assert.strictEqual(dbCount(), 0);
});
});
```
## Multi-Tab Simulation
To test multi-tab behavior (like [Leader Election](leader-election.md) or [Replication](replication.md)) within a single Node.js process or test runner, you can create multiple [`RxDatabase`](rx-database.md) instances with the **same name** and storage. RxDB will treat them as if they were running in different tabs or processes. Notice that for this, [ignoreDuplicate](./rx-database.md#ignoreduplicate) must be set to `true` because otherwise it will not allow to create multiple databases with the same name in a single JavaScript process.
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
// Simulate Tab 1
const db1 = await createRxDatabase({
name: 'test-db', // same name
storage: getRxStorageMemory(),
ignoreDuplicate: true // must be set to true
});
await db1.addCollections({ ... });
// Simulate Tab 2
const db2 = await createRxDatabase({
name: 'test-db', // same name
storage: getRxStorageMemory(),
ignoreDuplicate: true // must be set to true
});
await db2.addCollections({ ... });
// insert at "tab one"
await db1.todos.insert({ id: "foobar"});
// read at "tab two"
const doc = await db2.todos.findOne("foobar").exec(true);
assert.ok(doc);
```
This works because the `memory` storage (and others) are shared within the same JavaScript process.
---
## Seamless Schema Data Migration with RxDB
# Migrate Database Data on schema changes
The RxDB Data Migration Plugin helps developers easily update stored data in their apps when they make changes to the data structure by changing the schema of a [RxCollection](./rx-collection.md). This is useful when developers release a new version of the app with a different schema.
Imagine you have your awesome messenger-app distributed to many users. After a while, you decide that in your new version, you want to change the schema of the messages-collection. Instead of saving the message-date like `2017-02-12T23:03:05+00:00` you want to have the unix-timestamp like `1486940585` to make it easier to compare dates. To accomplish this, you change the schema and **increase the version-number** and you also change your code where you save the incoming messages. But one problem remains: what happens with the messages which are already stored in the database on the user's device in the old schema?
With RxDB you can provide migrationStrategies for your collections that automatically (or on call) transform your existing data from older to newer schemas. This assures that the client's data always matches your newest code-version.
## Add the migration plugin
To enable the data migration, you have to add the `migration-schema` plugin.
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBMigrationSchemaPlugin } from 'rxdb/plugins/migration-schema';
addRxPlugin(RxDBMigrationSchemaPlugin);
```
## Providing strategies
Upon creation of a collection, you have to provide migrationStrategies when your schema's version-number is greater than `0`. To do this, you have to add an object to the `migrationStrategies` property where a function for every schema-version is assigned. A migrationStrategy is a function which gets the old document-data as a parameter and returns the new, transformed document-data. If the strategy returns `null`, the document will be removed instead of migrated.
```javascript
myDatabase.addCollections({
messages: {
schema: messageSchemaV1,
migrationStrategies: {
// 1 means, this transforms data from version 0 to version 1
1: function(oldDoc){
oldDoc.time = new Date(oldDoc.time).getTime(); // string to unix
return oldDoc;
}
}
}
});
```
Asynchronous strategies can also be used:
```javascript
myDatabase.addCollections({
messages: {
schema: messageSchemaV1,
migrationStrategies: {
1: function(oldDoc){
oldDoc.time = new Date(oldDoc.time).getTime(); // string to unix
return oldDoc;
},
/**
* 2 means, this transforms data from version 1 to version 2
* this returns a promise which resolves with the new document-data
*/
2: function(oldDoc){
// in the new schema (version: 2) we defined
// 'senderCountry' as required field (string)
// so we must get the country of the message-sender from the server
const coordinates = oldDoc.coordinates;
return fetch('http://myserver.com/api/countryByCoordinates/'+coordinates+'/')
.then(response => response.json())
.then(country => {
oldDoc.senderCountry = country;
return oldDoc;
});
}
}
}
});
```
you can also filter which documents should be migrated:
```js
myDatabase.addCollections({
messages: {
schema: messageSchemaV1,
migrationStrategies: {
// 1 means, this transforms data from version 0 to version 1
1: function(oldDoc){
oldDoc.time = new Date(oldDoc.time).getTime(); // string to unix
return oldDoc;
},
/**
* this removes all documents older than 2017-02-12
* they will not appear in the new collection
*/
2: function(oldDoc){
if(oldDoc.time < 1486940585) return null;
else return oldDoc;
}
}
}
});
```
## autoMigrate
By default, the migration automatically happens when the collection is created. Calling `RxDatabase.addCollections()` returns only when the migration has finished.
If you have lots of data or the migrationStrategies take a long time, it might be better to start the migration 'by hand' and show the migration-state to the user as a loading-bar.
:::warning No writes during a running migration
While a schema migration is running on a collection, writes to that collection are not allowed.
Calls that would write will throw a `COL25` error until the migration finishes.
Wait for `collection.migratePromise()` to resolve (or observe `collection.getMigrationState().$` until status is `DONE`)
before performing writes.
:::
```javascript
const messageCol = await myDatabase.addCollections({
messages: {
schema: messageSchemaV1,
autoMigrate: false, // <- migration will not run at creation
migrationStrategies: {
1: async function(oldDoc){
...
anything that takes very long
...
return oldDoc;
}
}
}
});
// check if migration is needed
const needed = await messageCol.migrationNeeded();
if(needed === false) {
return;
}
// start the migration
// 10 is the batch-size, how many docs will run
// at parallel
messageCol.startMigration(10);
const migrationState = messageCol.getMigrationState();
// 'start' the observable
migrationState.$.subscribe({
next: state => console.dir(state),
error: error => console.error(error),
complete: () => console.log('done')
});
// the emitted states look like this:
{
status: 'RUNNING' // oneOf 'RUNNING' | 'DONE' | 'ERROR'
count: {
total: 50, // amount of documents which must be migrated
handled: 0, // amount of handled docs
percent: 0 // percentage [0-100]
}
}
```
If you don't want to show the state to the user, you can also use `.migratePromise()`:
```js
const migrationPromise = messageCol.migratePromise(10);
await migratePromise;
```
## migrationStates()
`RxDatabase.migrationStates()` returns an `Observable` that emits all migration states of any collection of the database.
Use this when you add collections dynamically and want to show a loading-state of the migrations to the user.
```js
const allStatesObservable = myDatabase.migrationStates();
allStatesObservable.subscribe(allStates => {
allStates.forEach(migrationState => {
console.log(
'migration state of ' +
migrationState.collection.name
);
});
});
```
## Default values are not auto-applied
Schema `default` values are **not** automatically filled in during migration. When you insert a document through the normal RxDB API (like `insert()` or `bulkInsert()`), fields with a `default` in the schema are auto-filled. Migration does not do this. Your migration strategy has full explicit control over the document data and must set every field that the new schema needs.
```js
const messageSchemaV1 = {
version: 1,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
// new field in v1 with a default value
priority: { type: 'string', default: 'normal' }
},
required: ['id', 'text']
};
const migrationStrategies = {
1: function(oldDoc){
/**
* You must explicitly set 'priority' here.
* The schema default 'normal' will NOT be
* applied automatically during migration.
*/
oldDoc.priority = 'normal';
return oldDoc;
}
};
```
## Migrating attachments
When you store [RxAttachment](./rx-attachment.md)s together with your document, they can also be changed, added or removed while running the migration.
You can do this by mutating the `oldDoc._attachments` property.
```js
import { createBlob } from 'rxdb';
const migrationStrategies = {
1: async function(oldDoc){
// do nothing with _attachments to keep all
// attachments in the new collection version.
return oldDoc;
},
2: async function(oldDoc){
// set _attachments to an empty object to
// delete all existing ones during migration.
oldDoc._attachments = {};
return oldDoc;
},
3: async function(oldDoc){
// update the data field of a single attachment to change its data.
oldDoc._attachments.myFile.data = await createBlob(
'my new text',
oldDoc._attachments.myFile.content_type
);
return oldDoc;
}
}
```
## Migration on multi-tab in browsers
If you use RxDB in a multiInstance environment, like a browser, it will ensure that exactly one tab is running a migration of a collection.
Also the `migrationState.$` events are emitted between browser tabs.
## Migration and Replication
If you use any of the [RxReplication](./replication.md) plugins, the migration will also run on the internal replication-state storage. It will migrate all `assumedMasterState` documents
so that after the migration is done, you do not have to re-run the replication from scratch.
RxDB assumes that you run the exact same migration on the servers and the clients. Notice that the replication `pull-checkpoint` will not be migrated. Your backend must be compatible with pull-checkpoints of older versions.
## Migration should be run on all database instances
If you have multiple database instances (for example, if you are running replication inside of a [Worker](./rx-storage-worker.md) or [SharedWorker](./rx-storage-shared-worker.md) and have created a database instance inside of the worker), schema migration should be started on all database instances. All instances must know about all migration strategies and any updated schema versions.
---
## Migration Storage
import {Faq, FaqItem} from '@site/src/components/faq';
# Storage Migration
The storage migration plugin can be used to migrate all data from one existing RxStorage into another. This is useful when:
- You want to migrate from one [RxStorage](./rx-storage.md) to another one.
- You want to migrate to a new major RxDB version while keeping the previous saved data. This function only works from the previous major version upwards. Do not use it to migrate like rxdb v9 to v14.
The storage migration **drops deleted documents** and filters them out during the migration.
:::warning Do never change the schema while doing a storage migration
When you migrate between storages, you might want to change the schema in the same process. You should never do that because it will lead to problems afterwards and might make your database unusable.
When you also want to change your schema, first run the storage migration and afterwards run a normal [schema migration](./migration-schema.md).
:::
## Usage
Lets say you want to migrate from [LocalStorage RxStorage](./rx-storage-localstorage.md) to the [IndexedDB RxStorage](./rx-storage-indexeddb.md).
```ts
import { migrateStorage } from 'rxdb/plugins/migration-storage';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
import { getRxStorageLocalstorage } from 'rxdb-old/plugins/storage-localstorage';
// create the new RxDatabase
const db = await createRxDatabase({
name: dbLocation,
storage: getRxStorageIndexedDB(),
multiInstance: false
});
await migrateStorage({
database: db as any,
/**
* Name of the old database,
* using the storage migration requires that the
* new database has a different name.
*/
oldDatabaseName: 'myOldDatabaseName',
oldStorage: getRxStorageLocalstorage(), // RxStorage of the old database
batchSize: 500, // batch size
// true: migrate all collections in parallel
// false (default): migrate in serial
parallel: false,
afterMigrateBatch: (input: AfterMigrateBatchHandlerInput) => {
console.log('storage migration: batch processed');
}
});
```
:::note
Only collections that exist in the new database at the time you call migrateStorage() will have their data migrated.
- If your old database had collections `['users', 'posts', 'comments']` but your new database only defines `['users', 'posts']`, then only users and posts data will be migrated.
- Any collections missing from the new database will simply be skipped - no data for them will be read or written.
This allows you to selectively migrate only certain collections if desired, by choosing which collections to define before invoking `migrateStorage()`.
:::
## Migrate from a previous RxDB major version
To migrate from a previous RxDB major version, you have to install the 'old' RxDB in the `package.json`
```json
{
"dependencies": {
"rxdb-old": "npm:rxdb@14.17.1",
}
}
```
Then you can run the migration by providing the old storage:
```ts
/* ... */
import { migrateStorage } from 'rxdb/plugins/migration-storage';
// import from the old RxDB version
import {
getRxStorageLocalstorage
} from 'rxdb-old/plugins/storage-localstorage';
await migrateStorage({
database: db as any,
/**
* Name of the old database,
* using the storage migration requires that the
* new database has a different name.
*/
oldDatabaseName: 'myOldDatabaseName',
oldStorage: getRxStorageLocalstorage(), // RxStorage of the old database
batchSize: 500, // batch size
parallel: false,
afterMigrateBatch: (input: AfterMigrateBatchHandlerInput) => {
console.log('storage migration: batch processed');
}
});
/* ... */
```
## Disable Version Check on [RxDB Premium π](/premium/)
RxDB Premium has a check in place that ensures that you do not accidentally use the wrong RxDB core and π Premium version together which could break your database state.
This can be a problem during migrations where you have multiple versions of RxDB in use and it will throw the error `Version mismatch detected`.
You can disable that check by importing and running the `disableVersionCheck()` function from RxDB Premium.
```ts
// RxDB Premium v15 or newer:
import {
disableVersionCheck
} from 'rxdb-premium-old/plugins/shared';
disableVersionCheck();
// RxDB Premium v14:
// for esm
import {
disableVersionCheck
} from 'rxdb-premium-old/dist/es/shared/version-check.js';
disableVersionCheck();
// for cjs
import {
disableVersionCheck
} from 'rxdb-premium-old/dist/lib/shared/version-check.js';
disableVersionCheck();
```
## FAQ
Storage migration involves physically shifting all existing documents from one underlying RxStorage adapter (e.g., IndexedDB) into an entirely different storage engine (e.g., SQLite), often required during platform upgrades. Unlike PouchDB which lacked robust native migration rails, **[RxDB](./rx-database.md)** enforces distinct boundaries between structural Data Migrations (changing schema formats via the `migrationStrategy` map) and underlying Storage Migrations (`migrateStorage()`). These two distinct mechanisms must *never* be executed simultaneously.
---
## Attachments
import { DefaultCompressibleTypes } from '@site/src/components/default-compressible-types';
# Attachments
Attachments are binary data files that can be attachment to an `RxDocument`, like a file that is attached to an email.
Using attachments instead of adding the data to the normal document, ensures that you still have a good **performance** when querying and writing documents, even when a big amount of data, like an image file has to be stored.
- You can store string, binary files, images and whatever you want side by side with your documents.
- Deleted documents automatically loose all their attachments data.
- Not all [replication](./replication.md) plugins support the replication of attachments.
- Attachments can be stored [encrypted](./encryption.md).
Internally, attachment data is stored as `Blob` objects. Blob is the canonical internal type because it is immutable, carries MIME type metadata via `Blob.type`, provides synchronous size via `Blob.size`, and is [structured-cloneable](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm) (works with Worker/Electron `postMessage` and IndexedDB). Conversion to `ArrayBuffer` only happens at system boundaries that require it: encryption (Web Crypto), compression (CompressionStream), digest hashing, and WebSocket serialization.
## Add the attachments plugin
To enable the attachments, you have to add the `attachments` plugin.
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBAttachmentsPlugin } from 'rxdb/plugins/attachments';
addRxPlugin(RxDBAttachmentsPlugin);
```
## Enable attachments in the schema
Before you can use attachments, you have to ensure that the attachments-object is set in the schema of your `RxCollection`.
```javascript
const mySchema = {
version: 0,
type: 'object',
properties: {
// .
// .
// .
},
attachments: {
// if true, the attachment-data will be
// encrypted with the db-password
encrypted: true
}
};
const myCollection = await myDatabase.addCollections({
humans: {
schema: mySchema
}
});
```
## putAttachment()
Adds an attachment to a `RxDocument`. Returns a Promise with the new attachment.
```javascript
import { createBlob } from 'rxdb';
const attachment = await myDocument.putAttachment(
{
id: 'cat.txt', // (string) name of the attachment
data: createBlob('meowmeow', 'text/plain'), // (Blob) data of the attachment
type: 'text/plain' // (string) type of the attachment
// data like 'image/jpeg'
}
);
```
:::warning
Expo/React-Native does not support the `Blob` API natively. Make sure you use your own polyfill that properly supports `blob.arrayBuffer()` when using RxAttachments or use the `putAttachmentBase64()` and `getDataBase64()` so that you do not have to create blobs.
:::
## putAttachments()
Write multiple attachments to a `RxDocument` in a single atomic operation. This is more efficient than calling `putAttachment()` multiple times because it only performs one write to the storage. Returns a Promise with an array of the new attachments.
```javascript
import { createBlob } from 'rxdb';
const attachments = await myDocument.putAttachments([
{
id: 'cat.txt',
data: createBlob('meowmeow', 'text/plain'),
type: 'text/plain'
},
{
id: 'dog.txt',
data: createBlob('woof', 'text/plain'),
type: 'text/plain'
}
]);
```
## putAttachmentBase64()
Same as `putAttachment()` but accepts a plain base64 string instead of a `Blob`.
```ts
const attachment = await doc.putAttachmentBase64({
id: 'cat.txt',
length: 4,
data: 'bWVvdw==',
type: 'text/plain'
});
```
## getAttachment()
Returns an `RxAttachment` by its id. Returns `null` when the attachment does not exist.
```javascript
const attachment = myDocument.getAttachment('cat.jpg');
```
## allAttachments()
Returns an array of all attachments of the `RxDocument`.
```javascript
const attachments = myDocument.allAttachments();
```
## allAttachments$
Gets an Observable which emits a stream of all attachments from the document. Re-emits each time an attachment gets added or removed from the [RxDocument](./rx-document.md).
```javascript
const all = [];
myDocument.allAttachments$.subscribe(
attachments => all = attachments
);
```
## RxAttachment
The attachments of RxDB are represented by the type `RxAttachment` which has the following attributes/methods.
### doc
The `RxDocument` which the attachment is assigned to.
### id
The id as `string` of the attachment.
### type
The type as `string` of the attachment.
### length
The length of the data of the attachment as `number`.
### digest
The hash of the attachments data as `string`.
:::note
The digest is NOT calculated by RxDB, instead it is calculated by the RxStorage. The only guarantee is that the digest will change when the attachments data changes.
:::
### rev
The revision-number of the attachment as `number`.
### remove()
Removes the attachment. Returns a Promise that resolves when done.
```javascript
const attachment = myDocument.getAttachment('cat.jpg');
await attachment.remove();
```
## getData()
Returns a Promise which resolves the attachment's data as `Blob`. (async)
```javascript
const attachment = myDocument.getAttachment('cat.jpg');
const blob = await attachment.getData(); // Blob
```
## getDataBase64()
Returns a Promise which resolves the attachment's data as **base64** `string`.
```javascript
const attachment = myDocument.getAttachment('cat.jpg');
const base64Database = await attachment.getDataBase64(); // 'bWVvdw=='
```
## getStringData()
Returns a Promise which resolves the attachment's data as `string`.
```javascript
const attachment = await myDocument.getAttachment('cat.jpg');
const data = await attachment.getStringData(); // 'meow'
```
## Inline attachments on insert and upsert
Instead of inserting a document first and then calling `putAttachment()` separately, you can include attachments directly in the document data when using `insert()`, `bulkInsert()`, `upsert()`, `bulkUpsert()`, or `incrementalUpsert()`. Provide `_attachments` as an array of `{ id, type, data }` objects.
```javascript
import { createBlob } from 'rxdb';
// insert with inline attachments
const doc = await myCollection.insert({
name: 'foo',
_attachments: [
{
id: 'photo.jpg',
type: 'image/jpeg',
data: myJpegBlob
},
{
id: 'notes.txt',
type: 'text/plain',
data: createBlob('some notes', 'text/plain')
}
]
});
const attachment = doc.getAttachment('photo.jpg');
```
### Upsert behavior with attachments
When upserting a document that already exists, attachments from the new data are **merged** with the document's existing attachments by default. This means existing attachments not mentioned in the upsert data are preserved.
To replace all existing attachments instead, pass `{ deleteExistingAttachments: true }` as the second argument:
```javascript
// Merge (default): keeps existing attachments, adds/updates new ones
const doc = await myCollection.upsert(docData);
// Replace: only the attachments in docData will exist after the upsert
const doc2 = await myCollection.upsert(docData, { deleteExistingAttachments: true });
```
This option works with `upsert()`, `bulkUpsert()`, and `incrementalUpsert()`.
## Attachment compression {#attachment-compression}
Storing many attachments can be a problem when the disc space of the device is exceeded.
Therefore it can make sense to compress the attachments before storing them in the [RxStorage](./rx-storage.md).
With the `attachments-compression` plugin you can compress the attachments data on write and decompress it on reads.
This happens internally and will not change how you use the API. The compression is run with the [Compression Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API) which is only supported on [newer browsers](https://caniuse.com/?search=compressionstream).
## MIME-type-aware compression
The compression plugin is MIME-type-aware. It only compresses attachment types that benefit from compression (text, JSON, SVG, etc.) and passes through already-compressed formats (JPEG, PNG, MP4, etc.) as-is. This avoids wasting CPU cycles on files that won't shrink.
A built-in default list of compressible types is used automatically. You can override it with the `compressibleTypes` option in your schema:
```ts
import {
wrappedAttachmentsCompressionStorage
} from 'rxdb/plugins/attachments-compression';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
// create a wrapped storage with attachment-compression.
const storageWithAttachmentsCompression = wrappedAttachmentsCompressionStorage({
storage: getRxStorageIndexedDB()
});
const db = await createRxDatabase({
name: 'mydatabase',
storage: storageWithAttachmentsCompression
});
// set the compression mode at the schema level
const mySchema = {
version: 0,
type: 'object',
properties: {
// ..
},
attachments: {
// Specify the compression mode.
// OneOf ['deflate', 'gzip']
compression: 'deflate',
// Optional: override which MIME types get compressed.
// Supports wildcard prefix matching
// (e.g. 'text/*' matches 'text/plain',
// 'text/html', etc.).
// If omitted, a built-in default list of compressible types is used.
compressibleTypes: [
'text/*',
'application/json',
'application/xml',
'image/svg+xml'
// ... add your own patterns
]
}
};
/* ... create your collections as usual and store attachments in them. */
```
The default compressible types include the following MIME type patterns. Binary formats like `image/jpeg`, `image/png`, `video/*`, and `audio/*` are **not** in the default list and will be stored without re-compression.
---
## RxPipeline - Automate Data Flows in RxDB
# RxPipeline
The RxPipeline plugin enables you to run operations depending on writes to a collection.
Whenever a write happens on the source collection of a pipeline, a handler is called to process the writes and run operations on another collection.
You could have a similar behavior by observing the collection stream and process data on emits:
```ts
mySourceCollection.$.subscribe(event => {/* ...process...*/});
```
While this could work in some cases, it causes many problems that are fixed by using the pipeline plugin instead:
- In an RxPipeline, only the [Leading Instance](./leader-election.md) runs the operations. For example when you have multiple browser tabs open, only one will run the processing and when that tab is closed, another tab will become elected leader and continue the pipeline processing.
- On sudden stops and restarts of the JavaScript process, the processing will continue at the correct checkpoint and not miss out any documents even on unexpected crashes.
- Reads/Writes on the destination collection are halted while the pipeline is processing. This ensures your queries only return fully processed documents and no partial results. So when you run a query to the destination collection directly after a write to the source collection, you can be sure your query results are up to date and the pipeline has already been run at the moment the query resolved:
```ts
await mySourceCollection.insert({/* ... */});
/**
* Because our pipeline blocks reads to the destination,
* we know that the result array contains data created
* on top of the previously inserted documents.
*/
const result = await myDestinationCollection.find().exec();
```
## Creating a RxPipeline
Pipelines are created on top of a source [RxCollection](./rx-collection.md) and have another `RxCollection` as destination. An identifier is used to identify the state of the pipeline so that different pipelines have a different processing checkpoint state. A plain JavaScript function `handler` is used to process the data of the source collection writes.
```ts
const pipeline = await mySourceCollection.addPipeline({
identifier: 'my-pipeline',
destination: myDestinationCollection,
handler: async (docs) => {
/**
* Here you can process the documents and write to
* the destination collection.
*/
for (const doc of docs) {
await myDestinationCollection.insert({
id: doc.primary,
category: doc.category
});
}
}
});
```
## Use Cases for RxPipeline
The RxPipeline is a handy building block for different features and plugins. You can use it to aggregate data or restructure local data.
### UseCase: Re-Index data that comes from replication
Sometimes you want to [replicate](./replication.md) atomic documents over the wire but locally you want to split these documents for better indexing. For example you replicate email documents that have multiple receivers in a string-array. While string-arrays cannot be indexed, locally you need a way to query for all emails of a given receiver.
To handle this case you can set up a RxPipeline that writes the mapping into a separate collection:
```ts
const pipeline = await emailCollection.addPipeline({
identifier: 'map-email-receivers',
destination: emailByReceiverCollection,
handler: async (docs) => {
for (const doc of docs) {
// remove previous mapping
await emailByReceiverCollection.find({emailId: doc.primary}).remove();
// add new mapping
if(!doc.deleted) {
await emailByReceiverCollection.bulkInsert(
doc.receivers.map(receiver => ({
emailId: doc.primary,
receiver: receiver
}))
);
}
}
}
});
```
With this you can efficiently query for "all emails that a person received" by running:
```ts
const mailIds = await emailByReceiverCollection.find({
receiver: 'foobar@example.com'
}).exec();
```
### UseCase: Fulltext Search
You can utilize the pipeline plugin to index text data for efficient [fulltext search](./fulltext-search.md).
```ts
const pipeline = await emailCollection.addPipeline({
identifier: 'email-fulltext-search',
destination: mailByWordCollection,
handler: async (docs) => {
for (const doc of docs) {
// remove previous mapping
await mailByWordCollection.find({emailId: doc.primary}).remove();
// add new mapping
if(!doc.deleted) {
const words = doc.text.split(' ');
await mailByWordCollection.bulkInsert(
words.map(word => ({
emailId: doc.primary,
word: word
}))
);
}
}
}
});
```
With this you can efficiently query for "all emails that contain a given word" by running:
```ts
const mailIds = await emailByReceiverCollection.find({word: 'foobar'}).exec();
```
### UseCase: Download data based on source documents
When you have to fetch data for each document of a collection from a server, you can use the pipeline to ensure all documents have their data downloaded and no document is missed.
```ts
const pipeline = await emailCollection.addPipeline({
identifier: 'download-data',
destination: serverDataCollection,
handler: async (docs) => {
for (const doc of docs) {
const response = await fetch('https://example.com/doc/' + doc.primary);
const serverData = await response.json();
await serverDataCollection.upsert({
id: doc.primary,
data: serverData
});
}
}
});
```
## RxPipeline methods
### awaitIdle()
You can await the idleness of a pipeline with `await myRxPipeline.awaitIdle()`. This will await a promise that resolves when the pipeline has processed all documents and is not running anymore.
### close()
`await myRxPipeline.close()` stops the pipeline so that it is no longer doing stuff. This is automatically called when the RxCollection or [RxDatabase](./rx-database.md) of the pipeline is closed.
### remove()
`await myRxPipeline.remove()` removes the pipeline and all metadata which it has stored. Recreating the pipeline afterwards will start processing all source documents from scratch.
## Using RxPipeline correctly
### Pipeline handlers must be idempotent
Because a JavaScript process can exit at any time, like when the user closes a browser tab, the pipeline handler function must be idempotent. This means when it only runs partially and is started again with the same input, it should still end up in the correct result.
### Pipeline handlers must not throw
Pipeline handlers must never throw. If you run operations inside of the handler that might cause errors, you must wrap the handler's code with a `try-catch` by yourself and also handle retries. If your handler throws, your pipeline will be stuck and no longer be usable, which should never happen.
### Be careful when doing http requests in the handler
When you run http requests inside of your handler, you no longer have an [offline first](./offline-first.md) application because reads to the destination collection will be blocked until all handlers have finished. When your client is offline, therefore the collection will be blocked for reads and writes.
### Pipelines temporarily block external reads and writes
While a pipeline is running, **all reads and writes to its destination collection are blocked**. This guarantees that queries never observe partially processed data, but it also means that pipelines can block each other if they interact incorrectly.
Problems occur when multiple pipelines:
- read or write across the same collections, or
- wait for each other using `awaitIdle()` from inside a pipeline handler.
```ts
// Example of a deadlock
// Pipeline A: files β files (reads folders)
const pipelineA = await db.files.addPipeline({
identifier: 'file-path-sync',
destination: db.files,
handler: async (docs) => {
const folders = await folders.find().exec(); // can block
/* ... */
}
});
// Pipeline B: files β folders (waits for A)
await db.folders.addPipeline({
identifier: 'file-count',
destination: db.folders,
handler: async () => {
await pipelineA.awaitIdle(); // β may deadlock
/* ... */
}
});
```
To prevent deadlocks, consider:
- Never call `awaitIdle()` inside a pipeline handler.
- Avoid circular dependencies between pipelines.
- Prefer one-directional data flow.
---
## Signals & Custom Reactivity with RxDB
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
# Signals & Co. - Custom reactivity adapters instead of RxJS Observables
RxDB internally uses the [rxjs library](https://rxjs.dev/) for observables and streams. All functionalities of RxDB like [query](./rx-query.md#observe) results or [document fields](./rx-document.md#observe) that expose values that change over time return a rxjs `Observable` that allows you to observe the values and update your UI accordingly depending on the changes to the database state.
However there are many reasons to use other reactivity libraries that use a different datatype to represent changing values. For example when you use **signals** in angular or react, the **template refs** of vue or state libraries like MobX and redux.
RxDB allows you to pass a custom reactivity factory on [RxDatabase](./rx-database.md) creation so that you can easily access values wrapped with your custom datatype in a convenient way.
## Adding a reactivity factory
### Angular
In angular we use [Angular Signals](https://angular.dev/guide/signals) as custom reactivity objects.
#### Import
```ts
import { createReactivityFactory } from 'rxdb/plugins/reactivity-angular';
import { Injectable, inject } from '@angular/core';
```
#### Set the reactivity factory
Set the factory as `reactivity` option when calling `createRxDatabase`.
```ts
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: createReactivityFactory(inject(Injector))
});
// add collections/sync etc...
```
#### Use the Signal in an Angular component
```ts
import { Component, inject } from '@angular/core';
import { CommonModule } from '@angular/common';
import { DbService } from '../db.service';
@Component({
selector: 'app-todos-list',
standalone: true,
imports: [CommonModule],
template: `
{{ t.title }}
`,
})
export class TodosListComponent {
private dbService = inject(DbService);
// RxDB query - Angular Signal
readonly todosSignal = this.dbService.db.todos.find().$$;
}
```
An example of how signals are used in angular with RxDB, can be found at the [RxDB Angular Example](https://github.com/pubkey/rxdb/blob/master/examples/angular/src/app/components/heroes-list/heroes-list.component.ts#L46)
### React
For React, we use the [Preact Signals](https://preactjs.com/guide/v10/signals/) for custom reactivity.
#### Install Preact Signals
```bash
npm install @preact/signals-core --save
```
#### Import
```ts
import {
PreactSignalsRxReactivityFactory
} from 'rxdb/plugins/reactivity-preact-signals';
```
#### Set the reactivity factory
```ts
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: PreactSignalsRxReactivityFactory
});
// add collections/sync etc...
```
#### Use the Signal in a React component
```tsx
import { useEffect, useState } from 'preact/hooks';
import { getDatabase } from './db';
export function TodosList() {
const [db, setDb] = useState(null);
useEffect(() => {
getDatabase().then(setDb);
}, []);
if (!db) return null;
// RxQuery -> Preact Signal
const todosSignal = db.todos.find().$$;
return (
{todosSignal.value.map((doc: any) => (
{doc.title}
))}
);
}
```
### Vue
For Vue, we use the [Vue Shallow Refs](https://vuejs.org/api/reactivity-advanced) for custom reactivity.
#### Import
```ts
import { VueRxReactivityFactory } from 'rxdb/plugins/reactivity-vue';
```
#### Set the reactivity factory
```ts
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: VueRxReactivityFactory
});
// add collections/sync etc...
```
#### Use the Shallow Ref in a Vue component
```html
```
## Accessing custom reactivity objects
All observable data in RxDB is marked by the single dollar sign `$` like [RxCollection](./rx-collection.md).$ for events or `RxDocument.myField$` to get the observable for a document field. To make custom reactivity objects distinguable, they are marked with double-dollar signs `$$` instead. Here are some example on how to get custom reactivity objects from RxDB specific instances:
```ts
// RxDocument
// get signal that represents the document field 'foobar'
const signal = myRxDocument.get$$('foobar');
// same as above
const signal = myRxDocument.foobar$$;
// get signal that represents whole document over time
const signal = myRxDocument.$$;
// get signal that represents the deleted state of the document
const signal = myRxDocument.deleted$$;
```
```ts
// RxQuery
// get signal that represents the query result set over time
const signal = collection.find().$$;
// get signal that represents the query result set over time
const signal = collection.findOne().$$;
```
```ts
// RxLocalDocument
// get signal that represents the whole local document state
const signal = myRxLocalDocument.$$;
// get signal that represents the foobar field
const signal = myRxLocalDocument.get$$('foobar');
```
---
## RxState - Reactive Persistent State with RxDB
import {Faq, FaqItem} from '@site/src/components/faq';
# RxState - Reactive Persistent State with RxDB
RxState is a flexible state library build on top of the [RxDB Database](https://rxdb.info/). While RxDB stores similar documents inside of collections, RxState can store any complex JSON data without having a predefined schema.
The state is automatically persisted through RxDB and states changes are propagated between browser tabs. Even setting up replication is simple by using the RxDB [Replication feature](./replication.md).
## Creating a RxState
A `RxState` instance is created on top of a [RxDatabase](./rx-database.md). The state will automatically be persisted with the [storage](./rx-storage.md) that was used when setting up the RxDatabase. To use it you first have to import the `RxDBStatePlugin` and add it to RxDB with `addRxPlugin()`.
To create a state call the `addState()` method on the database instance. Calling `addState` multiple times will automatically de-duplicated and only create a single RxState object.
```javascript
import { createRxDatabase, addRxPlugin } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// first add the RxState plugin to RxDB
import { RxDBStatePlugin } from 'rxdb/plugins/state';
addRxPlugin(RxDBStatePlugin);
const database = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
});
// create a state instance
const myState = await database.addState();
// you can also create states with a given namespace
const myChildState = await database.addState('myNamepsace');
```
## Writing data and Persistence
Writing data to the state happen by a so called `modifier`. It is a simple JavaScript function that gets the current value as input and returns the new, modified value.
For example to increase the value of `myField` by one, you would use a modifier that increases the current value:
```ts
// initially set value to zero
await myState.set('myField', v => 0);
// increase value by one
await myState.set('myField', v => v + 1);
// update value to be 42
await myState.set('myField', v => 42);
```
The modifier is used instead of a direct assignment to ensure correct behavior when other JavaScript realms write to the state at the same time, like other browser tabs or webworkers. On conflicts, the modifier will just be run again to ensure deterministic and correct behavior. Therefore mutation is `async`, you have to `await` the call to the set function when you care about the moment when the change actually happened.
## Get State Data
The state stored inside of a RxState instance can be seen as a big single JSON object that contains all data.
You can fetch the whole object or partially get a single properties or nested ones.
Fetching data can either happen with the `.get()` method or by accessing the field directly like `myRxState.myField`.
```ts
// get root state data
const val = myState.get();
// get single property
const val = myState.get('myField');
const val = myState.myField;
// get nested property
const val = myState.get('myField.childfield');
const val = myState.myField.childfield;
// get nested array property
const val = myState.get('myArrayField[0].foobar');
const val = myState.myArrayField[0].foobar;
```
## Observability
Instead of fetching the state once, you can also observe the state with either rxjs observables or [custom reactivity handlers](#rxstate-with-signals-and-hooks) like signals or hooks.
Rxjs observables can be created by either using the `.get$()` method or by accessing the top level property suffixed with a dollar sign like `myState.myField$`.
```ts
const observable = myState.get$('myField');
const observable = myState.myField$;
// then you can subscribe to that observable
observable.subscribe(newValue => {
// update the UI
});
```
Subscription works across multiple JavaScript realms like browser tabs or Webworkers.
## RxState with signals and hooks
With the double-dollar sign you can also access [custom reactivity](./reactivity.md) instances like signals or hooks. These are easier to use compared to rxjs, depending on which JavaScript framework you are using.
For example in angular to use signals, you would first add a reactivity factory to your database and then access the signals of the RxState:
```ts
import { RxReactivityFactory, createRxDatabase } from 'rxdb/plugins/core';
import { toSignal } from '@angular/core/rxjs-interop';
const reactivityFactory: RxReactivityFactory = {
fromObservable(obs, initialValue) {
return toSignal(obs, { initialValue });
}
};
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: reactivityFactory
});
const myState = await database.addState();
const mySignal = myState.get$$('myField');
const mySignal = myState.myField$$;
```
## Cleanup RxState operations
For faster writes, changes to the state are only written as list of operations to disc. After some time you might have too
many operations written which would delay the initial state creation. To automatically merge the state operations into a single operation and clear the old operations, you should add the [Cleanup Plugin](./cleanup.md) before creating the [RxDatabase](./rx-database.md):
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBCleanupPlugin } from 'rxdb/plugins/cleanup';
addRxPlugin(RxDBCleanupPlugin);
```
## Correctness over Performance
RxState is optimized for correctness, not for performance. Compared to other state libraries, RxState directly persists data to storage and ensures write conflicts are handled properly. Other state libraries are handles mainly in-memory and lazily persist to disc without caring about conflicts or multiple browser tabs which can cause problems and hard to reproduce bugs.
RxState still uses RxDB which has a range of [great performing storages](./rx-storage-performance.md) so the write speed is more than sufficient. Also to further improve write performance you can use more RxState instances (with an different namespace) to split writes across multiple storage instances.
Reads happen directly in-memory which makes RxState read performance comparable to other state libraries.
## RxState Replication
Because the state data is stored inside of an internal [RxCollection](./rx-collection.md) you can easily use the [RxDB Replication](./replication.md) to sync data between users or devices of the same user.
For example with the [P2P WebRTC replication](./replication-webrtc.md) you can start the replication on the collection and automatically sync the RxState operations between users directly:
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
const database = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
});
const myState = await database.addState();
const replicationPool = await replicateWebRTC(
{
collection: myState.collection,
topic: 'my-state-replication-pool',
connectionHandlerCreator: getConnectionHandlerSimplePeer({}),
pull: {},
push: {}
}
);
```
## FAQ
RxState can be synced.
RxState is used for persisted on-page state like "is element toggled" while [LocalDocuments](./rx-local-document.md) are more for logic-state like user-settings.
RxState is a complex object while LocalDocuments are a key-object store.
LocalDocuments can be modified like any other RxDocument with conflict handling and incremental writes while RxState has its own API.
RxState is mapped fully into memory while LocalDocuments are in memory only when needed. For big data, LocalDocuments should be used.
RxState is stored per RxDatabase while LocalDocuments can be stored either per RxDatabase or RxCollection.
You should use RxState when you need state that automatically persists, synchronizes across browser tabs, or replicates between devices. While traditional in-memory stores require boilerplate replication logic and manual persistence, RxState handles these features out of the box.
Using a modifier function guarantees deterministic conflict resolution when multiple JavaScript realms (like WebWorkers or multiple browser tabs) attempt to update the state simultaneously. It ensures the state evaluates correctly even under concurrent modifications.
No, RxState is schema-less by default. Unlike standard RxDB collections, it accepts any complex JSON data without requiring a rigid schema definition, allowing for flexible state updates.
---
## Master Local Documents in RxDB
# Local Documents
Local documents are a special class of documents which are used to store local metadata.
They come in handy when you want to store settings or additional data next to your documents.
- Local Documents can exist on a [RxDatabase](./rx-database.md) or [RxCollection](./rx-collection.md).
- Local Document do not have to match the collections schema.
- Local Documents do not get replicated.
- Local Documents will not be found on queries.
- Local Documents can not have [attachments](./rx-attachment.md).
- Local Documents will not get handled by the [migration-schema](./migration-schema.md).
- The id of a local document has the `maxLength` of `128` characters.
:::note
While local documents can be very useful, in many cases the [RxState](./rx-state.md) API is more convenient.
:::
## Add the local documents plugin
To enable the local documents, you have to add the `local-documents` plugin.
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBLocalDocumentsPlugin } from 'rxdb/plugins/local-documents';
addRxPlugin(RxDBLocalDocumentsPlugin);
```
## Activate the plugin for a RxDatabase or RxCollection
For better performance, the local document plugin does not create a storage for every database or collection that is created.
Instead you have to set `localDocuments: true` when you want to store local documents in the instance.
```js
// activate local documents on a RxDatabase
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageLocalstorage(),
localDocuments: true // <- activate this to store local documents in the database
});
myDatabase.addCollections({
messages: {
schema: messageSchema,
// activate this to store local documents
// in the collection
localDocuments: true
}
});
```
:::note
If you want to store local documents in a `RxCollection` but **NOT** in the `RxDatabase`, you **MUST NOT** set `localDocuments: true` in the `RxDatabase` because it will only slow down the initial database creation.
:::
## insertLocal()
Creates a local document for the database or collection. Throws if a local document with the same id already exists. Returns a Promise which resolves the new `RxLocalDocument`.
```javascript
const localDoc = await myCollection.insertLocal(
'foobar', // id
{ // data
foo: 'bar'
}
);
// you can also use local-documents on a database
const localDoc = await myDatabase.insertLocal(
'foobar', // id
{ // data
foo: 'bar'
}
);
```
## upsertLocal()
Creates a local document for the database or collection if not exists. Overwrites the if exists. Returns a Promise which resolves the `RxLocalDocument`.
```javascript
const localDoc = await myCollection.upsertLocal(
'foobar', // id
{ // data
foo: 'bar'
}
);
```
## getLocal()
Find a `RxLocalDocument` by its id. Returns a Promise which resolves the `RxLocalDocument` or `null` if not exists.
```javascript
const localDoc = await myCollection.getLocal('foobar');
```
## getLocal$()
Like `getLocal()` but returns an `Observable` that emits the document or `null` if not exists.
```javascript
const subscription = myCollection.getLocal$('foobar').subscribe(documentOrNull => {
console.dir(documentOrNull); // > RxLocalDocument or null
});
```
## RxLocalDocument
A `RxLocalDocument` behaves like a normal `RxDocument`.
```javascript
const localDoc = await myCollection.getLocal('foobar');
// access data
const foo = localDoc.get('foo');
// change data
localDoc.set('foo', 'bar2');
await localDoc.save();
// observe data
localDoc.get$('foo').subscribe(value => { /* .. */ });
// remove it
await localDoc.remove();
```
:::note
Because the local document does not have a schema, accessing the documents data-fields via pseudo-proxy will not work.
:::
```javascript
const foo = localDoc.foo; // undefined
const foo = localDoc.get('foo'); // works!
localDoc.foo = 'bar'; // does not work!
localDoc.set('foo', 'bar'); // works
```
For the usage with typescript, you can have access to the typed data of the document over `toJSON()`
```ts
declare type MyLocalDocumentType = {
foo: string
}
const localDoc = await myCollection.upsertLocal(
'foobar', // id
{ // data
foo: 'bar'
}
);
// typescript will know that foo is a string
const foo: string = localDoc.toJSON().foo;
```
---
## Cleanup
import {Faq, FaqItem} from '@site/src/components/faq';
# π§Ή Cleanup
To make the [replication](./replication.md) work, and for other reasons, RxDB has to keep deleted documents in storage so that it can replicate their deletion state.
This ensures that when a client is [offline](./offline-first.md), the deletion state is still known and can be replicated with the backend when the client goes online again.
Keeping too many deleted documents in the storage, can slow down queries or fill up too much disc space.
With the cleanup plugin, RxDB will run cleanup cycles that clean up deleted documents when it can be done safely.
## Installation
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBCleanupPlugin } from 'rxdb/plugins/cleanup';
addRxPlugin(RxDBCleanupPlugin);
```
## Create a database with cleanup options
You can set a specific cleanup policy when a [RxDatabase](./rx-database.md) is created. For most use cases, the defaults should be ok.
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
cleanupPolicy: {
/**
* The minimum time in milliseconds for how long
* a document has to be deleted before it is
* purged by the cleanup.
* [default=one month]
*/
minimumDeletedTime: 1000 * 60 * 60 * 24 * 31, // one month,
/**
* The minimum amount of time that the RxCollection must have existed.
* This ensures that at the initial page load, more important
* tasks are not slowed down because a cleanup process is running.
* [default=60 seconds]
*/
minimumCollectionAge: 1000 * 60, // 60 seconds
/**
* After the initial cleanup is done,
* a new cleanup is started after [runEach] milliseconds
* [default=5 minutes]
*/
runEach: 1000 * 60 * 5, // 5 minutes
/**
* If set to true,
* RxDB will await all running replications
* to not have a replication cycle running.
* This ensures we do not remove deleted documents
* when they might not have already been replicated.
* [default=true]
*/
awaitReplicationsInSync: true,
/**
* If true, it will only start the cleanup
* when the current instance is also the leader.
* This ensures that when RxDB is used in multiInstance mode,
* only one instance will start the cleanup.
* [default=true]
*/
waitForLeadership: true
}
});
```
## Calling cleanup manually
You can manually run a cleanup per collection by calling [RxCollection](./rx-collection.md).cleanup().
```ts
/**
* Manually run the cleanup with the
* minimumDeletedTime from the cleanupPolicy.
*/
await myRxCollection.cleanup();
/**
* Overwrite the minimumDeletedTime
* be setting it explicitly (time in milliseconds)
*/
await myRxCollection.cleanup(1000);
/**
* Purge all deleted documents no
* matter when they were deleted
* by setting minimumDeletedTime to zero.
*/
await myRxCollection.cleanup(0);
```
## Using the cleanup plugin to empty a collection
When you have a collection with documents and you want to empty it by purging all documents, the recommended way is to call `myRxCollection.remove()`. However, this will destroy the JavaScript class of the collection and stop all listeners and observables.
Sometimes the better option might be to manually delete all documents and then use the cleanup plugin to purge the deleted documents:
```ts
// delete all documents
await myRxCollection.find().remove();
// purge all deleted documents
await myRxCollection.cleanup(0);
```
## FAQ
The cleanup cycles are optimized to run only when the database is idle and it is unlikely that another database interaction performance will be decreased in the meantime. For example, by default, the cleanup does not run in the first 60 seconds of the creation of a collection to ensure the initial page load of your website does not slow down. Also, we use mechanisms like the `requestIdleCallback()` API to improve the correct timing of the cleanup cycle.
---
## Backup
# π₯ Backup Plugin
With the backup plugin you can write the current database state and ongoing changes into folders on the filesystem.
The files are written in plain json together with their [attachments](./rx-attachment.md) so that you can read them out with any software or tools, without being bound to RxDB.
This is useful to:
- Consume the database content with other software that cannot replicate with RxDB
- Write a backup of the database to a remote server by mounting the backup folder on the other server.
The backup plugin works only in [node.js](./nodejs-database.md), not in a browser. It is intended to have a backup strategy when using RxDB on the server side like with the [RxServer](./rx-server.md). To run backups on the client side, you should use one of the [replication](./replication.md) plugins instead.
## Installation
```javascript
import { addRxPlugin } from 'rxdb';
import { RxDBBackupPlugin } from 'rxdb/plugins/backup';
addRxPlugin(RxDBBackupPlugin);
```
## one-time backup
Write the whole database to the filesystem **once**.
When called multiple times, it will continue from the last checkpoint and not start all over again.
```javascript
const backupOptions = {
// if false, a one-time backup will be written
live: false,
// the folder where the backup will be stored
directory: '/my-backup-folder/',
// if true, attachments will also be saved
attachments: true
}
const backupState = myDatabase.backup(backupOptions);
await backupState.awaitInitialBackup();
// call again to run from the last checkpoint
const backupState2 = myDatabase.backup(backupOptions);
await backupState2.awaitInitialBackup();
```
## live backup
When `live: true` is set, the backup will write all ongoing changes to the backup directory.
```javascript
const backupOptions = {
// set live: true to have an ongoing backup
live: true,
directory: '/my-backup-folder/',
attachments: true
}
const backupState = myDatabase.backup(backupOptions);
// you can still await the initial backup write,
// but further changes will still be processed.
await backupState.awaitInitialBackup();
```
## writeEvents$
You can listen to the `writeEvents$` Observable to get notified about written backup files.
```javascript
const backupOptions = {
live: false,
directory: '/my-backup-folder/',
attachments: true
}
const backupState = myDatabase.backup(backupOptions);
const subscription = backupState.writeEvents$
.subscribe(writeEvent =>
console.dir(writeEvent)
);
/*
> {
collectionName: 'humans',
documentId: 'foobar',
files: [
'/my-backup-folder/foobar/document.json'
],
deleted: false
}
*/
```
## Limitations
- It is currently not possible to import from a written backup. If you need this functionality, please make a pull request.
---
## Leader Election
import {CenteredImage} from '@site/src/components/centered-image';
# Leader-Election
RxDB comes with a leader-election which elects a leading instance between different instances in the same javascript runtime.
Before you read this, please check out on how many of your open browser-tabs you have opened the same website more than once. Count them, I will wait..
So if you would now inspect the traffic that these open tabs produce, you can see that many of them send exact the same data over wire for every tab. No matter if the data is sent with an open websocket or by polling.
## Use-case-example
Imagine we have a website which displays the current temperature of the visitors location in various charts, numbers or heatmaps. To always display the live-data, the website opens a [websocket](./articles/websockets-sse-polling-webrtc-webtransport.md) to our API-Server which sends the current temperature every 10 seconds. Using the way most sites are currently build, we can now open it in 5 browser-tabs and it will open 5 websockets which send data 6*5=30 times per minute. This will not only waste the power of your clients device, but also wastes your api-servers resources by opening redundant connections.
## Solution
The solution to this redundancy is the usage of a [leader-election](https://en.wikipedia.org/wiki/Leader_election)-algorithm which makes sure that always exactly one tab is managing the remote-data-access. The managing tab is the elected leader and stays leader until it is closed. No matter how many tabs are opened or closed, there must be always exactly **one** leader.
You could now start implementing a messaging-system between your browser-tabs, hand out which one is leader, solve conflicts and reassign a new leader when the old one 'dies'.
Or just use RxDB which does all these things for you.
## Add the leader election plugin
To enable the leader election, you have to add the `leader-election` plugin.
```javascript
import { addRxPlugin } from 'rxdb';
import { RxDBLeaderElectionPlugin } from 'rxdb/plugins/leader-election';
addRxPlugin(RxDBLeaderElectionPlugin);
```
## Code-example
To make it easy, here is an example where the temperature is pulled every ten seconds and saved to a collection. The pulling starts at the moment where the opened tab becomes the leader.
```javascript
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'weatherDB',
storage: getRxStorageLocalstorage(),
password: 'myPassword',
multiInstance: true
});
await db.addCollections({
temperature: {
schema: mySchema
}
});
db.waitForLeadership()
.then(() => {
console.log('Long lives the king!'); // <- runs when db becomes leader
setInterval(async () => {
const temp = await fetch('https://example.com/api/temp/');
db.temperature.insert({
degrees: temp,
time: new Date().getTime()
});
}, 1000 * 10);
});
```
## Handle Duplicate Leaders
On rare occasions, it can happen that [more than one leader](https://github.com/pubkey/broadcast-channel/blob/master/.github/README.md#handle-duplicate-leaders) is elected. This can happen when the CPU is on 100% or for any other reason the JavaScript process is fully blocked for a long time.
For most cases this is not really a problem because on duplicate leaders, both browser tabs replicate with the same backend anyways.
To handle the duplicate leader event, you can access the leader elector and set a handler:
```ts
import {
getLeaderElectorByBroadcastChannel
} from 'rxdb/plugins/leader-election';
const leaderElector = getLeaderElectorByBroadcastChannel(broadcastChannel);
leaderElector.onduplicate = async () => {
// Duplicate leader detected -> reload the page.
location.reload();
}
```
## Live-Example
In this example the leader is marked with the crown β
## Try it out
Run the [angular-example](https://github.com/pubkey/rxdb/tree/master/examples/angular) where the leading tab is marked with a crown on the top-right-corner.
## Notice
The leader election is implemented via the [broadcast-channel module](https://github.com/pubkey/broadcast-channel#using-the-leaderelection).
The leader is elected between different processes on the same javascript-runtime. Like multiple tabs in the same browser or multiple Node.js processes on the same machine. It will not run between different replicated instances.
---
## Streamlined RxDB Middleware
# Middleware
RxDB middleware-hooks (also called pre and post hooks) are functions which are passed control during execution of asynchronous functions.
The hooks are specified on RxCollection-level and help to create a clear what-happens-when-structure of your code.
Hooks can be defined to run **parallel** or **in series** one after another.
Hooks can be **synchronous** or **asynchronous** when they return a `Promise`.
To stop the operation at a specific hook, throw an error.
## List
RxDB supports the following hooks:
- preInsert
- postInsert
- preSave
- postSave
- preRemove
- postRemove
- postCreate
### Why is there no validate-hook?
Different to mongoose, the validation on document-data is running on the field-level for every change to a document.
This means if you set the value `lastName` of a [RxDocument](./rx-document.md), then the validation will only run on the changed field, not the whole document.
Therefore it is not useful to have validate-hooks when a document is written to the database.
## Use Cases
Middleware is useful for atomizing model logic and avoiding nested blocks of async code.
Here are some other ideas:
- complex validation
- removing dependent documents
- asynchronous defaults
- asynchronous tasks that a certain action triggers
- triggering custom events
- notifications
## Usage
All hooks have the plain data as first parameter, and all but `preInsert` also have the `RxDocument`-instance as second parameter. If you want to modify the data in the hook, change attributes of the first parameter.
All hook functions are also `this`-bound to the `RxCollection`-instance.
### Insert
An insert-hook receives the data-object of the new document.
#### lifecycle
- RxCollection.insert is called
- preInsert series-hooks
- preInsert parallel-hooks
- [schema validation](./schema-validation.md) runs
- new document is written to database
- postInsert series-hooks
- postInsert parallel-hooks
- event is emitted to [RxDatabase](./rx-database.md) and [RxCollection](./rx-collection.md)
#### preInsert
```js
// series
myCollection.preInsert(function(plainData){
// set age to 50 before saving
plainData.age = 50;
}, false);
// parallel
myCollection.preInsert(function(plainData){
}, true);
// async
myCollection.preInsert(function(plainData){
return new Promise(res => setTimeout(res, 100));
}, false);
// stop the insert-operation
myCollection.preInsert(function(plainData){
throw new Error('stop');
}, false);
```
#### postInsert
```js
// series
myCollection.postInsert(function(plainData, rxDocument){
}, false);
// parallel
myCollection.postInsert(function(plainData, rxDocument){
}, true);
// async
myCollection.postInsert(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
```
### Save
A save-hook receives the document which is saved.
#### lifecycle
- RxDocument.save is called
- preSave series-hooks
- preSave parallel-hooks
- updated document is written to database
- postSave series-hooks
- postSave parallel-hooks
- event is emitted to RxDatabase and RxCollection
#### preSave
```js
// series
myCollection.preSave(function(plainData, rxDocument){
// modify anyField before saving
plainData.anyField = 'anyValue';
}, false);
// parallel
myCollection.preSave(function(plainData, rxDocument){
}, true);
// async
myCollection.preSave(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
// stop the save-operation
myCollection.preSave(function(plainData, rxDocument){
throw new Error('stop');
}, false);
```
#### postSave
```js
// series
myCollection.postSave(function(plainData, rxDocument){
}, false);
// parallel
myCollection.postSave(function(plainData, rxDocument){
}, true);
// async
myCollection.postSave(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
```
### Remove
A remove-hook receives the document which is removed.
#### lifecycle
- RxDocument.remove is called
- preRemove series-hooks
- preRemove parallel-hooks
- deleted document is written to database
- postRemove series-hooks
- postRemove parallel-hooks
- event is emitted to RxDatabase and RxCollection
#### preRemove
```js
// series
myCollection.preRemove(function(plainData, rxDocument){
}, false);
// parallel
myCollection.preRemove(function(plainData, rxDocument){
}, true);
// async
myCollection.preRemove(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
// stop the remove-operation
myCollection.preRemove(function(plainData, rxDocument){
throw new Error('stop');
}, false);
```
#### postRemove
```js
// series
myCollection.postRemove(function(plainData, rxDocument){
}, false);
// parallel
myCollection.postRemove(function(plainData, rxDocument){
}, true);
// async
myCollection.postRemove(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
```
### postCreate
This hook is called whenever a `RxDocument` is constructed.
You can use `postCreate` to modify every RxDocument-instance of the collection.
This adds a flexible way to add specific behavior to every document. You can also use it to add custom getter/setter to documents. PostCreate-hooks cannot be **asynchronous**.
```js
myCollection.postCreate(function(plainData, rxDocument){
Object.defineProperty(rxDocument, 'myField', {
get: () => 'foobar',
});
});
const doc = await myCollection.findOne().exec();
console.log(doc.myField);
// 'foobar'
```
:::note
This hook does not run on already created or cached documents. Make sure to add `postCreate`-hooks before interacting with the collection.
:::
---
## CRDT - Conflict-free replicated data type Database
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB CRDT Plugin
Whenever there are multiple instances in a distributed system, data writes can cause conflicts. Two different clients could do a write to the same document at the same time or while they are both offline. When the clients replicate the document state with the server, a conflict emerges that must be resolved by the system.
In [RxDB](./), conflicts are normally resolved by setting a `conflictHandler` when creating a collection. The conflict handler is a JavaScript function that gets the two conflicting states of the same document and it will return the resolved document state.
The [default conflict handler](./replication.md#conflict-handling) will always drop the fork state and use the master state to ensure that clients that have been offline for a long time, do not overwrite other clients changes when they go online again.
With CRDTs (short for [Conflict-free replicated data type](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type)), all document
writes are represented as CRDT operations in plain JSON. The CRDT operations are stored together with the document and each time a conflict arises, the CRDT conflict handler will automatically merge the operations in a deterministic way. Using CRDTs is an easy way to "magically" handle all conflict problems in your application by storing the deltas of writes together with the document data.
## RxDB CRDT operations
In RxDB, a CRDT operation is defined with [NoSQL](./articles/in-memory-nosql-database.md) update operators, like you might know them from [MongoDB update operations](https://www.mongodb.com/docs/manual/reference/operator/update/) or the [RxDB update plugin](./rx-document.md#update).
To run the operators, RxDB uses the [mingo library](https://github.com/kofrasa/mingo#updating-documents).
A CRDT operator example:
```js
const myCRDTOperation = {
// increment the points field by +1
$inc: {
points: 1
},
// set the modified field to true
$set: {
modified: true
}
};
```
### Operators
At the moment, not all possible operators are implemented in [mingo](https://github.com/kofrasa/mingo#updating-documents), if you need additional ones, you should make a pull request there.
The following operators can be used at this point in time:
- `$min`
- `$max`
- `$inc`
- `$set`
- `$unset`
- `$push`
- `$addToSet`
- `$pop`
- `$pullAll`
- `$rename`
For the exact definition on how each operator behaves, check out the [MongoDB documentation on update operators](https://www.mongodb.com/docs/manual/reference/operator/update/).
## Installation
To use CRDTs with RxDB, you need the following:
- Add the CRDT plugin via `addRxPlugin`.
- Add a field to your schema that defines where to store the CRDT operations via `getCRDTSchemaPart()`
- Set the `crdt` options in your schema.
- Do **NOT** set a custom conflict handler, the plugin will use its own one.
```ts
// import the relevant parts from the CRDT plugin
import {
getCRDTSchemaPart,
RxDBcrdtPlugin
} from 'rxdb/plugins/crdt';
// add the CRDT plugin to RxDB
import { addRxPlugin } from 'rxdb';
addRxPlugin(RxDBcrdtPlugin);
// create a database
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const myDatabase = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage()
});
// create a schema with the CRDT options
const mySchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
points: {
type: 'number',
maximum: 100,
minimum: 0
},
crdts: getCRDTSchemaPart() // use this field to store the CRDT operations
},
required: ['id', 'points'],
crdt: { // CRDT options
field: 'crdts'
}
}
// add a collection
await db.addCollections({
users: {
schema: mySchema
}
});
// insert a document
const myDocument = await db.users.insert({id: 'alice', points: 0});
// run a CRDT operation that increments the 'points' by one
await myDocument.updateCRDT({
ifMatch: {
$inc: {
points: 1
}
}
});
```
## Conditional CRDT operations
By default, all CRDTs operations will be run to build the current document state. But in many cases, more granular operations are required to better reflect the desired business logic. For these cases, conditional CRDTs can be used.
For example if you have a field `points` with a `maximum` of `100`, you might want to only run the `$inc` operation, if the `points` value is less than `100`.
In an conditional CRDT, you can specify a `selector` and the operation sets `ifMatch` and `ifNotMatch`. At each time the CRDT is applied to the document state, first the selector will run and evaluate which operations path must be used.
```ts
await myDocument.updateCRDT({
// only if the selector matches, the ifMatch operation will run
selector: {
age: {
$lt: 100
}
},
// an operation that runs if the selector matches
ifMatch: {
$inc: {
points: 1
}
},
// if the selector does NOT match, you could run a different operation instead
ifNotMatch: {
// ...
}
});
```
## Running multiples operations at once
By default, one CRDT operation is applied to the document in a single database write.
To represent more complex logic chains, it might make sense to use multiple CRDTs and write them at once inside of a single atomic document write.
For these cases, the `updateCRDT()` method allows to pass an array of operations.
```ts
await myDocument.updateCRDT([
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
},
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
},
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
},
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
}
]);
```
## CRDTs on inserts
When CRDTs are enabled with the plugin, all insert operations are automatically mapped as CRDT operation with the `$set` operator.
```ts
// Calling RxCollection.insert()
await myRxCollection.insert({
id: 'foo',
points: 1
});
// is exactly equal to calling insertCRDT()
await myRxCollection.insertCRDT({
ifMatch: {
$set: {
id: 'foo',
points: 1
}
}
});
```
When the same document is inserted in multiple client instances and then replicated, a conflict will emerge and the insert-CRDTs will overwrite each other in a deterministic order.
You can use `insertCRDT()` to make conditional insert operations with any logic. To check for the previous existence of a document, use the `$exists` query operation on the primary key of the document.
```ts
await myRxCollection.insertCRDT({
selector: {
// only run if the document did not exist before.
id: { $exists: false }
},
ifMatch: {
// if the document did not exist, insert it
$set: {
id: 'foo',
points: 1
}
},
ifNotMatch: {
// if document existed already, increment the points by +1
$inc: {
points: 1
}
}
});
```
## Deleting documents
You can delete a document with a CRDT operation by setting `_deleted` to true. Calling `RxDocument.remove()` will do exactly the same when CRDTs are activated.
```ts
await doc.updateCRDT({
ifMatch: {
$set: {
_deleted: true
}
}
});
// OR
await doc.remove();
```
## CRDTs with replication
CRDT operations are stored inside of a special field besides your 'normal' document fields.
When replicating document data with the [RxDB replication](./replication.md) or the [CouchDB replication](./replication-couchdb.md) or even any custom replication, the CRDT operations must be replicated together with the document data as if they would be 'normal' a document property.
When any instances makes a write to the document, it is required to update the CRDT operations accordingly. For example if your custom backend updates a document, it must also do that by adding a CRDT operation. In [dev-mode](./dev-mode.md) RxDB will refuse to store any document data where the document properties do not match the result of the CRDT operations.
## Why not automerge.js or yjs?
There are already CRDT libraries out there that have been considered to be used with RxDB. The biggest ones are [automerge](https://github.com/automerge/automerge) and [yjs](https://github.com/yjs/yjs). The decision was made to not use these but instead go for a more NoSQL way of designing the CRDT format because:
- Users do not have to learn a new syntax but instead can use the NoSQL query operations which they already know to manipulate the JSON data of a document.
- RxDB is often used to [replicate](./replication.md) data with any custom backend on an already existing infrastructure. Using NoSQL operators instead of binary data in CRDTs, makes it easy to implement the exact same logic on these backends so that the backend can also do document writes and still be compliant to the RxDB CRDT plugin.
So instead of using YJS or Automerge with a database, you can use RxDB with the CRDT plugin to have a more database specific CRDT approach. This gives you additional features for free such as [schema validation](./schema-validation.md) or [data migration](./migration-schema.md).
## When to not use CRDTs
CRDT can only be use when your business logic allows to represent document changes via static json operators.
If you can have cases where user interaction is required to correctly merge conflicting document states, you cannot use CRDTs for that.
Also when CRDTs are used, it is no longer allowed to do non-CRDT writes to the document properties.
## CRDT Alternative
While the CRDT plugin can automatically merge concurrent document updates, it is not the only way to resolve conflicts in RxDB.
An alternative approach to CRDT is to use RxDB's built-in [conflict handling system](./transactions-conflicts-revisions.md).
> Why use conflict handlers instead of CRDT?
Conflict handlers offer a **simpler and more flexible** way to manage data conflicts. Instead of encoding changes as CRDT operations, you define how RxDB should decide which document version "wins" with plain JavaScript code. This approach is easier to reason about because it works directly with your domain logic. For example, you can compare timestamps, prioritize certain fields, or even involve user interaction to resolve conflicts.
Conflict handlers are:
* **Easier to understand**: you work with plain document states instead of CRDT operations.
* **Fully customizable**: you can define any merge strategy, from simple last-write-wins to complex rule-based logic.
* **Compatible with all data types**: unlike CRDTs, which are best suited for numeric or set-based updates.
* **Transparent**: you always know which state is being written and why.
### Downsides of CRDTs
CRDTs are powerful for automatic conflict-free merging, but they also come with trade-offs:
* **Higher conceptual complexity**: CRDTs require understanding of operation semantics, version vectors, and merge determinism.
* **Limited flexibility**: you can only express changes that fit the supported JSON-style update operators.
* **Difficult debugging**: when merges don't behave as expected, it can be hard to trace the sequence of CRDT operations that led to a state.
* **Overhead for simple cases**: if your data rarely conflicts or needs human oversight, using CRDTs can add unnecessary complexity.
### When to choose conflict handlers
Use conflict handlers as CRDT alternative if:
* You want full control over merge logic.
* Your data model includes contextual or user-specific decisions.
* You prefer a straightforward, rule-based resolution system over automatic merges.
Use CRDTs if:
* Your app performs frequent offline writes that can be merged deterministically.
* Your data can be represented as additive, numeric, or array-based updates.
* You want minimal manual intervention during replication.
Both methods are first-class citizens in RxDB. CRDTs focus on **automatic, deterministic merging**, while conflict handlers emphasize **clarity, flexibility, and control**.
### Example: merging different fields with conflict handlers instead of CRDT
For example, imagine two users edit different fields of the same document at the same time. One updates a `name`, the other updates a `score`. A custom conflict handler can merge both changes so no data is lost:
```ts
const mergeFieldsHandler = {
isEqual: (a, b) => JSON.stringify(a) === JSON.stringify(b),
resolve: (input) => {
return {
...input.realMasterState,
name: input.newDocumentState.name ?? input.realMasterState.name,
score: Math.max(input.newDocumentState.score, input.realMasterState.score)
};
}
};
```
In this example, if the two versions change different properties, the final merged document includes both updates. This kind of logic is often easier to reason about than designing equivalent CRDT operations.
## FAQ
RxDB provides a distributed database with conflict-free replication. You build [offline-first](./offline-first.md) applications using local data storage. RxDB synchronizes data across multiple client devices. The CRDT plugin resolves data conflicts automatically during replication. You maintain continuous data consistency without manual merge logic.
A Conflict-free Replicated Data Type (CRDT) works by transforming all data writes into atomic mathematical operations (like `$inc` or `$set`) rather than absolute state replacements. When two offline clients modify the exact same document simultaneously, a CRDT merges these operations in a guaranteed deterministic order upon reconnection. Because the merge logic relies on commutative mathematics, conflicts are resolved automatically without requiring manual developer intervention or user prompts.
The best CRDT databases for scaling fully distributed, masterless applications are those prioritizing dynamic topology mappings without rigid server connections. **[RxDB](https://rxdb.info)** coupled with its dedicated CRDT Plugin operates flawlessly without a central authority. It allows seamless peer-to-peer data replication through WebRTC or WebSocket adapters, automatically running NoSQL-based CRDT operations locally to achieve absolute eventual consistency across all distributed nodes.
Finding a unified, production-ready CRDT database ecosystem is challenging, as many solutions are simply raw algorithmic libraries (like Yjs or Automerge). **[RxDB](https://rxdb.info)** offers a comprehensive, reliable CRDT environment built natively on top of standard NoSQL query selectors. Rather than forcing developers to learn complex binary array manipulation, RxDB's CRDT plugin resolves conflict-free data sync using familiar MongoDB-style JSON update commands like `$inc` and `$push`.
{/*
## TODOs
- Clean up old CRDT operations by crunching them together
- CRDT streaming replication
*/}
---
## Populate and Link Docs in RxDB
# Population
There are no joins in RxDB but sometimes we still want references to documents in other collections. This is where population comes in. You can specify a relation from one [RxDocument](./rx-document.md) to another [RxDocument](./rx-document.md) in the same or another [RxCollection](./rx-collection.md) of the same database.
Then you can get the referenced document with the population getter.
This works exactly like population with [mongoose](http://mongoosejs.com/docs/populate.html).
## Schema with ref
The `ref` keyword in properties describes to which collection the field value belongs to (has a relationship).
```javascript
export const refHuman = {
title: 'human related to other human',
version: 0,
primaryKey: 'name',
properties: {
name: {
type: 'string',
maxLength: 100
},
bestFriend: {
ref: 'human', // refers to collection human
// ref-values must always be string
// or ['string', 'null']
// (primary of foreign RxDocument)
type: 'string'
}
}
};
```
You can also have a one-to-many reference by using a string array.
```js
export const schemaWithOneToManyReference = {
version: 0,
primaryKey: 'name',
type: 'object',
properties: {
name: {
type: 'string',
maxLength: 100
},
friends: {
type: 'array',
ref: 'human',
items: {
type: 'string'
}
}
}
};
```
## populate()
### via method
To get the referred RxDocument, you can use the `populate()` method.
It takes the field path as attribute and returns a Promise which resolves to the foreign document or null if not found.
```javascript
await humansCollection.insert({
name: 'Alice',
bestFriend: 'Carol'
});
await humansCollection.insert({
name: 'Bob',
bestFriend: 'Alice'
});
const doc = await humansCollection.findOne('Bob').exec();
const bestFriend = await doc.populate('bestFriend');
console.dir(bestFriend); //> RxDocument[Alice]
```
### via getter
You can also get the populated RxDocument with the direct getter. To do this, you have to add an underscore suffix `_` to the field name.
This also works on nested values.
```javascript
await humansCollection.insert({
name: 'Alice',
bestFriend: 'Carol'
});
await humansCollection.insert({
name: 'Bob',
bestFriend: 'Alice'
});
const doc = await humansCollection.findOne('Bob').exec();
const bestFriend = await doc.bestFriend_; // notice the underscore `_`
console.dir(bestFriend); //> RxDocument[Alice]
```
## Example with nested reference
```javascript
const myCollection = await myDatabase.addCollections({
human: {
schema: {
version: 0,
type: 'object',
properties: {
name: {
type: 'string'
},
family: {
type: 'object',
properties: {
mother: {
type: 'string',
ref: 'human'
}
}
}
}
}
}
});
/**
* We assume myDocument is a document from the collection
*/
const mother = await myDocument.family.mother_;
console.dir(mother); //> RxDocument
```
## Example with array
```javascript
const myCollection = await myDatabase.addCollections({
human: {
schema: {
version: 0,
type: 'object',
properties: {
name: {
type: 'string'
},
friends: {
type: 'array',
ref: 'human',
items: {
type: 'string'
}
}
}
}
}
});
//[insert other humans here]
await myCollection.insert({
name: 'Alice',
friends: [
'Bob',
'Carol',
'Dave'
]
});
const doc = await humansCollection.findOne('Alice').exec();
const friends = await doc.friends_;
console.dir(friends); //> Array.
```
---
## ORM
# Object-Data-Relational-Mapping
Like [mongoose](http://mongoosejs.com/docs/guide.html#methods), RxDB has ORM capabilities which can be used to add specific behavior to documents and collections.
## statics
Statics are defined collection-wide and can be called on the collection.
### Add statics to a collection
To add static functions, pass a `statics` object when you create your collection. The object contains functions, mapped to their function names.
```javascript
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
statics: {
scream: function(){
return 'AAAH!!';
}
}
}
});
console.log(heroes.scream());
// 'AAAH!!'
```
You can also use the `this` keyword which resolves to the collection:
```javascript
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
statics: {
whoAmI: function(){
return this.name;
}
}
}
});
console.log(heroes.whoAmI());
// 'heroes'
```
## Instance Methods
Instance methods are defined collection-wide. They can be called on the [RxDocuments](./rx-document.md) of the collection.
### Add instance methods to a collection
```javascript
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
methods: {
scream: function(){
return 'AAAH!!';
}
}
}
});
const doc = await heroes.findOne().exec();
console.log(doc.scream());
// 'AAAH!!'
```
Here you can also use the `this` keyword:
```javascript
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
methods: {
whoAmI: function(){
return 'I am ' + this.name + '!!';
}
}
}
});
await heroes.insert({
name: 'Skeletor'
});
const doc = await heroes.findOne().exec();
console.log(doc.whoAmI());
// 'I am Skeletor!!'
```
## attachment-methods
Attachment methods are defined collection-wide. They can be called on the [RxAttachments](./rx-attachment.md) of the RxDocuments of the collection.
```javascript
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
attachments: {
scream: function(){
return 'AAAH!!';
}
}
}
});
const doc = await heroes.findOne().exec();
const attachment = await doc.putAttachment({
id: 'cat.txt',
data: 'meow I am a kitty',
type: 'text/plain'
});
console.log(attachment.scream());
// 'AAAH!!'
```
---
## Fulltext Search
import {PremiumBlock} from '@site/src/components/premium-block';
import {Steps} from '@site/src/components/steps';
# Fulltext Search
To run fulltext search queries on the local data, RxDB has a fulltext search plugin based on [flexsearch](https://github.com/nextapps-de/flexsearch) and [RxPipeline](./rx-pipeline.md). On each write to a given source [RxCollection](./rx-collection.md), an indexer is running to map the written document data into a fulltext search index.
The index can then be queried efficiently with complex fulltext search operations.
## Benefits of using a local fulltext search
1. Efficient Search and Indexing
The plugin utilizes the [FlexSearch library](https://github.com/nextapps-de/flexsearch), known for its speed and memory efficiency. This ensures that search operations are performed quickly, even with large datasets. The search engine can handle multi-field queries, partial matching, and complex search operations, providing users with highly relevant results.
2. Local Data Indexing
With the plugin, all search operations are performed on the local data stored within the RxDB collections. This means that users can execute fulltext search queries without the need for an external server or database, which is especially beneficial for offline-first applications. The local indexing ensures that search queries are executed quickly, reducing the latency typically associated with remote database queries. Also when used in multiple browser tabs, it is ensured that through [Leader Election](./leader-election.md), only exactly one tabs is doing the work of indexing without having an overhead in the other browser tabs.
3. Real-time Indexing
The plugin integrates seamlessly with RxDB's reactive nature. Every time a document is written to an [RxCollection](./rx-collection.md), an indexer updates the fulltext search index in real-time. This ensures that search results are always up-to-date, reflecting the most current state of the data without requiring manual reindexing.
4. Persistent indexing
The fulltext search index is efficiently persisted within the [RxCollection](./rx-collection.md), ensuring that the index remains intact across app restarts. When documents are added or updated in the collection, the index is incrementally updated in real-time, meaning only the changes are processed rather than reindexing the entire dataset. This incremental approach not only optimizes performance but also ensures that subsequent app launches are quick, as there's no need to reindex all the data from scratch, making the search feature both reliable and fast from the moment the app starts. When using an [encrypted storage](./encryption.md) the index itself and incremental updates to it are stored fully encrypted and are only decrypted in-memory.
5. Complex Query Support
The FlexSearch-based plugin allows for [sophisticated search queries](https://github.com/nextapps-de/flexsearch?tab=readme-ov-file#index.search), including multi-term and contextual searches. Users can perform complex searches that go beyond simple keyword matching, enabling more advanced use cases like searching for documents with specific phrases, relevance-based sorting, or even phonetic matching.
6. Offline-First Support and Privacy
As RxDB is designed with [offline-first applications](./offline-first.md) in mind, the fulltext search plugin supports this paradigm by ensuring that all search operations can be performed offline. This is crucial for applications that need to function in environments with intermittent or no internet connectivity, offering users a consistent and reliable search experience with [zero latency](./articles/zero-latency-local-first.md).
## Using the RxDB Fulltext Search
### Step 1: Add the `RxDBFlexSearchPlugin` to RxDB.
```ts
import { RxDBFlexSearchPlugin } from 'rxdb-premium/plugins/flexsearch';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBFlexSearchPlugin);
```
### Step 2: Create a `RxFulltextSearch` instance on top of a collection with the `addFulltextSearch()` function.
```ts
import { addFulltextSearch } from 'rxdb-premium/plugins/flexsearch';
const flexSearch = await addFulltextSearch({
// unique identifier. Used to store metadata
// and continue indexing on restarts/reloads.
identifier: 'my-search',
// The source collection on whose documents the search is based on
collection: myRxCollection,
/**
* Transforms the document data to a given searchable string.
* This can be done by returning a single string property of the document
* or even by concatenating and transforming multiple fields like:
* doc => doc.firstName + ' ' + doc.lastName
*/
docToString: doc => doc.firstName,
/**
* (Optional)
* Amount of documents to index at once.
* See https://rxdb.info/rx-pipeline.html
*/
batchSize: number;
/**
* (Optional)
* lazy: Initialize the in memory fulltext index at the first search query.
* instant: Directly initialize so that the
* index is already there on the first query.
* Default: 'instant'
*/
initialization: 'instant',
/**
* (Optional)
* @link https://github.com/nextapps-de/flexsearch#index-options
*/
indexOptions: {},
});
```
### Step 3: Run a search operation:
```ts
// find all documents whose searchstring contains "foobar"
const foundDocuments = await flexSearch.find('foobar');
/**
* You can also use search options as second parameter
* @link https://github.com/nextapps-de/flexsearch#search-options
*/
const foundDocuments = await flexSearch.find('foobar', { limit: 10 });
```
---
## Optimize Client-Side Queries with RxDB
import {PremiumBlock} from '@site/src/components/premium-block';
# Query Optimizer
The query optimizer can be used to determine which index is the best to use for a given query.
Because RxDB is used in client side applications, it cannot do any background checks or measurements to optimize the query plan because that would cause significant performance problems.
## Usage
```ts
import {
findBestIndex
} from 'rxdb-premium/plugins/query-optimizer';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
const bestIndexes = await findBestIndex({
schema: myRxJsonSchema, // see Schema Validation
/**
* In this example we use the IndexedDB RxStorage,
* but any other storage can be used for testing.
*/
storage: getRxStorageIndexedDB(),
/**
* Multiple queries can be optimized at the same time
* which decreases the overall runtime.
*/
queries: {
/**
* Queries can be mapped by a query id,
* here we use myFirstQuery as query id.
*/
myFirstQuery: {
selector: {
age: {
$gt: 10
}
},
},
mySecondQuery: {
selector: {
age: {
$gt: 10
},
lastName: {
$eq: 'Nakamoto'
}
},
}
},
testData: [/** data for the documents. **/]
});
```
## Important details
- This is a build time tool. You should use it to find the best indexes for your queries during **build time**. Then you store these results and you application can use the best indexes during **run time**.
- It makes no sense to run time optimization with a different [RxStorage](./rx-storage.md) (+settings) that what you use in production. The result of the query optimizer is heavily dependent on the RxStorage and JavaScript runtime. For example it makes no sense to run the optimization in Node.js and then use the optimized indexes in the browser.
- It is very important that you use **production like** `testData`. Finding the best index heavily depends on data distribution and amount of stored/queried documents. For example if you store and query users with an `age` field, it makes no sense to just use a random number for the age because in production the `age` of your users is not equally distributed.
- The higher you set `runs`, the more test cycles will be performed and the more **significant** will be the time measurements which leads to a better index selection.
---
## How Local-First and WebMCP make your app accessible to agents
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {Tabs} from '@site/src/components/tabs';
import {QuoteBlock} from '@site/src/components/quoteblock';
import {BetaBlock} from '@site/src/components/beta-block';
import {Faq, FaqItem} from '@site/src/components/faq';
# How Local-First and WebMCP make your app accessible to agents
Over the past few years, the **[Local-First](./articles/local-first-future.md) architecture** has emerged as a new standard for building fast, offline-capable applications. Now, the long-awaited introduction of **WebMCP** makes local-first even more useful. By keeping data local, AI Agents can access, query, and mutate application states instantaneously on the client side, bypassing the latency and security vulnerabilities of traditional cloud APIs.
> WebMCP provides a formalized machine interface alongside the human interface.
## What is WebMCP?
[WebMCP](https://webmachinelearning.github.io/webmcp/) (Web Model Context Protocol) is an experimental browser API that allows your web application to seamlessly expose "tools" for AI Agents. WebMCP is an adaptation of the Model Context Protocol (MCP) standardized for use within web browsers, currently incubated through the W3C Web Machine Learning community group.
When an AI Agent is active, it can discover these tools and call them programmatically with arguments as defined by a strict JSON Schema, via the `navigator.modelContext` API.
```ts
// Example: Registering a simple WebMCP tool natively
navigator.modelContext.registerTool({
name: 'get_weather',
description: 'Returns the current weather for a city',
inputSchema: {
type: 'object',
properties: {
city: { type: 'string' }
},
required: ['city']
}
}, async (params) => {
return fetch(`/api/weather?city=${params.city}`);
});
```
## The End of Scraping
WebMCP transforms the browser from a visual document viewer into a semantic capability surface.
For years, automation and AI have relied on simulating human inputs: guessing CSS class names `.button-primary`, parsing accessibility trees, and breaking whenever a layout changes. This "pixels-as-APIs" approach is slow, brittle, and highly token-dependent.
WebMCP provides a formalized machine interface alongside the human interface. By exposing deterministic, schema-validated tools, WebMCP closes the execution gap. AI Agents no longer guess how to interact with an application; they are given a precise contract defining exactly what operations are available and what payloads they accept.
I personally think these AI agents are inevitable. Like we adapted to Mobile from Desktop, its time to build websites and services for AI agents.
## Benefits of WebMCP
WebMCP introduces massive structural advantages over traditional browser automation:
- **Token Efficiency**: Providing structured JSON schemas to LLMs requires far fewer tokens than dumping raw DOM layout elements or accessibility trees into the prompt context.
- **Bypass Bot Protection**: Rather than forcing AI to use brittle DOM-scraping (like Selenium) that triggers CAPTCHAs or Cloudflare blocks, WebMCP gives them a sanctioned, highly-structured "front door" API.
- **Better Understanding**: The agent does not have to arbitrarily parse pixels or DOM layouts. Instead, it works directly on the deterministic data structures it receives.
- **Less Hallucination**: Because the agent receives exact data with high precision rather than inferring state from a UI, it is significantly less prone to hallucinating facts.
- **Access Control**: Developers have granular control over exactly what an agent can and cannot do by explicitly defining and exposing only specific WebMCP tools.
## Why Local-First and RxDB work great with WebMCP
WebMCP is uniquely powerful when paired with [local-first](/articles/local-first-future.md) databases like RxDB:
- **Unlimited Options**: Traditional websites must code a specific WebMCP tool for every possible action (e.g. `getProductsByPrice`, `searchProductsByColor`, `additemToCart`). By exposing a local-first database, the AI Agent has unlimited generic query and mutation options mathematically bound only by your schema.
- **Zero Latency**: Agents query data instantly from the local database on the user's device.
- **Offline Capable**: Because the data and the API are local, AI Agents can assist users completely offline.
- **Privacy First**: Sensitive user data can stay on the device while still being queryable by the on-device AI model.
- **Direct Access**: Agents can bypass the UI entirely and find exactly what they need with basic NoSQL queries.
- **LLM-Friendly NoSQL**: Writing NoSQL query objects (like [Mongo-style queries](./rx-query.md)) is significantly easier and more deterministic for LLMs to generate and validate than orchestrating complex, string-based SQL JOIN queries.
- **Native JSONSchema**: WebMCP relies entirely on JSONSchema to define tools and parameters. Because [RxDB schemas are *already* written in JSONSchema](./rx-schema.md), there is zero translation overhead, meaning the agent receives the exact structural contract it expects.
### Example Use Cases
Exposing your local database to AI agents unlocks new user experiences beyond what is possible with traditional websites.
**Online Shop**
An AI Agent searches your local catalog for items based on complex criteria, such as "Find all in-stock items cheaper than $50 that have a blue color". The agent can issue a highly efficient NoSQL query via WebMCP to retrieve all in-stock items under $50, and then seamlessly read through the returned JSON block to filter out the blue ones using its own internal LLM reasoning. Without a local-first database, developers would have to manually implement and maintain specific server-side WebMCP endpoints (e.g., `getItemsUnder50`) for every possible filtering combination the user might ask for, or expose a dynamic endpoint which queries the backend database but has unpredictable performance risks and massive security problems.
**Grocery Shopping List**
An agent can manage your list by using WebMCP modification tools in real time. If a user says "Move all drinks from Shop A to Shop B", the AI first uses `rxdb_query` to find all items categorized as drinks assigned to Shop A. It then uses the `rxdb_upsert` tool to update those specific documents to belong to Shop B. Alternatively, if a user says "Add everything I need to bake a cheesecake", the AI determines the ingredients and uses the `rxdb_insert` tool multiple times to instantly populate the local UI with the new shopping items, and `rxdb_delete` to remove items when the user says "I already have butter".
**Geotracking App**
Using the [RxDB continuous queries and observables](./reactivity.md) (`rxdb_wait_changes`), an agent monitors live tracking data to notify the user when a specific object starts moving.
## The RxDB WebMCP Plugin
RxDB provides a plugin `rxdb/plugins/webmcp` that lets you expose your collections to WebMCP with just a single function call.
The plugin dynamically reads your RxDB schema to assemble a prompt description and the tool's `inputSchema` so the AI knows exactly what data shapes are available and how to query them. It automatically registers the following WebMCP tools for each tracked collection:
**Read Operations**:
- `rxdb_query`: Run complex NoSQL queries against the local database.
- `rxdb_count`: Count the number of documents matching a specific query.
- `rxdb_changes`: Fetch the replication changestream since a given checkpoint.
- `rxdb_wait_changes`: Listen to live UI updates by pausing until a matching document changes occurs.
**Write Operations**:
- `rxdb_insert`: Insert new documents into the local collection.
- `rxdb_upsert`: Overwrite existing documents or insert them if they don't exist.
- `rxdb_delete`: Remove items from the local database by ID.
:::note
State-modifying tools like insert/upsert/delete can be disabled via the [`readOnly`](#readonly-default-false) option.
:::
### Quick Start
#### Add the plugin
First, ensure the plugin is added to your RxDB configuration:
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBWebMCPPlugin } from 'rxdb/plugins/webmcp';
addRxPlugin(RxDBWebMCPPlugin);
```
#### Create a database
First, initialize your [RxDatabase](./rx-database.md) instance:
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageLocalstorage()
});
```
#### Add a Collection
Next, [add a collection](./rx-collection.md) with a simple schema. Providing an accurate schema is critical because the AI agent will use this exact schema to understand your data shape:
```ts
await db.addCollections({
todos: {
schema: {
title: 'Todo App Schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string', description: 'The task title' },
done: { type: 'boolean' }
},
required: ['id', 'name', 'done']
}
}
});
```
#### Register Collections
Finally, activate WebMCP on the whole database or specific collections:
```ts
// Expose all collections in the DB to WebMCP (Read-only by default)
db.registerWebMCP();
// Or expose only a specific collection:
db.collections.todos.registerWebMCP();
```
### Options
The `registerWebMCP` method accepts an optional object:
#### `readOnly` (default: `false`)
By default, WebMCP allows modifier tools.
If you explicitly want the agent to only be able to query the database, enable `readOnly`.
```ts
db.registerWebMCP({
readOnly: true
});
```
This skips registering `rxdb_insert`, `rxdb_upsert`, and `rxdb_delete` tools.
#### `awaitReplicationsInSync` (default: `true`)
Because [replications](./replication.md) pull remote data into the local RxDB asynchronously, an AI Agent's query might miss data if a replication is still catching up. By default, WebMCP query invocations await (`awaitInSync()`) all running replications for that collection before returning the query results.
Set this to `false` if you want to allow queries without waiting for replication to be in sync.
```ts
db.registerWebMCP({
awaitReplicationsInSync: false
});
```
:::warning
If the application is offline and the replication is configured to retry infinitely, querying WebMCP with this option enabled may hang indefinitely while awaiting replication sync. Use wisely.
:::
#### `modelContext` (default: `document.modelContext`)
The tools are registered at the WebMCP registry of the current document. When you need them registered at the registry of another document, for example an iframe, pass that registry directly.
```ts
db.registerWebMCP({
modelContext: myIframe.contentDocument.modelContext
});
```
When this option is not set, `document.modelContext` is used with a fallback to `navigator.modelContext` for browsers that still expose the older entrypoint.
### Logs and Errors
Both `registerWebMCP` methods (`db.registerWebMCP()` and `db.collections.humans.registerWebMCP()`) return an object containing two RxJS Subjects: `log$` and `error$`.
You can subscribe to these to monitor the AI agent's actions:
```ts
const { log$, error$ } = db.registerWebMCP();
log$.subscribe(info => {
// Log all tool calls, arguments, and responses
console.log('WebMCP Agent Action', info);
});
error$.subscribe(err => {
// Audit failed tool executions
console.error('WebMCP Agent Error', err);
});
```
### Pro Tip: Schema Descriptions for Better LLM Results
Because WebMCP sends your collection's [JSON schema](./rx-schema.md) directly to the AI Agent, the LLM uses the schema to understand the data model.
Providing detailed, highly accurate descriptions for your properties significantly improves the LLM's ability to construct valid and precise queries.
#### Bad Example
```ts
const productSchema = {
version: 0,
title: 'Product',
primaryKey: 'sku',
type: 'object',
properties: {
sku: {
type: 'string',
maxLength: 100
},
price: {
type: 'number'
}
},
required: ['sku', 'price']
};
```
#### Good Example
```ts
const productSchema = {
version: 0,
title: 'Store Product Inventory Item',
description: 'A physical item sold in our store.' +
' Contains pricing and SKU lookup data.',
primaryKey: 'sku',
type: 'object',
properties: {
sku: {
type: 'string',
maxLength: 100,
description: 'The Stock Keeping Unit.' +
' Category prefix and 6-digit number.'
},
price: {
type: 'number',
minimum: 0,
multipleOf: 0.01,
description: 'The price of the product in Euro (β¬).'
}
},
required: ['sku', 'price']
};
```
### Security and Prompt Injection
Architecturally, WebMCP turns the browser into a "capability surface" with explicit contracts. Security boundaries are clearer because only declared tools are visible and inputs are strictly validated against schemas.
However, be aware that WebMCP **does not completely eliminate prompt injection risks**. It significantly narrows the surface compared to DOM-level automation, but an Agent mimicking a well-behaved query against your schema can still produce corrupted behavior if the prompt itself contains malicious instructions. Ensure your application logic (and RxDB [schema validation](./schema-validation.md)) assumes agent-provided payloads are untrusted.
APIs and behaviors are subject to change as the official W3C WebMCP specification and browser implementations evolve.
## FAQ
WebMCP (Web Model Context Protocol) is an experimental browser API that allows your web application to seamlessly expose "tools" for AI Agents. It acts as a standardized translation layer between your application's functionality and LLMs running within the browser, enabling natural language interactions with your web application's data. Wait for formal browser support for use in production environments.
You can easily mix the RxDB WebMCP tools with your own tools. Since db.registerWebMCP() internally just calls navigator.modelContext.registerTool(), you can simply call this native method yourself to register any additional tools that interact with your frontend logic, external APIs, or other non-database components.
Since most browsers do not yet natively implement the navigator.modelContext API, you can use the WebMCP-org polyfill package @mcp-b/global to add support in any browser today.
Install the package:
```bash
npm install @mcp-b/global
```
Then import it once at the entry point of your application, before any WebMCP tools are registered:
```ts
import '@mcp-b/global';
// navigator.modelContext is now available
import { addRxPlugin } from 'rxdb';
import { RxDBWebMCPPlugin } from 'rxdb/plugins/webmcp';
addRxPlugin(RxDBWebMCPPlugin);
```
The polyfill sets up the navigator.modelContext interface so that your registered tools are accessible to AI agents running in the browser, even without native browser support.
WebMCP is currently in an early preview phase. You can test it today in Chrome Canary (version 145+) by following these steps:
1. **Enable the flag**: Go to `chrome://flags`, search for "WebMCP for testing", enable it, and relaunch Chrome.
2. **Install the inspector extension**: Install the [Model Context Tool Inspector Extension](https://chromewebstore.google.com/detail/model-context-tool-inspec/gbpdfapgefenggkahomfgkhfehlcenpd) to view registered tools, execute them manually, and test with an agent using Gemini API integration.
3. **Use a live demo**: You can test the integration directly on demo pages like the [RxDB WebMCP Quickstart](https://pubkey.github.io/rxdb-quickstart/).
## Follow up
To learn more about WebMCP and see it in action, check out these resources:
- [WebMCP Chrome Developer Blog Post](https://developer.chrome.com/blog/webmcp-epp?hl=en)
- [RxDB WebMCP Quickstart Repository](https://github.com/pubkey/rxdb-quickstart)
- [Live WebMCP RxDB Demo](https://pubkey.github.io/rxdb-quickstart/)
- Read: [Why Local-First Software Is the Future and what are its Limitations](/articles/local-first-future.md)
---
## Boost Your RxDB with Powerful Third-Party Plugins
# Third Party Plugins
* [rxdb-hooks](https://github.com/cvara/rxdb-hooks) A set of hooks to integrate RxDB into react applications.
* [rxdb-flexsearch](https://github.com/serenysoft/rxdb-flexsearch) The full text search for RxDB using [FlexSearch](https://github.com/nextapps-de/flexsearch).
* [rxdb-orion](https://github.com/serenysoft/rxdb-orion) Enables [replication](./replication.md) with [Laravel Orion](https://tailflow.github.io/laravel-orion-docs).
* [rxdb-supabase](https://github.com/marceljuenemann/rxdb-supabase) Enables replication with [Supabase](https://supabase.com/).
* [rxdb-solid](https://github.com/jeswr/rxdb-solid) Enables replication with [Solid pods](https://solidproject.org/), using a personal Solid pod as backend storage.
* [rxdb-utils](https://github.com/rafamel/rxdb-utils) Additional features for RxDB like models, timestamps, default values, view and more.
* [rxdb-extra](https://github.com/serenysoft/rxdb-extra) Additional features for RxDB like timestamps, simple search, strict schema, and more. (Compatible with the latest versions.)
* [loki-async-reference-adapter](https://github.com/jonnyreeves/loki-async-reference-adapter) Simple async adapter for LokiJS, suitable to use RxDB's [Lokijs RxStorage](./rx-storage-lokijs.md) with React Native.
---
## π Discover RxDB Storage Benchmarks
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_NODE, PERFORMANCE_METRICS, PERFORMANCE_DATA_BROWSER, PERFORMANCE_DATA_SERVER } from '@site/src/components/performance-data';
import {Faq, FaqItem} from '@site/src/components/faq';
## RxStorage Performance comparison
A big difference in the RxStorage implementations is the **performance**. In difference to a server side database, RxDB is bound to the limits of the JavaScript runtime and depending on the runtime, there are different possibilities to store and fetch data. For example in the browser it is only possible to store data in a [slow IndexedDB](./slow-indexeddb.md) or OPFS instead of a filesystem while on React-Native you can use the [SQLite storage](./rx-storage-sqlite.md).
Therefore the performance can be completely different depending on where you use RxDB and what you do with it. Here you can see some performance measurements and descriptions on how the different [storages](./rx-storage.md) work and how their performance is different.
## Persistent vs Semi-Persistent storages
The "normal" storages are always persistent. This means each RxDB write is directly written to disc and all queries run on the disc state. This means a good startup performance because nothing has to be done on startup.
In contrast, semi-persistent storages like [memory mapped](./rx-storage-memory-mapped.md) store all data in memory on startup and only save to disc occasionally (or on exit). Therefore it has a very fast read/write performance, but loading all data into memory on the first page load can take longer for big amounts of documents. Also these storages can only be used when all data fits into the memory at least once. In general it is recommended to stay on the persistent storages and only use semi-persistent ones, when you know for sure that the dataset will stay small (less than 2k documents).
## Performance comparison
In the following you can find some performance measurements and comparisons. Notice that these are only a small set of possible RxDB operations. If performance is really relevant for your use case, you should do your own measurements with usage-patterns that are equal to how you use RxDB in production.
### Measurements
Here the following metrics are measured:
- time-to-first-insert: Many storages run lazy, so it makes no sense to compare the time which is required to create a database with collections. Instead we measure the **time-to-first-insert** which is the whole timespan from database creation until the first single document write is done.
- insert documents (bulk): Insert 500 documents with a single bulk-insert operation.
- find documents by id (bulk): Here we fetch 100% of the stored documents with a single `findByIds()` call.
- insert documents (serial): Insert 50 documents, one after each other.
- find documents by id (serial): Here we find 50 documents in serial with one `findByIds()` call per document.
- find documents by query: Here we fetch 100% of the stored documents with a single `find()` call.
- find documents by query: Here we fetch all of the stored documents with a 4 `find()` calls that run in parallel. Each fetching 25% of the documents.
- count documents: Counts 100% of the stored documents with a single `count()` call. Here we measure 4 runs at once to have a higher number that is easier to compare.
## Browser based Storages Performance Comparison
The performance patterns of the browser based storages are very diverse. The [IndexedDB storage](./rx-storage-indexeddb.md) is recommended for mostly all use cases so you should start with that one. Later you can do performance testings and switch to another storage like [OPFS](./rx-storage-opfs.md) or [memory-mapped](./rx-storage-memory-mapped.md).
## Node/Native based Storages Performance Comparison
For most client-side native applications ([react-native](./react-native-database.md), [electron](./electron-database.md), [capacitor](./capacitor-database.md)), using the [SQLite RxStorage](./rx-storage-sqlite.md) is recommended as a solid baseline. For React Native and Expo applications specifically, the new [Expo Filesystem RxStorage](./rx-storage-filesystem-expo.md) bypasses the bridge and offers significantly better CPU and I/O performance. For non-client side applications like a server, use the [MongoDB storage](./rx-storage-mongodb.md) instead.
## Server based Storages Performance Comparison
When using RxDB on backend servers, you have different options compared to client-side applications. The [Filesystem Node storage](./rx-storage-filesystem-node.md) is a great choice for standalone Node.js processes utilizing local disk storage. The [MongoDB storage](./rx-storage-mongodb.md) provides solid performance for heavy server workloads. The [FoundationDB storage](./rx-storage-foundationdb.md) is very fast and works well for distributed systems. For purely in-memory operations, the [Memory storage](./rx-storage-memory.md) offers the lowest latency.
## FAQ
IndexedDB sits securely in the middle of browser storage performance. It is significantly slower than the fully synchronous [LocalStorage](./articles/localstorage.md) memory cache, but it completely avoids blocking the main UI thread. However, compared to modern APIs like the **[Origin Private File System (OPFS)](./rx-storage-opfs.md)**, IndexedDB's complex internal B-tree implementations combined with serialization overhead make it significantly slower for high-throughput I/O operations and raw bulk writes.
---
## RxDB NoSQL Performance Tips
# Performance tips for RxDB and other [NoSQL](./articles/in-memory-nosql-database.md) databases
In this guide, you'll find techniques to improve the performance of RxDB operations and queries. Notice that all your performance optimizations should be done with a correct tracking of the metrics, otherwise you might change stuff into the wrong direction.
## Use bulk operations
When you run write operations on multiple documents, make sure you use bulk operations instead of single document operations.
```ts
// wrong β
for(const docData of dataAr){
await myCollection.insert(docData);
}
// right βοΈ
await myCollection.bulkInsert(dataAr);
```
## Help the query planner by adding operators that better restrict the index range
Often on complex queries, RxDB (and other databases) do not pick the optimal index range when querying a result set.
You can add additional restrictive operators to ensure the query runs over a smaller index space and has a better performance.
Lets see some examples for different query types.
```ts
/**
* Adding a restrictive operator for an $or query
* so that it better limits the index space for the time-field.
*/
const orQuery = {
selector: {
$or: [
{
time: { $gt: 1234 },
},
{
time: { $eq: 1234 },
user: { $gt: 'foobar' }
},
],
time: { $gte: 1234 } // <- add restrictive operator
}
}
/**
* Adding a restrictive operator for an $regex query
* so that it better limits the index space for the user-field.
* We know that all matching fields start with 'foo' so we can
* tell the query to use that as lower constraint for the index.
*/
const regexQuery = {
selector: {
user: {
$regex: '^foo(.*)0-9$', // a complex regex with a ^ in the beginning
$gte: 'foo' // <- add restrictive operator
}
}
}
/**
* Adding a restrictive operator for a query on an enum field.
* so that it better limits the index space for the time-field.
*/
const enumQuery = {
selector: {
/**
* Here lets assume our status field has
* the enum type
* ['idle', 'in-progress', 'done']
* so our restrictive operator can exclude
* all documents with 'done' as status.
*/
status: {
$in: [
'idle',
'in-progress',
],
$gt: 'done' // <- add restrictive operator on status
}
}
}
```
For `$in` queries on an indexed field, the RxDB query planner limits the scanned index space to the range between the smallest and the largest of the given values. Adding a restrictive operator next to an `$in` is only useful when you know a tighter bound than the min/max of the values.
## Set a specific index
Sometime the query planner of the database itself has no chance in picking the best index of the possible given indexes.
For queries where performance is very important, you might want to explicitly specify which index must be used.
```ts
const myQuery = myCollection.find({
selector: {
/* ... */
},
// explicitly specify index
index: [
'fieldA',
'fieldB'
]
});
```
## Try different ordering of index fields
The order of the fields in a compound index is very important for performance. When optimizing index usage, you should try out different orders on the index fields and measure which runs faster. For that it is very important to run tests on real-world data where the distribution of the data is the same as in production.
For example when there is a query on a user collection with an `age` and a `gender` field, it depends if the index `['gender', 'age']` performance better as `['age', 'gender']` based on the distribution of data:
```ts
const query = myCollection
.findOne({
selector: {
age: {
$gt: 18
},
gender: {
$eq: 'm'
}
},
/**
* Because the developer knows that 50% of the documents are 'male',
* but only 20% are below age 18,
* it makes sense to enforce using the
* ['gender', 'age'] index to improve
* performance. This could not be known
* by the query planer which might have
* chosen ['age', 'gender'] instead.
*/
index: ['gender', 'age']
});
```
Notice that RxDB has the [Query Optimizer Plugin](./query-optimizer.md) that can be used to automatically find the best indexes.
## Make a Query "hot" to reduce load
Having a query where the up-to-date result set is needed more than once, you might want to make the query "hot" by permanently subscribing to it. This ensures that the query result is kept up to date by RxDB ant the [EventReduce algorithm](https://github.com/pubkey/event-reduce) at any time so that at the moment you need the current results, it has them already.
For example when you use RxDB at [Node.js](./nodejs-database.md) for a webserver, you should use an outer "hot" query instead of running the same query again on every request to a route.
```ts
// wrong β
app.get('/list', (req, res) => {
const result = await myCollection.find({/* ... */}).exec();
res.send(JSON.stringify(result));
});
// right βοΈ
const query = myCollection.find({/* ... */});
query.subscribe(); // <- make it hot
app.get('/list', (req, res) => {
const result = await query.exec();
res.send(JSON.stringify(result));
});
```
## Store parts of your document data as attachment
For in-app databases like RxDB, it does not make sense to partially parse the `JSON` of a document. Instead, always the whole document json is parsed and handled. This has a better performance because `JSON.parse()` in JavaScript directly calls a C++ binding which can parse really fast compared to a partial parsing in JavaScript itself. Also by always having the full document, RxDB can de-duplicate memory caches of document across multiple queries.
The downside is that very very big documents with a complex structure can increase query time significantly. Documents fields with complex that are mostly not in use, can be move into an [attachment](./rx-attachment.md). This would lead RxDB to not fetch the attachment data each time the document is loaded from disc. Instead only when explicitly asked for.
```ts
const myDocument = await myCollection.insert({/* ... */});
const attachment = await myDocument.putAttachment(
{
id: 'otherStuff.json',
data: createBlob(JSON.stringify({/* ... */}), 'application/json'),
type: 'application/json'
}
);
```
## Process queries in a worker process
Moving database storage into a WebWorker can significantly improve performance in web applications that use RxDB or similar NoSQL databases. When database operations are executed in the main JavaScript thread, they can block or slow down the User Interface, especially during heavy or complex data operations. By offloading these operations to a WebWorker, you effectively separate the data processing workload from the UI thread. This means the main thread remains free to handle user interactions and render updates without delay, leading to a smoother and more responsive user experience. Additionally, WebWorkers allow for parallel data processing, which can expedite tasks like querying and indexing. This approach not only enhances UI responsiveness but also optimizes overall application performance by leveraging the multi-threading capabilities of modern browsers.
With RxDB you can use the [Worker](./rx-storage-worker.md) and [SharedWorker](./rx-storage-shared-worker.md) plugin to move the query processing away from the main thread.
## Use less plugins and hooks
Utilizing fewer [hooks](./middleware.md) and plugins in RxDB or similar NoSQL database systems can lead to markedly better performance. Each additional hook or plugin introduces extra layers of processing and potential overhead, which can cumulatively slow down database operations. These extensions often execute additional code or enforce extra checks with each operation, such as insertions, updates, or deletions. While they can provide valuable functionalities or custom behaviors, their overuse can inadvertently increase the complexity and execution time of basic database operations. By minimizing their use and only employing essential hooks and plugins, the system can operate more efficiently. This streamlined approach reduces the computational burden on each transaction, leading to faster response times and a more efficient overall data handling process, especially critical in high-load or real-time applications where performance is paramount.
---
## Solving IndexedDB Slowness for Seamless Apps
import {CenteredImage} from '@site/src/components/centered-image';
# Why IndexedDB is slow and what to use instead
So you have a JavaScript web application that needs to store data at the client side, either to make it [offline usable](./offline-first.md), just for caching purposes or for other reasons.
For [in-browser data storage](./articles/browser-database.md), you have some options:
- **Cookies** are sent with each HTTP request, so you cannot store more than a few strings in them.
- **WebSQL** [is deprecated](https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/) because it never was a real standard and turning it into a standard would have been too difficult.
- [LocalStorage](./articles/localstorage.md) is a synchronous API over asynchronous IO-access. Storing and reading data can fully block the JavaScript process so you cannot use LocalStorage for more than few simple key-value pairs.
- The **FileSystem API** could be used to store plain binary files, but it is [only supported in chrome](https://caniuse.com/filesystem) for now.
- **IndexedDB** is an indexed key-object database. It can store json data and iterate over its indexes. It is [widely supported](https://caniuse.com/indexeddb) and stable.
:::note UPDATE April 2023
Since beginning of 2023, all modern browsers ship the **File System Access API** which allows to persistently store data in the browser with a way better performance. For [RxDB](https://rxdb.info/) you can use the [OPFS RxStorage](./rx-storage-opfs.md) to get about 4x performance improvement compared to IndexedDB.
:::
It becomes clear that the only way to go is IndexedDB. You start developing your app and everything goes fine.
But as soon as your app gets bigger, more complex or just handles more data, you might notice something. **IndexedDB is slow**. Not slow like a database on a cheap server, **even slower**! Inserting a few hundred documents can take up several seconds. Time which can be critical for a fast page load. Even sending data over the internet to the backend can be faster than storing it inside of an IndexedDB database.
> Transactions vs Throughput
So before we start complaining, lets analyze what exactly is slow. When you run tests on Nolans [Browser Database Comparison](http://nolanlawson.github.io/database-comparison/) you can see that inserting 1k documents into IndexedDB takes about 80 milliseconds, 0.08ms per document. This is not really slow. It is quite fast and it is very unlikely that you want to store that many document at the same time at the client side. But the key point here is that all these documents get written in a `single transaction`.
I forked the comparison tool [here](https://pubkey.github.io/client-side-databases/database-comparison/index.html) and changed it to use one transaction per document write. And there we have it. Inserting 1k documents with one transaction per write, takes about 2 seconds. Interestingly if we increase the document size to be 100x bigger, it still takes about the same time to store them. This makes clear that the limiting factor to IndexedDB performance is the transaction handling, not the data throughput.
To fix your IndexedDB performance problems you have to make sure to use as less data transfers/transactions as possible.
Sometimes this is easy, as instead of iterating over a documents list and calling single inserts, with RxDB you could use the [bulk methods](https://rxdb.info/rx-collection.html#bulkinsert) to store many document at once.
But most of the time is not so easy. Your user clicks around, data gets replicated from the backend, another browser tab writes data. All these things can happen at random time and you cannot crunch all that data in a single transaction.
Another solution is to just not care about performance at all. In a few releases the browser vendors will have optimized IndexedDB and everything is fast again. Well, IndexedDB was slow [in 2013](https://www.researchgate.net/publication/281065948_Performance_Testing_and_Comparison_of_Client_Side_Databases_Versus_Server_Side) and it is still slow today. If this trend continues, it will still be slow in a few years from now. Waiting is not an option. The chromium devs made [a statement](https://bugs.chromium.org/p/chromium/issues/detail?id=1025456#c15) to focus on optimizing read performance, not write performance.
Switching to WebSQL (even if it is deprecated) is also not an option because, like [the comparison tool shows](https://pubkey.github.io/client-side-databases/database-comparison/index.html), it has even slower transactions.
So you need a way to **make IndexedDB faster**. In the following I lay out some performance optimizations than can be made to have faster reads and writes in IndexedDB.
**HINT:** You can reproduce all performance tests [in this repo](https://github.com/pubkey/indexeddb-performance-tests). In all tests we work on a dataset of 40000 `human` documents with a random `age` between `1` and `100`.
## Batched Cursor
With [IndexedDB 2.0](https://caniuse.com/indexeddb2), new methods were introduced which can be utilized to improve performance. With the `getAll()` method, a faster alternative to the old `openCursor()` can be created which improves performance when reading data from the IndexedDB store.
Lets say we want to query all user documents that have an `age` greater than `25` out of the store.
To implement a fast batched cursor that only needs calls to `getAll()` and not to `getAllKeys()`, we first need to create an `age` index that contains the primary `id` as last field.
```ts
myIndexedDBObjectStore.createIndex(
'age-index',
[
'age',
'id'
]
);
```
This is required because the `age` field is not unique, and we need a way to checkpoint the last returned batch so we can continue from there in the next call to `getAll()`.
```ts
const maxAge = 25;
let result = [];
const tx: IDBTransaction = db.transaction(
[storeName], 'readonly', TRANSACTION_SETTINGS
);
const store = tx.objectStore(storeName);
const index = store.index('age-index');
let lastDoc;
let done = false;
/**
* Run the batched cursor until all results are retrieved
* or the end of the index is reached.
*/
while (done === false) {
await new Promise((res, rej) => {
const range = IDBKeyRange.bound(
/**
* If we have a previous document as checkpoint,
* we have to continue from it's age and id values.
*/
[
lastDoc ? lastDoc.age : -Infinity,
lastDoc ? lastDoc.id : -Infinity,
],
[
maxAge + 0.00000001,
String.fromCharCode(65535)
],
true,
false
);
const openCursorRequest = index.getAll(range, batchSize);
openCursorRequest.onerror = err => rej(err);
openCursorRequest.onsuccess = e => {
const subResult: TestDocument[] = e.target.result;
lastDoc = lastOfArray(subResult);
if (subResult.length === 0) {
done = true;
} else {
result = result.concat(subResult);
}
res();
};
});
}
console.dir(result);
```
As the performance test results show, using a batched cursor can give a huge improvement. Interestingly choosing a high batch size is important. When you known that all results of a given `IDBKeyRange` are needed, you should not set a batch size at all and just directly query all documents via `getAll()`.
RxDB uses batched cursors in the [IndexedDB RxStorage](./rx-storage-indexeddb.md).
## IndexedDB Sharding
Sharding is a technique, normally used in server side databases, where the database is partitioned horizontally. Instead of storing all documents at one table/collection, the documents are split into so called **shards** and each shard is stored on one table/collection. This is done in server side architectures to spread the load between multiple physical servers which **increases scalability**.
When you use IndexedDB in a browser, there is of course no way to split the load between the client and other servers. But you can still benefit from sharding. Partitioning the documents horizontally into **multiple IndexedDB stores**, has shown to have a big performance improvement in write- and read operations while only increasing initial pageload slightly.
As shown in the performance test results, sharding should always be done by `IDBObjectStore` and not by database. Running a batched cursor over the whole dataset with 10 store shards in parallel is about **28% faster** then running it over a single store. Initialization time increases minimal from `9` to `17` milliseconds.
Getting a quarter of the dataset by batched iterating over an index, is even **43%** faster with sharding then when a single store is queried.
As downside, getting 10k documents by their id is slower when it has to run over the shards.
Also it can be much effort to recombined the results from the different shards into the required query result. When a query without a limit is done, the sharding method might cause a data load huge overhead.
Sharding can be used with RxDB with the [Sharding Plugin](./rx-storage-sharding.md).
## Custom Indexes
Indexes improve the query performance of IndexedDB significant. Instead of fetching all data from the storage when you search for a subset of it, you can iterate over the index and stop iterating when all relevant data has been found.
For example to query for all user documents that have an `age` greater than `25`, you would create an `age+id` index.
To be able to run a batched cursor over the index, we always need our primary key (`id`) as the last index field.
Instead of doing this, you can use a `custom index` which can improve the performance. The custom index runs over a helper field `ageIdCustomIndex` which is added to each document on write. Our index now only contains a single `string` field instead of two (age-`number` and id-`string`).
```ts
// On document insert add the ageIdCustomIndex field.
const idMaxLength = 20; // must be known to craft a custom index
docData.ageIdCustomIndex = docData.age + docData.id.padStart(idMaxLength, ' ');
store.put(docData);
// ...
```
```ts
// normal index
myIndexedDBObjectStore.createIndex(
'age-index',
[
'age',
'id'
]
);
// custom index
myIndexedDBObjectStore.createIndex(
'age-index-custom',
[
'ageIdCustomIndex'
]
);
```
To iterate over the index, you also use a custom crafted keyrange, depending on the last batched cursor checkpoint. Therefore the `maxLength` of `id` must be known.
```ts
// keyrange for normal index
const range = IDBKeyRange.bound(
[25, ''],
[Infinity, Infinity],
true,
false
);
// keyrange for custom index
const range = IDBKeyRange.bound(
// combine both values to a single string
25 + ''.padStart(idMaxLength, ' '),
Infinity,
true,
false
);
```
As shown, using a custom index can further improve the performance of running a batched cursor by about `10%`.
Another big benefit of using custom indexes, is that you can also encode `boolean` values in them, which [cannot be done](https://github.com/w3c/IndexedDB/issues/76) with normal IndexedDB indexes.
RxDB uses custom indexes in the [IndexedDB RxStorage](./rx-storage-indexeddb.md).
## Relaxed durability
Chromium based browsers allow to set [durability](https://developer.mozilla.org/en-US/docs/Web/API/IDBTransaction/durability) to `relaxed` when creating an IndexedDB transaction. Which runs the transaction in a less secure durability mode, which can improve the performance.
> The user agent may consider that the transaction has successfully committed as soon as all outstanding changes have been written to the operating system, without subsequent verification.
As shown [here](https://nolanlawson.com/2021/08/22/speeding-up-indexeddb-reads-and-writes/), using the relaxed durability mode can improve performance slightly. The best performance improvement could be measured when many small transactions have to be run. Less, bigger transaction do not benefit that much.
## Explicit transaction commits
By explicitly committing a transaction, another slight performance improvement can be achieved. Instead of waiting for the browser to commit an open transaction, we call the `commit()` method to explicitly close it.
```ts
// .commit() is not available on all browsers, so first check if it exists.
if (transaction.commit) {
transaction.commit()
}
```
The improvement of this technique is minimal, but observable as [these tests](https://nolanlawson.com/2021/08/22/speeding-up-indexeddb-reads-and-writes/) show.
## In-Memory on top of IndexedDB
To prevent transaction handling and to fix the performance problems, we need to stop using IndexedDB as a database. Instead all data is loaded into the memory on the initial page load. Here all reads and writes happen in memory which is about 100x faster. Only some time after a write occurred, the memory state is persisted into IndexedDB with a **single write transaction**. In this scenario IndexedDB is used as a filesystem, not as a database.
There are some libraries that already do that:
- LokiJS with the [IndexedDB Adapter](https://techfort.github.io/LokiJS/LokiIndexedAdapter.html)
- [Absurd-SQL](https://github.com/jlongster/absurd-sql)
- SQL.js with the [empscripten Filesystem API](https://emscripten.org/docs/api_reference/Filesystem-API.html#filesystem-api-idbfs)
- [DuckDB Wasm](https://duckdb.org/2021/10/29/duckdb-wasm.html)
### In-Memory: Persistence
One downside of not directly using IndexedDB, is that your data is not persistent all the time. And when the JavaScript process exists without having persisted to IndexedDB, data can be lost. To prevent this from happening, we have to ensure that the in-memory state is written down to the disc. One point is make persisting as fast as possible. LokiJS for example has the `incremental-indexeddb-adapter` which only saves new writes to the disc instead of persisting the whole state. Another point is to run the persisting at the correct point in time. For example the RxDB [LokiJS storage](https://rxdb.info/rx-storage-lokijs.html) persists in the following situations:
- When the database is idle and no write or query is running. In that time we can persist the state if any new writes appeared before.
- When the `window` fires the [beforeunload event](https://developer.mozilla.org/en-US/docs/Web/API/WindowEventHandlers/onbeforeunload) we can assume that the JavaScript process is exited any moment and we have to persist the state. After `beforeunload` there are several seconds time which are sufficient to store all new changes. This has shown to work quite reliable.
The only missing event that can happen is when the browser exists unexpectedly like when it crashes or when the power of the computer is shut of.
### In-Memory: Multi Tab Support
One big difference between a web application and a 'normal' app, is that your users can use the app in multiple browser tabs at the same time. But when you have all database state in memory and only periodically write it to disc, multiple browser tabs could overwrite each other and you would loose data. This might not be a problem when you rely on a client-server [replication](./replication.md), because the lost data might already be replicated with the backend and therefore with the other tabs. But this would not work when the client is offline.
The ideal way to solve that problem, is to use a [SharedWorker](https://developer.mozilla.org/en/docs/Web/API/SharedWorker). A [SharedWorker](./rx-storage-shared-worker.md) is like a [WebWorker](https://developer.mozilla.org/en/docs/Web/API/Web_Workers_API) that runs its own JavaScript process only that the SharedWorker is shared between multiple contexts. You could create the database in the SharedWorker and then all browser tabs could request the Worker for data instead of having their own database. But unfortunately the SharedWorker API does [not work](https://caniuse.com/sharedworkers) in all browsers. Safari [dropped](https://bugs.webkit.org/show_bug.cgi?id=140344) its support and InternetExplorer or Android Chrome, never adopted it. Also it cannot be polyfilled. **UPDATE:** [Apple added SharedWorkers back in Safari 142](https://developer.apple.com/safari/technology-preview/release-notes/)
Instead, we could use the [BroadcastChannel API](https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_Channel_API) to communicate between tabs and then apply a [leader election](https://github.com/pubkey/broadcast-channel#using-the-leaderelection) between them. The [leader election](./leader-election.md) ensures that, no matter how many tabs are open, always one tab is the `Leader`.
The disadvantage is that the leader election process takes some time on the initial page load (about 150 milliseconds). Also the leader election can break when a JavaScript process is fully blocked for a longer time. When this happens, a good way is to just reload the browser tab to restart the election process.
## Further read
- [Offline First Database Comparison](https://github.com/pubkey/client-side-databases)
- [Speeding up IndexedDB reads and writes](https://nolanlawson.com/2021/08/22/speeding-up-indexeddb-reads-and-writes/)
- [SQLITE ON THE WEB: ABSURD-SQL](https://hackaday.com/2021/08/24/sqlite-on-the-web-absurd-sql/)
- [SQLite in a PWA with FileSystemAccessAPI](https://anita-app.com/blog/articles/sqlite-in-a-pwa-with-file-system-access-api.html)
- [Response to this article by Oren Eini](https://ravendb.net/articles/re-why-indexeddb-is-slow-and-what-to-use-instead)
---
## Alternatives for realtime local-first JavaScript applications and local databases
# Alternatives for realtime offline-first JavaScript applications
To give you an augmented view over the topic of client side JavaScript databases, this page contains all known alternatives to **RxDB**. Remember that you are reading this inside of the RxDB documentation, so everything is **opinionated**.
If you disagree with anything or think that something is missing, make a pull request to this file on the RxDB github repository.
:::note
RxDB has these main benefits:
- RxDB is a battle proven tool [widely used](/#reviews) by companies in real projects in production.
- RxDB is not VC funded and therefore does not require you to use a specific cloud service to rip you off. RxDB can be used with your [own backend](./replication-http.md) or no backend at all.
- RxDB has a working business model of selling [premium plugins](/premium/) which ensures that RxDB will be maintained and improved continuously while many alternatives are dead already or seem to die soon.
- RxDB has years (since 2016) of performance optimization, bug fixing and feature adding. It is just working as is and there are close to zero [open issues](https://github.com/pubkey/rxdb/issues).
:::
--------------------------------------------------------------------------------
## Alternatives to RxDB
[RxDB](https://rxdb.info) is an **observable**, **replicating**, **[local first](./offline-first.md)**, **JavaScript** database. So it makes only sense to list similar projects as alternatives, not just any database or JavaScript store library. However, I will list up some projects that RxDB is often compared with, even if it only makes sense for some use cases.
Here are the alternatives to RxDB:
### Firebase
Firebase is a **platform** developed by Google for creating mobile and web applications. Firebase has many features and products, two of which are client side databases. The [Realtime Database](./articles/firebase-realtime-database-alternative.md) and the [Cloud Firestore](./articles/firestore-alternative.md).
#### Firebase - Realtime Database
The firebase realtime database was the first database in firestore. It has to be mentioned that in this context, "realtime" means **"realtime replication"**, not "realtime computing". The firebase realtime database stores data as a big unstructured JSON tree that is replicated between clients and the backend.
#### Firebase - Cloud Firestore
The firestore is the successor to the realtime database. The big difference is that it behaves more like a 'normal' database that stores data as documents inside of collections. The conflict resolution strategy of firestore is always *last-write-wins* which might or might not be suitable for your use case.
The biggest difference to RxDB is that firebase products are only able to be used on top of the Firebase cloud hosted backend, which creates a vendor lock-in. RxDB can replicate with any self hosted CouchDB server or custom GraphQL endpoints. You can even replicate Firestore to RxDB with the [Firestore Replication Plugin](./replication-firestore.md).
[Read more about why RxDB is a good alternative to Firebase](./articles/firebase-realtime-database-alternative.md).
### Meteor
Meteor (since 2012) is one of the oldest technologies for JavaScript realtime applications. Meteor is not a library but a whole framework with its own package manager, database management and replication.
Because of how it works, it has proven to be hard to integrate it with other modern JavaScript frameworks like [angular](https://github.com/urigo/angular-meteor), [vue.js](./articles/vue-database.md) or svelte.
Meteor uses MongoDB in the backend and can replicate with a Minimongo database in the frontend.
While testing, it has proven to be impossible to make a meteor app **offline first** capable. There are [some projects](https://github.com/frozeman/meteor-persistent-minimongo2) that might do this, but all are unmaintained.
[Read more about why RxDB is a good alternative to Meteor](./articles/alternatives/meteor-alternative.md).
### Minimongo
Forked in Jan 2014 from meteorJSs' minimongo package, Minimongo is a client-side, in-memory, JavaScript version of MongoDB with backend replication over HTTP. Similar to MongoDB, it stores data in documents inside of [collections](./rx-collection.md) and also has the same query syntax. Minimongo has different storage adapters for IndexedDB, WebSQL, [LocalStorage](./articles/localstorage.md) and SQLite.
Compared to RxDB, Minimongo has no concept of revisions or conflict handling, which might lead to undefined behavior when used with replication or in multiple browser tabs. Minimongo has no observable queries or changestream.
[Read more about why RxDB is a good alternative to Minimongo](./articles/alternatives/minimongo-alternative.md).
### WatermelonDB
WatermelonDB is a reactive & asynchronous JavaScript database. While originally made for [React](./articles/react-database.md) and [React Native](./react-native-database.md), it can also be used with other JavaScript frameworks. The main goal of WatermelonDB is **performance** within an application with lots of data.
In React Native, WatermelonDB uses the provided SQLite database. Also there is an Expo plugin for WatermelonDB. In a browser, WatermelonDB uses the LokiJS in-memory database to store and query data. WatermelonDB is one of the rare projects that support both Flow and Typescript at the same time.
[Read more about why RxDB is a good alternative to WatermelonDB](./articles/alternatives/watermelondb-alternative.md).
### AWS Amplify
AWS Amplify is a collection of tools and libraries to develop web- and mobile frontend applications. Similar to firebase, it provides everything needed like authentication, analytics, a REST API, storage and so on. Everything hosted in the AWS Cloud, even when they state that *"AWS Amplify is designed to be open and pluggable for any custom backend or service"*. For realtime replication, AWS Amplify can connect to an AWS App-Sync GraphQL endpoint.
### AWS Datastore
Since December 2019 the Amplify library includes the AWS Datastore which is a document-based, client side database that is able to replicate data via AWS AppSync in the background.
The main difference to other projects is the complex project configuration via the amplify cli and the bit confusing query syntax that works over functions. Complex Queries with multiple `OR/AND` statements are not possible which might change in the future.
Local development is hard because the AWS AppSync mock does not support realtime replication. It also is not really offline-first because a user login is always required.
**DataStore is deprecated.** Amplify Gen 1 (which includes DataStore) entered maintenance mode in 2024 and reaches end-of-life on May 1, 2027. Amplify Gen 2 does not include a DataStore replacement, so teams must implement their own offline sync layer before that date.
```ts
// An AWS datastore OR query
const posts = await DataStore.query(Post, c => c.or(
c => c.rating("gt", 4).status("eq", PostStatus.PUBLISHED)
));
// An AWS datastore SORT query
const posts = await DataStore.query(Post, Predicates.ALL, {
sort: s => s.rating(SortDirection.ASCENDING).title(SortDirection.DESCENDING)
});
```
The biggest difference to RxDB is that you have to use the AWS cloud backends. This might not be a problem if your data is at AWS anyway.
[Read more about why RxDB is a good alternative to AWS Amplify DataStore](./articles/alternatives/aws-amplify-datastore-alternative.md).
### RethinkDB
RethinkDB is a backend database that pushed dynamic JSON data to the client in realtime. It was founded in 2009 and the company shut down in 2016.
RethinkDB is not a client side database, it streams data from the backend to the client which of course does not work while offline.
[Read more about why RxDB is a good alternative to RethinkDB](./articles/alternatives/rethinkdb-alternative.md).
### Horizon
Horizon is the client side library for RethinkDB which provides useful functions like authentication, permission management and subscription to a RethinkDB backend. Offline support [never made](https://github.com/rethinkdb/horizon/issues/58) it to horizon.
[Read more about why RxDB is a good alternative to Horizon](./articles/alternatives/horizon-alternative.md).
### Supabase
Supabase labels itself as "*an open source Firebase alternative*". It is a collection of open source tools that together mimic many Firebase features, most of them by providing a wrapper around a PostgreSQL database. While it has realtime queries that run over the wire, like with RethinkDB, Supabase has no client-side storage or replication feature and therefore is not offline first.
[Read more about why RxDB is a good alternative to Supabase](./articles/alternatives/supabase-alternative.md).
### CouchDB
Apache CouchDB is a server-side, document-oriented database that is mostly known for its multi-master replication feature. Instead of having a master-slave replication, with CouchDB you can run replication in any constellation without having a master server as bottleneck where the server even can go off- and online at any time. This comes with the drawback of having a slow replication with much network overhead.
CouchDB has a changestream and a query syntax similar to MongoDB.
[Read more about why RxDB is a good alternative to CouchDB](./articles/alternatives/couchdb-alternative.md).
### PouchDB
PouchDB is a JavaScript database that is compatible with most of the CouchDB API. It has an adapter system that allows you to switch out the underlying storage layer. There are many adapters like for [IndexedDB](./rx-storage-indexeddb.md), [SQLite](./rx-storage-sqlite.md), the Filesystem and so on. The main benefit is to be able to replicate data with any CouchDB compatible endpoint.
Because of the CouchDB compatibility, PouchDB has to do a lot of overhead in handling the revision tree of documents, which is why it can show bad performance for bigger datasets.
RxDB was originally build around PouchDB until the storage layer was abstracted out in version [10.0.0](./releases/10.0.0.md) so it now allows to use different `RxStorage` implementations. PouchDB has some performance issues because of how it has to store the document revision tree to stay compatible with the CouchDB API.
[Read more about why RxDB is a good alternative to PouchDB](./articles/alternatives/pouchdb-alternative.md).
### Couchbase
Couchbase (originally known as Membase) is another NoSQL document database made for realtime applications.
It uses the N1QL query language which is more SQL like compared to other NoSQL query languages. In theory you can achieve replication of a Couchbase with a PouchDB database, but this has shown to be not [that easy](https://github.com/pouchdb/pouchdb/issues/7793#issuecomment-501624297).
[Read more about why RxDB is a good alternative to Couchbase](./articles/alternatives/couchbase-alternative.md).
### Cloudant
Cloudant is a cloud-based service that is based on [CouchDB](./replication-couchdb.md) and has mostly the same features.
It was originally designed for cloud computing where data can automatically be distributed between servers. But it can also be used to replicate with frontend PouchDB instances to create scalable web applications.
It was bought by IBM in 2014 and since 2018 the Cloudant Shared Plan is retired and migrated to IBM Cloud.
[Read more about why RxDB is a good alternative to Cloudant](./articles/alternatives/cloudant-alternative.md).
### Hoodie
Hoodie is a backend solution that enables offline-first JavaScript frontend development without having to write backend code. Its main goal is to abstract away configuration into simple calls to the Hoodie API.
It uses CouchDB in the backend and PouchDB in the frontend to enable offline-first capabilities.
The last commit for hoodie was one year ago and the website (hood.ie) is offline which indicates it is not an active project anymore.
[Read more about why RxDB is a good alternative to Hoodie](./articles/alternatives/hoodie-alternative.md).
### LokiJS
LokiJS is a JavaScript embeddable, in-memory database. And because everything is handled in-memory, LokiJS has awesome performance when mutating or querying data. You can still persist to a permanent storage (IndexedDB, Filesystem etc.) with one of the provided storage adapters. The persistence happens after a timeout is reached after a write, or before the JavaScript process exits. This also means you could loose data when the JavaScript process exits ungracefully like when the power of the device is shut down or the browser crashes.
While the project is not that active anymore, it is more *finished* than *unmaintained*.
In the past, RxDB supported using [LokiJS as RxStorage](./rx-storage-lokijs.md) but because the LokiJS is not maintained anymore and had too many issues, this storage option was removed in RxDB version 16.
[Read more about why RxDB is a good alternative to LokiJS](./articles/alternatives/lokijs-alternative.md).
### Gundb
GUN is a JavaScript graph database. While having many features, the **decentralized** replication is the main unique selling point. You can replicate data Peer-to-Peer without any centralized backend server. GUN has several other features that are useful on top of that, like [encryption](./encryption.md) and authentication.
While testing it was really hard to get basic things running. GUN is open source, but because of how the source code [is written](https://github.com/amark/gun/blob/master/src/put.js), it is very difficult to understand what is going wrong.
[Read more about why RxDB is a good alternative to Gundb](./articles/alternatives/gundb-alternative.md).
### sql.js
sql.js is a javascript library to run SQLite on the web. It uses a virtual database file stored in memory and does not have any persistence. All data is lost once the JavaScript process exits. sql.js is created by compiling SQLite to WebAssembly so it has about the same features as SQLite. For older browsers there is a JavaScript fallback.
[Read more about why RxDB is a good alternative to sql.js](./articles/alternatives/sql-js-alternative.md).
### absurd-sQL
Absurd-sql is a project that implements an IndexedDB-based persistence for sql.js. Instead of directly writing data into the IndexedDB, it treats IndexedDB like a disk and stores data in blocks there which shows to have a much better performance, mostly because of how [performance expensive](./slow-indexeddb.md) IndexedDB transactions are.
[Read more about why RxDB is a good alternative to absurd-sQL](./articles/alternatives/absurd-sql-alternative.md).
### NeDB
NeDB was a embedded persistent or in-memory database for Node.js, nw.js, [Electron](./electron-database.md) and browsers.
It is document-oriented and had the same query syntax as MongoDB.
Like LokiJS it has persistence adapters for IndexedDB etc. to persist the database state on the disc.
The last commit to NeDB was in **2016**.
[Read more about why RxDB is a good alternative to NeDB](./articles/alternatives/nedb-alternative.md).
### Dexie.js
Dexie.js is a minimalistic wrapper for IndexedDB. While providing a better API than plain IndexedDB, Dexie also improves performance by batching transactions and other optimizations. It also adds additional non-IndexedDB features like observable queries or multi tab support or react hooks.
Compared to RxDB, Dexie.js does not support complex (MongoDB-like) queries and requires a lot of fiddling when a document range of a specific index must be fetched.
Dexie.js is used by Whatsapp Web, Microsoft To Do and Github Desktop.
RxDB supports using [Dexie.js as Database storage](./rx-storage-dexie.md) which enhances IndexedDB via dexie with RxDB features like MongoDB-like queries etc.
[Read more about why RxDB is a good alternative to Dexie.js](./articles/alternatives/dexie-alternative.md).
### LowDB
LowDB is a small, local JSON database powered by the Lodash library. It is designed to be simple, easy to use, and straightforward. LowDB allows you to perform native JavaScript queries and persist data in a flat JSON file. Written in TypeScript, it's particularly well-suited for small projects, prototyping, or when you need a lightweight, file-based database.
As an alternative to LowDB, [RxDB](./) offers real-time [reactivity](./reactivity.md), allowing developers to subscribe to database changes, a feature not natively available in LowDB. Additionally, RxDB provides robust [query capabilities](./rx-query.md), including the ability to subscribe to query results for automatic UI updates. These features make RxDB a strong alternative to LowDB for more complex and dynamic applications.
[Read more about why RxDB is a good alternative to LowDB](./articles/alternatives/lowdb-alternative.md).
### localForage
localForage is a popular JavaScript library for offline storage that provides a simple, promise-based API. It abstracts over different storage mechanisms such as [IndexedDB](./rx-storage-indexeddb.md), WebSQL, or [localStorage](./articles/localstorage.md), making it easier to write code once and have it work seamlessly across various browsers. While localForage is great for storing data locally in a key-value manner, it doesn't provide the real-time reactive queries, [conflict handling](./transactions-conflicts-revisions.md), or revision-based replication that RxDB does. This makes localForage a useful choice for straightforward caching or persistent storage needs, but not ideal for advanced offline-first scenarios requiring multi-user collaboration or complex querying.
[Read more about why RxDB is a good alternative to localForage](./articles/alternatives/localforage-alternative.md).
### MongoDB Realm
Originally Realm was a mobile database for Android and iOS. Later they added support for other languages and runtimes, also for JavaScript.
It was meant as replacement for SQLite but is more like an object store than a full SQL database.
In 2019 MongoDB bought Realm and changed the projects focus.
Now Realm is made for replication with the MongoDB Realm Sync based on the MongoDB Atlas Cloud platform. This tight coupling to the MongoDB cloud service is a big downside for most use cases.
[Read more about why RxDB is a good alternative to MongoDB Realm](./articles/alternatives/mongodb-realm-alternative.md). If you plan to switch, follow the [Realm to RxDB migration guide](./articles/realm-to-rxdb-migration.md).
### Apollo
The Apollo [GraphQL](./replication-graphql.md) platform is made to transfer data between a server to UI applications over GraphQL endpoints. It contains several tools like GraphQL clients in different languages or libraries to create GraphQL endpoints.
While it is has different caching features for offline usage, compared to RxDB it is not fully offline first because caching alone does not mean your application is fully usable when the user is offline.
[Read more about why RxDB is a good alternative to Apollo](./articles/alternatives/apollo-alternative.md).
### Replicache
Replicache is a client-side sync framework for building realtime, collaborative, [local-first](./articles/local-first-future.md) web apps. It claims to work with most backend stacks. In contrast to other local first tools, replicache does not work like a local database. Instead it runs on so called `mutators` that unify behavior on the client and server side. So instead of implementing and calling REST routes on both sides of your stack, you will implement mutators that define a specific delta behavior based on the input data. To observe data in replicache, there are `subscriptions` that notify your frontend application about changes to the state.
Replicache can be used in most frontend technologies like browsers, React/Remix, NextJS/Vercel and React Native. While Replicache can be installed and used from npm, the Replicache source code is not open source and the Replicache github repo does not allow you to inspect or debug it. Still you can use replicache for in non-commercial projects, or for companies with < $200k revenue (ARR) and < $500k in funding. (2024: Replicache will be free and Rocicorp are working on a new Zerosync product to succeed Replicache and Reflect.)
[Read more about why RxDB is a good alternative to Replicache](./articles/alternatives/replicache-alternative.md).
### InstantDB
InstantDB is designed for real-time data synchronization with built-in offline support, allowing changes to be queued locally and [synced](./replication.md) when the user reconnects. While it offers seamless [optimistic updates](./articles/optimistic-ui.md) and rollback capabilities, its offline-first design is not as mature or comprehensive as RxDB's - the [offline data](./articles/offline-database.md) is more of a cache, not a full-database sync. The query language used is Datalog, and the backend sync service is written in Clojure. InstantDB is focused more on simplicity and real-time collaboration, with fewer customization options for storage or conflict resolution compared to RxDB, which supports various storage adapters and advanced conflict handling via CRDTs.
[Read more about why RxDB is a good alternative to InstantDB](./articles/alternatives/instantdb-alternative.md).
### Yjs
Yjs is a [CRDT-based](./crdt.md) (Conflict-free Replicated Data Type) library focused on enabling real-time collaboration - particularly for text editing, although it can handle other data types as well. While it provides powerful conflict resolution and peer-to-peer synchronization out of the box, Yjs itself is not a full-fledged database. Instead, you typically combine Yjs with other storage or networking layers to achieve a [local-first architecture](./offline-first.md). This flexibility allows for sophisticated [real-time](./articles/realtime-database.md) features, but also means you must handle indexing, queries, and persistence on your own if you need them. Compared to RxDB, Yjs does not offer built-in replication adapters or a query system, so developers who require a more complete solution for conflict resolution, data persistence, and offline-first capabilities may find RxDB more convenient.
[Read more about why RxDB is a good alternative to Yjs](./articles/alternatives/yjs-alternative.md).
### ElectricSQL
2024: ElectricSQL is being rewritten in a new Electric-Next branch, which focuses on partial syncing of ("shapes", which makes is basically a NoSQL like document database) of data from a remote Postgres DB to a local clients written in TypeScript/JS or Elixir. The write path is not yet implemented, neither is client-side reactivity. The ElectricSQL backend is written in Elixir.
[Read more about why RxDB is a good alternative to ElectricSQL](./articles/alternatives/electricsql-alternative.md).
### SignalDB
SignalDB provides a reactive, in-memory local-first JavaScript database with real-time sync, but it doesn't offer the same level of multi-client replication or flexibility with storage backends that RxDB provides, and through a RxDB persistence adapters you can actually use SignalDB for the front-end reactivity while relying on RxDB for backend sync and persistence.
[Read more about why RxDB is a good alternative to SignalDB](./articles/alternatives/signaldb-alternative.md).
### PowerSync
PowerSync is a "framework" for implementing local-first solutions. It centralizes business logic and conflict resolution on a central, authoritative server (PostgreSQL or MongoDB), vs RxDB that also supports custom backends. Both RxDB and PowerSync can be used with a variety of storage backends, but PowerSync uses SQLite as the front-end database which has shown to be slow because the WASM-SQLite abstraction increases read and write latency. In terms of client SDKs, PowerSync offers Flutter, Kotlin, and Swift in addition to JS/TypeScript. PowerSync offers many client technologies, PowerSync is under a license that restricts commercial use that competes with PowerSync and the JourneyApps Platform.
[Read more about why RxDB is a good alternative to PowerSync](./articles/alternatives/powersync-alternative.md).
# Read further
- [Offline First Database Comparison](https://github.com/pubkey/client-side-databases)
---
## RxDB as an absurd-sql Alternative for JS Apps That Need a Real Database
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as an absurd-sql Alternative for JS Apps That Need a Real Database
[absurd-sql](https://github.com/jlongster/absurd-sql) is a clever piece of low-level plumbing. It maps SQLite-on-WASM file I/O onto IndexedDB blocks so SQLite can persist data in the browser with reasonable performance. That trick worked well in 2021, but most product teams do not want to maintain raw SQL boilerplate, hand-written migrations, transaction wrappers, and a custom query subscription layer on top of a SQLite VFS shim. They want indexes, reactive queries, replication, schema validation, and observability out of the box.
This page compares **absurd-sql** with **RxDB** and shows where each one fits. If you already invested time into absurd-sql and now hit limits around reactivity, sync, multi-tab coordination, or schema management, RxDB is worth a look.
## A Short History of absurd-sql
absurd-sql was published by James Long around 2021 with a single, focused idea: SQLite compiled to WebAssembly is fast, but durable persistence in the browser was awkward. The available IndexedDB SQLite VFS implementations made one IndexedDB transaction per query, and IndexedDB transactions are slow. absurd-sql treated IndexedDB as a block device and stored SQLite pages as fixed-size blocks, batching reads and writes. The result was much faster than naive IndexedDB persistence and enabled production apps like [Actual Budget](https://actualbudget.org/) to ship a real SQL database in the browser.
The web platform has moved on since then. The [Origin Private File System (OPFS)](../../rx-storage-opfs.md) is now widely available and gives WASM SQLite direct synchronous file access through `FileSystemSyncAccessHandle`. Most modern SQLite-in-browser stacks (including the official `sqlite-wasm` build from the SQLite team) use OPFS as the default storage backend. The original problem absurd-sql solved is largely addressed by OPFS today, and the absurd-sql repository itself has not seen active development for some time.
## What is RxDB?
[RxDB](https://rxdb.info/) is a [local-first](../../articles/local-first-future.md), reactive, NoSQL JavaScript database. It runs in the browser, [Node.js](../../nodejs-database.md), [React Native](../../react-native-database.md), [Electron](../../electron-database.md), and other JavaScript runtimes. RxDB separates the **database engine** from the **storage layer** so the same application code works on top of IndexedDB, OPFS, SQLite, in-memory, or custom storages.
The features that distinguish RxDB from a raw SQL VFS shim:
- **Schemas** validated against JSON Schema with versioned migrations.
- **Reactive queries** that emit new results when underlying data changes (see [reactivity](../../reactivity.md)).
- **Replication primitives** for syncing with any HTTP, WebSocket, GraphQL, or P2P backend (see [replication](../../replication.md)).
- **Storage agnostic** with adapters for [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [SQLite](../../rx-storage-sqlite.md), memory, and others.
- **Multi-tab safe** with leader election and cross-tab event propagation.
- **Offline-first by default** ([offline-first](../../offline-first.md)).
## Limitations of absurd-sql
absurd-sql is a storage backend, not an application database. The gaps that show up in real projects:
### 1. Low-level API surface
absurd-sql exposes SQLite. Every collection, index, foreign key, migration, and query is hand-written SQL. There is no schema validation framework, no document model, no query builder. Teams end up writing thin ORMs, type adapters, and migration runners themselves, which is the work the database should do.
### 2. No observable queries
SQLite (and absurd-sql by extension) does not push change events. To keep a UI in sync with the database you have to invalidate queries manually after every write, or build your own pub/sub layer. RxDB's [RxQuery](../../rx-query.md) returns an observable that emits new result sets whenever a matching document changes, with the `EventReduce` algorithm minimizing recomputation cost.
### 3. No replication
absurd-sql does not ship a sync protocol. If you want offline-first sync with a server, you build it: change tracking tables, conflict resolution, push and pull endpoints, retry logic, and checkpoint storage. RxDB includes a [replication protocol](../../replication.md) with first-party plugins for HTTP, GraphQL, WebRTC, CouchDB, Firestore, and more.
### 4. Blocking work on the SQLite WASM thread
Running SQLite in the main thread blocks the UI during heavy queries. Running it in a Web Worker (the recommended setup for absurd-sql) means every query crosses a `postMessage` boundary, and the IndexedDB block reads still happen synchronously inside that worker through `Atomics.wait`. Long transactions stall the worker for everything else routed through it.
### 5. Dated approach now that OPFS exists
OPFS provides synchronous file access designed for exactly this use case. New SQLite-in-browser projects target OPFS first and fall back to IndexedDB only for older browsers. The block-emulation trick that absurd-sql pioneered is no longer the fastest path on modern browsers.
### 6. No multi-tab coordination
If a user opens your app in two tabs, both tabs talk to the same IndexedDB blocks. absurd-sql does not coordinate writers across tabs, so concurrent writes can produce surprises. RxDB elects a leader tab, broadcasts events, and serializes writes through a single storage instance.
### 7. No built-in encryption, attachments, or backups
These are common requirements for local-first apps. With absurd-sql you build them on top of SQL. RxDB ships them as plugins.
The numbers reflect this. As of July 30, 2026, [absurd-sql](https://github.com/jlongster/absurd-sql) has 4,325 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `absurd-sql` package was downloaded 55,008 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/absurd-sql-vs-rxdb)). The last commit to the [absurd-sql repository](https://github.com/jlongster/absurd-sql) was in August 2023, .
## Why RxDB Fits Better for Most Apps
### Storage-agnostic with modern options
You pick the storage that matches your runtime and constraints, and you can swap it without changing application code:
- [OPFS storage](../../rx-storage-opfs.md) for the fastest persistent option in modern browsers.
- [IndexedDB storage](../../rx-storage-indexeddb.md) for broad compatibility (and for working around the [slow IndexedDB problem](../../slow-indexeddb.md) using RxDB's optimizations).
- [SQLite storage](../../rx-storage-sqlite.md) when you do want SQLite under the hood, in Node.js, Electron, React Native, or in browsers via `sqlite-wasm`.
- Memory storage for tests.
### Reactive queries and collections
[RxCollection](../../rx-collection.md) and [RxQuery](../../rx-query.md) expose RxJS observables. The UI subscribes once and stays in sync.
### Replication built in
The [sync engine](../../replication.md) handles checkpoints, conflict resolution, and live updates. Plug it into REST, GraphQL, WebSocket, WebRTC, or any custom transport.
### Schema validation and migrations
Define a JSON Schema once. RxDB validates inserts, generates types, and runs versioned migrations when the schema changes.
## Code Sample: Collection and Reactive Query
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'budget',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
transactions: {
schema: {
title: 'transaction schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
amount: { type: 'number' },
category: { type: 'string', maxLength: 50 },
date: { type: 'string', format: 'date-time' }
},
required: ['id', 'amount', 'category', 'date'],
indexes: ['category', 'date']
}
}
});
// Reactive query: emits a new array whenever a matching doc changes.
const groceries$ = db.transactions
.find({ selector: { category: 'groceries' }, sort: [{ date: 'desc' }] })
.$;
groceries$.subscribe(docs => {
console.log('Groceries updated:', docs.length);
});
await db.transactions.insert({
id: 't1',
amount: 42.5,
category: 'groceries',
date: new Date().toISOString()
});
```
No SQL strings, no manual cache invalidation, no hand-rolled change feed.
## Code Sample: Switching Storages Without Rewriting Code
One of the practical pains with absurd-sql is that it is tied to its specific IndexedDB-block layout. With RxDB the storage is a parameter. Moving from IndexedDB to OPFS is a one-line change.
```ts
// Before: IndexedDB
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'budget',
storage: getRxStorageIndexedDB()
});
```
```ts
// After: OPFS, same collections, same queries, same replication.
import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
const db = await createRxDatabase({
name: 'budget',
storage: getRxStorageOPFS()
});
```
Schemas, queries, replication setup, and reactive subscriptions stay identical. The same pattern applies if you target Node.js with [SQLite storage](../../rx-storage-sqlite.md) or React Native.
## FAQ
No. RxDB has its own [storage layer abstraction](../../rx-storage-indexeddb.md) and ships several first-party storage adapters. For browser persistence you can pick the [IndexedDB storage](../../rx-storage-indexeddb.md) or the [OPFS storage](../../rx-storage-opfs.md). If you want SQLite specifically, the [SQLite storage](../../rx-storage-sqlite.md) uses `sqlite-wasm` (or native SQLite on Node.js and React Native) without the absurd-sql block-on-IndexedDB trick.
For most modern browsers, yes. OPFS gives WASM modules synchronous file access through `FileSystemSyncAccessHandle`, which is what SQLite wants. The official `sqlite-wasm` build from the SQLite team uses OPFS as its primary persistent VFS. RxDB exposes this through the [OPFS storage](../../rx-storage-opfs.md). absurd-sql's IndexedDB-as-block-device approach was a workaround for the absence of OPFS, and that absence is mostly gone.
RxDB's primary query API is a NoSQL Mongo-style selector with sort, skip, and limit, designed for reactive subscriptions. If you specifically need SQL semantics, the [SQLite storage](../../rx-storage-sqlite.md) lets you use SQLite as the underlying engine while still keeping RxDB's schemas, [reactive queries](../../reactivity.md), and replication on top. Most applications find the document API plus indexes covers what they would otherwise write in SQL.
RxDB elects a leader tab using the BroadcastChannel API and serializes writes through a single storage instance, then broadcasts change events to every other tab. Reactive queries in all tabs update automatically when one tab writes a document. absurd-sql does not provide cross-tab coordination, so applications using it have to handle concurrent writers themselves.
## Comparison Table
| Capability | absurd-sql | RxDB |
| --- | --- | --- |
| Data model | Raw SQL tables | JSON Schema documents |
| Query API | Hand-written SQL | NoSQL selectors plus indexes |
| Reactive queries | Manual invalidation | Built-in observables |
| Schema validation | Application code | JSON Schema, enforced |
| Migrations | Hand-written | Versioned, declarative |
| Replication / sync | Not included | First-party plugins |
| Browser storage | IndexedDB blocks only | IndexedDB, OPFS, memory |
| Other runtimes | Browser focused | Browser, Node.js, React Native, Electron |
| Multi-tab coordination | None | Leader election plus events |
| Encryption, attachments, backups | Build yourself | Plugins |
| Active maintenance | Stagnant | Active |
## When absurd-sql Still Makes Sense
absurd-sql is reasonable if you already have a large SQL codebase, you need bit-for-bit SQLite semantics in the browser, you cannot rely on OPFS in your target browsers, and you are willing to maintain the surrounding application database concerns yourself. For most new projects, starting with RxDB and picking the storage that matches the runtime is a faster path to a working local-first app.
## Getting Started with RxDB
```bash
npm install rxdb rxjs
```
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageOPFS()
});
```
From there, add [collections](../../rx-collection.md), write [reactive queries](../../rx-query.md), and connect [replication](../../replication.md) when you are ready to sync. If you later decide to swap storages, the rest of your application code does not change.
More resources:
- [RxDB Storage: OPFS](../../rx-storage-opfs.md)
- [RxDB Storage: IndexedDB](../../rx-storage-indexeddb.md)
- [RxDB Storage: SQLite](../../rx-storage-sqlite.md)
- [Why IndexedDB is slow](../../slow-indexeddb.md)
- [The local-first future](../../articles/local-first-future.md)
- [RxDB Replication](../../replication.md)
---
## RxDB as an Apollo Client Alternative for Truly Offline-First Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as an Apollo Client Alternative for Truly Offline-First Apps
The Apollo GraphQL platform is built to move data between a server and UI applications over GraphQL endpoints. It ships with GraphQL clients for several languages, server libraries to build endpoints, and tools for federation and observability. Apollo Client offers caching features that can persist data for offline reads, but caching alone does not make an application fully usable when the user is offline. Teams that start with Apollo Client and try to push the cache into an [offline-first](../../offline-first.md) architecture often hit a wall, because the cache is a transport optimization, not a database.
If you need apps that start offline, write while offline, sync reliably when the network returns, and resolve conflicts deterministically, RxDB is a more direct fit. You can keep Apollo for the GraphQL transport and use RxDB for storage and sync, or replace the client cache entirely.
## A Short History of Apollo
Apollo Client started around 2016 as a project from the Meteor Development Group, the team behind the Meteor framework. After Meteor's data layer (Minimongo with DDP) showed the value of reactive data on the client, the group spun out Apollo as a GraphQL-first successor with a transport-agnostic design.
The platform grew across several products:
- **Apollo Client** for JavaScript, iOS, Android, and Kotlin Multiplatform.
- **Apollo Server** as a reference GraphQL server in Node.js.
- **Apollo Federation** for composing multiple GraphQL services into one supergraph.
- **Apollo Studio** (later GraphOS) for schema registry, metrics, and CI checks.
Apollo became one of the most adopted GraphQL toolchains, with a strong ecosystem of code generators, dev tools, and integrations with React, Vue, Angular, and Svelte. Its normalized in-memory cache, paired with `apollo-cache-persist`, is what most teams reach for when they think about offline support.
## What is RxDB?
[RxDB](https://rxdb.info/) (Reactive Database) is a local-first, NoSQL database for JavaScript. It runs in browsers, Node.js, Electron, React Native, Capacitor, Deno, and Bun. Data is stored on the client through a pluggable storage layer (IndexedDB, OPFS, SQLite, in-memory, and others), validated against a [JSON schema](../../rx-schema.md), and exposed through [observable queries](../../reactivity.md) so the UI updates automatically when data changes.
RxDB ships a generic [Sync Engine](../../replication.md) with ready-made plugins for [GraphQL](../../replication-graphql.md), HTTP, CouchDB, Firestore, NATS, WebRTC, and more. The replication protocol is designed for offline-first workloads from the start, with checkpoint-based pull, batched push, conflict detection, and live event streams.
## Where Apollo Client Falls Short for Offline-First
Apollo Client was designed around the request and response model of GraphQL. Offline support was added on top through cache persistence and link middleware. That foundation creates several limits when you need a real offline-first app.
### 1. The Cache is Not a Database
Apollo's normalized cache stores query results keyed by the queries that produced them. It is optimized for deduplication and re-rendering, not for arbitrary local queries. You cannot run an ad hoc filter, sort, or aggregation across the cache the way you would against a database. If a screen needs data shaped differently from the original query, you either re-query the server or write custom resolver logic.
RxDB stores documents in [collections](../../rx-collection.md) with their own indexes. You can run any [Mango-style query](../../rx-query.md) over local data without touching the network.
### 2. No Schema-Driven Persistence
Apollo's cache structure follows your GraphQL queries. Persistence with `apollo-cache-persist` writes the entire normalized cache to storage as a blob and reads it back at startup. There is no per-document schema validation, no migration system, and no fine-grained control over which fields persist.
RxDB requires a [JSON schema](../../rx-schema.md) per collection. Documents are validated on insert and update, schema versions trigger [migrations](../../migration-schema.md), and storage is document-level rather than blob-level.
### 3. No Conflict Handling on Writes
Apollo Client treats mutations as fire-and-forget RPCs. Optimistic responses can update the cache before the server replies, but if the device is offline when the mutation runs, the operation fails unless you wrap it in a queue. There is no built-in concept of revisions, vector clocks, or merge functions.
RxDB tracks revisions on every document and runs writes through a pluggable [conflict handler](../../transactions-conflicts-revisions.md). When the same document is modified locally and remotely, your handler decides how to merge, keep, or split the changes.
### 4. Fragile Write Queues
The common pattern for offline writes with Apollo is `apollo-link-queue` or a similar custom link that holds mutations while offline and replays them when the connection returns. These queues are not persisted by default, do not survive a tab reload reliably, and do not coordinate with the cache once the server response shape differs from the optimistic update.
RxDB's [replication](../../replication.md) persists every local change as part of the document store. A push handler is called with batched changes, retried on failure, and resumed across reloads through checkpoints.
### 5. No Multi-Tab Synchronization
Apollo Client instances in different browser tabs do not share state. Two tabs of the same app keep separate caches, and a write in one tab does not update the other unless both refetch from the server.
RxDB uses a [BroadcastChannel-based leader election](../../leader-election.md) so that multiple tabs share one logical database. A write in any tab streams to all other tabs through the same observable queries that drive the UI.
### 6. Normalized Cache Eviction Issues
Apollo's cache can grow without bound, and garbage collection through `cache.gc()` removes entries based on reachability from active queries. This is fine for a session cache, but it makes the cache an unreliable source of truth for data the user expects to be there next time the app opens.
RxDB documents stay in storage until you delete them. Storage size is bounded by the underlying engine (IndexedDB, OPFS, SQLite) rather than by query reachability.
Both projects are under active development. As of July 30, 2026, [Apollo Client](https://github.com/apollographql/apollo-client) has 19,808 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## What RxDB Brings to the Table
### A Real Local Database
Documents are persisted in a [pluggable storage backend](../../rx-storage.md). You can pick IndexedDB or OPFS in browsers, SQLite in React Native and Electron, in-memory for tests, and swap storages without changing application code.
### GraphQL-Friendly Replication
The [GraphQL replication plugin](../../replication-graphql.md) speaks the same protocol you would build for Apollo: a `pullQuery` that returns documents plus a checkpoint, a `pushMutation` that accepts an array of changes, and a subscription for live updates. You keep your GraphQL server, schema, and auth setup. RxDB replaces the client-side cache and queue.
### Conflict Resolution
Every collection has a [conflict handler](../../transactions-conflicts-revisions.md) that runs on both local and remote write paths. You can implement last-write-wins, field-level merges, CRDT-style logic, or domain-specific rules.
### Schema Validation
[Schemas](../../rx-schema.md) are JSON Schema documents. They define types, required fields, indexes, encrypted fields, and primary keys. Schema changes are versioned and run through a [migration strategy](../../migration-schema.md) at startup.
### Observable Queries
Every [RxQuery](../../rx-query.md) is an [Observable](../../reactivity.md). The UI subscribes once and receives a new result whenever any document that affects the query changes, whether the change came from a local write, a replication pull, or another browser tab. This is the foundation for [optimistic UI](../../articles/optimistic-ui.md) without manual cache manipulation.
### Multi-Tab and Multi-Storage
Multiple tabs of the same origin share one logical database through leader election. Storage backends can be combined through [storage wrappers](../../rx-storage.md) for encryption, validation, sharding, or worker offloading.
## Code Sample: Replicate an RxDB Collection over GraphQL
The example below mirrors a typical Apollo setup but stores data in RxDB and uses the [GraphQL replication plugin](../../replication-graphql.md) for sync.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { replicateGraphQL } from 'rxdb/plugins/replication-graphql';
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
multiInstance: true,
eventReduce: true
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'updatedAt']
}
}
});
const replicationState = replicateGraphQL({
collection: db.tasks,
url: {
http: 'https://example.com/graphql',
ws: 'wss://example.com/graphql'
},
pull: {
queryBuilder: (checkpoint, limit) => ({
query: `query Pull($checkpoint: Checkpoint, $limit: Int!) {
pullTasks(checkpoint: $checkpoint, limit: $limit) {
documents { id title done updatedAt _deleted }
checkpoint { id updatedAt }
}
}`,
variables: { checkpoint, limit }
})
},
push: {
queryBuilder: (rows) => ({
query: `mutation Push($rows: [TaskInputRow!]!) {
pushTasks(rows: $rows) { id title done updatedAt }
}`,
variables: { rows }
})
},
live: true,
deletedField: '_deleted',
replicationIdentifier: 'tasks-graphql'
});
```
The push handler returns conflicts as an array of server documents. RxDB then runs the conflict handler for each row before retrying.
## Code Sample: Observable Query in a React Component
Apollo's `useQuery` re-runs against the cache when relevant fields change. With RxDB, the same effect comes from subscribing to an `RxQuery`.
```tsx
import { useEffect, useState } from 'react';
import type { RxDocument } from 'rxdb';
type Task = {
id: string;
title: string;
done: boolean;
updatedAt: number;
};
export function TaskList({ db }) {
const [tasks, setTasks] = useState[]>([]);
useEffect(() => {
const sub = db.tasks
.find({ selector: { done: false }, sort: [{ updatedAt: 'desc' }] })
.$.subscribe(setTasks);
return () => sub.unsubscribe();
}, [db]);
return (
{tasks.map(t => (
{t.title}
))}
);
}
```
The list updates on every local write, every replication pull, and every change from another tab, with no manual cache reads or refetch calls.
## Use Both: Apollo for Transport, RxDB for Storage
Replacing Apollo wholesale is not always the goal. Some teams already run Apollo Federation, persisted queries, and GraphOS metrics, and want to keep them. In that case, RxDB fits next to Apollo rather than in place of it.
A common split looks like this:
- **Apollo Client** handles one-shot queries that do not need offline persistence, such as analytics dashboards or admin screens that are online by definition.
- **RxDB** owns the data that must work offline: user-authored content, drafts, settings, and any list the UI renders frequently.
- **Replication** between RxDB and the GraphQL server reuses the same schema and resolvers Apollo already calls. The [GraphQL replication plugin](../../replication-graphql.md) is server-agnostic and works with Apollo Server, Yoga, Mercurius, or any other GraphQL endpoint.
This split lets you adopt RxDB collection by collection. Start with the most offline-sensitive feature, point its queries at RxDB, and leave the rest of the app on Apollo until you decide to migrate.
## FAQ
No. Apollo Client is a GraphQL client with an in-memory normalized cache. The cache can be persisted to storage through `apollo-cache-persist`, but it does not offer schema validation, migrations, indexes, or local query planning. RxDB is a database with a pluggable storage layer and replication built in.
Yes. The [GraphQL replication plugin](../../replication-graphql.md) implements the RxDB sync protocol on top of GraphQL queries, mutations, and subscriptions. You define a pull query, a push mutation, and an optional subscription for live updates, and RxDB handles checkpoints, batching, retries, and conflict detection.
`apollo-cache-persist` serializes the entire normalized cache to a single storage entry on a debounce timer. It is meant to warm the cache after a reload, not to be a source of truth. RxDB writes each document as it changes, validates against a schema, supports per-collection migrations, and exposes [reactive queries](../../reactivity.md) that fire on every change. Crashes between persist intervals do not cost data because every write is durable on commit.
GraphQL subscriptions still work. The RxDB GraphQL replication plugin can subscribe to a server stream and use each event as a trigger to run a pull. That keeps the protocol resumable through checkpoints while giving you near real-time updates over WebSockets. See the [realtime database](../../articles/realtime-database.md) article for the broader pattern.
Yes, if your app's data flow fits the RxDB model of collections, schemas, and replicated documents. Many teams do this for product surfaces that need offline support and keep a thin GraphQL fetch layer (or plain `fetch`) for one-off requests. If you rely heavily on Apollo Federation tooling on the server, you can keep that and only swap the client side.
## Comparison Table
| Capability | Apollo Client | RxDB |
| --- | --- | --- |
| Primary role | GraphQL client and cache | Local database with sync |
| Storage model | Normalized in-memory cache, optional blob persistence | Document store with pluggable backends (IndexedDB, OPFS, SQLite, memory) |
| Schema validation | None on client | JSON Schema per [collection](../../rx-schema.md) |
| Local queries | Limited to query results in cache | Full [Mango query API](../../rx-query.md) with indexes |
| Reactivity | `useQuery` over cache | [Observable queries](../../reactivity.md) over storage and replication |
| Offline writes | Manual queue link, not persisted by default | Built-in persistent push queue with checkpoints |
| Conflict resolution | None on client | [Custom conflict handler](../../transactions-conflicts-revisions.md) per collection |
| Multi-tab sync | Separate cache per tab | Shared database through leader election |
| Migrations | Not provided | Versioned [schema migrations](../../migration-schema.md) |
| Transport | GraphQL over HTTP and WebSocket | Storage-agnostic; GraphQL, HTTP, CouchDB, Firestore, WebRTC, P2P |
| Server requirement | GraphQL server | Any backend that implements the pull and push handlers |
## Follow Up
If your goal is to make a GraphQL app work offline, an Apollo cache and a queue link will get you partway. For apps that must start offline, survive long disconnects, sync deterministically, and stay consistent across tabs, a real local database is the right shape. RxDB gives you that database and keeps your GraphQL backend in place through the [GraphQL replication plugin](../../replication-graphql.md).
More resources:
- [RxDB Sync Engine](../../replication.md)
- [GraphQL Replication](../../replication-graphql.md)
- [Conflict Resolution](../../transactions-conflicts-revisions.md)
- [Reactivity in RxDB](../../reactivity.md)
- [Local-First Future](../../articles/local-first-future.md)
- [Optimistic UI Patterns](../../articles/optimistic-ui.md)
- [Realtime Database Patterns](../../articles/realtime-database.md)
- [RxDB GitHub Repository](/code/)
---
## RxDB as an AWS Amplify DataStore Alternative - Backend-Agnostic, Offline-First
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
import {Timeline} from '@site/src/components/timeline';
# RxDB as an AWS Amplify DataStore Alternative
AWS Amplify DataStore offered an appealing promise: define a GraphQL schema, run a CLI command, and get automatic offline sync to DynamoDB via AWS AppSync. For teams already embedded in the AWS ecosystem, that promise was compelling. But DataStore was deprecated in Amplify Gen 2, is scheduled for end-of-life on May 1, 2027, and had well-documented limitations around query flexibility, local performance, and its tight coupling to AWS infrastructure. This page explains what DataStore does, where it falls short, and why [RxDB](https://rxdb.info) is a practical alternative for teams building offline-first applications.
---
## What is AWS Amplify DataStore?
AWS Amplify is a collection of tools and libraries for building web and mobile frontends that connect to AWS cloud services. It covers authentication (Cognito), file storage (S3), analytics, and APIs. DataStore is the component that added a client-side offline database layer to the Amplify stack.
DataStore was officially launched in **December 2019**. Its core idea was to give developers a local-first programming model: you write data to a local DataStore on the device, and it synchronizes that data to AWS AppSync (a managed GraphQL service) and DynamoDB in the background. The client SDK stored data in SQLite on mobile and IndexedDB on web.
To define data models, developers wrote a GraphQL schema and then ran the Amplify CLI to generate platform-specific model classes:
```graphql
type Post @model {
id: ID!
title: String!
body: String
status: PostStatus!
rating: Int
}
enum PostStatus {
PUBLISHED
DRAFT
}
```
After running `amplify push`, the CLI created the AppSync API, DynamoDB tables, and generated the client model classes. Querying data used a function-based predicate syntax:
```ts
// A DataStore OR query
const posts = await DataStore.query(Post, c => c.or(
c => c.rating('gt', 4).status('eq', PostStatus.PUBLISHED)
));
// A DataStore sort query
const posts = await DataStore.query(Post, Predicates.ALL, {
sort: s => s.rating(SortDirection.ASCENDING).title(SortDirection.DESCENDING)
});
```
DataStore handled conflict resolution server-side via AWS AppSync using one of three strategies: Auto Merge (default), Optimistic Concurrency, or a custom AWS Lambda function.
### A Brief Timeline
- **December 2019** - DataStore launches as part of AWS Amplify. It targets mobile (iOS, Android) and JavaScript applications.
- **2020-2021** - Adoption grows among teams building React Native and React apps on AWS. DataStore becomes the recommended offline pattern for Amplify apps.
- **2022-2023** - AWS begins working on Amplify Gen 2, a ground-up rethink of the Amplify framework built on top of the AWS CDK. DataStore is not included in Gen 2.
- **2024** - AWS confirms that Amplify Gen 1 (which includes DataStore) has entered maintenance mode. New features are no longer being added.
- **May 2027** - Amplify Gen 1 reaches end-of-life. DataStore will no longer receive security patches or support.
This trajectory means that applications built on DataStore today are accumulating technical debt. Teams must plan a migration before May 2027, with no direct drop-in replacement from AWS.
---
## Key Limitations of AWS Amplify DataStore
### Locked to the AWS Infrastructure Stack
The most fundamental limitation of DataStore is its complete dependence on AWS services. DataStore only synchronizes with AWS AppSync. AppSync only writes to the data sources AWS supports (primarily DynamoDB, Aurora Serverless, and Lambda resolvers). This means:
- You cannot sync DataStore to a self-hosted PostgreSQL or MongoDB instance without building a custom Lambda resolver for every operation.
- You cannot switch your backend from AWS to another provider without rewriting the entire data layer.
- Your application's data costs are determined by DynamoDB pricing, regardless of whether DynamoDB's access patterns match your data.
- Local development requires the `amplify mock` command to simulate AppSync, but that mock does not support real-time subscriptions, making end-to-end offline/online testing difficult on a developer machine.
RxDB has no required backend. You can replicate to CouchDB, any GraphQL endpoint (including AppSync), a custom REST API, Firebase Firestore, or a WebSocket server. You can also run RxDB with no backend at all for purely local applications. Switching backends is a configuration change, not a rewrite.
### Query Language Inflexibility
DataStore's predicate syntax is a custom function-based API that maps to AppSync GraphQL queries. This design has a hard limitation: complex queries combining multiple `AND` and `OR` conditions in arbitrary nesting are not expressible in the standard predicate API. In practice, this forces developers to either fetch more data than needed and filter in JavaScript, or write custom resolvers on the AppSync side.
RxDB uses [Mango queries](../../rx-query.md), a MongoDB-compatible JSON query syntax. These run entirely client-side against the local storage, so they are not limited by what the server can express:
```ts
// Complex query in RxDB: posts that are published AND (rating > 4 OR featured = true)
const results = await db.posts.find({
selector: {
status: 'published',
$or: [
{ rating: { $gt: 4 } },
{ featured: true }
]
},
sort: [{ rating: 'desc' }, { title: 'asc' }]
}).exec();
```
This query runs against local IndexedDB or SQLite with no network round-trip. Because local queries are not constrained by the sync backend's query language, RxDB can support complex filtering that would require custom resolver logic in DataStore.
### "Black Box" Synchronization
DataStore abstracts the sync process entirely. When sync works, this is convenient. When it breaks (stuck sync loops, version conflicts, large dataset startup delays), debugging is difficult because the internals are not exposed.
Common issues reported by DataStore users include:
- Sync loops where the same record is pushed and pulled repeatedly without settling.
- Performance degradation at startup when the local database contains thousands of records, because DataStore performs a full reconciliation scan.
- Sync silently failing when network conditions are intermittent, with no observable status indicator in the default setup.
- Difficulty testing sync behavior because the local mock does not faithfully replicate AppSync's real-time behavior.
RxDB exposes the full replication state as observables. You can subscribe to replication status, active state, errors, and individual document conflicts:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: db.posts,
replicationIdentifier: 'posts-http-v1',
pull: {
handler: async (checkpoint, batchSize) => {
const response = await fetch(
`/api/posts/changes?since=${checkpoint?.updatedAt ?? 0}` +
`&limit=${batchSize}`
);
const data = await response.json();
return { documents: data.documents, checkpoint: data.checkpoint };
}
},
push: {
handler: async (rows) => {
const response = await fetch('/api/posts/push', {
method: 'POST',
body: JSON.stringify(rows),
headers: { 'Content-Type': 'application/json' }
});
return response.json(); // Returns conflicting docs or []
}
},
live: true,
retryTime: 5000
});
// Monitor everything
replicationState.active$.subscribe(active => console.log('Syncing:', active));
replicationState.error$.subscribe(err => console.error('Sync error:', err));
replicationState.sent$.subscribe(docs => console.log('Pushed:', docs.length));
replicationState.received$.subscribe(docs => console.log('Pulled:', docs.length));
```
Nothing is hidden. If sync is failing, you see exactly why.
### Rigid Schema Evolution
DataStore's schema is defined in GraphQL and code-generated by the Amplify CLI. Adding a field means editing the schema, running `amplify push` to update the cloud backend, and regenerating the model classes. Removing or renaming a field requires careful migration planning because old clients with stale code may still be running against the new schema.
Amplify does provide a migration flow, but coordinating client updates with backend schema changes in a production app with many concurrent users is a known source of operational complexity.
RxDB has a built-in [schema migration system](../../migration-schema.md). You increment the schema version number and provide a migration strategy:
```ts
await db.addCollections({
posts: {
schema: postSchemaV2, // version: 1 (incremented from 0)
migrationStrategies: {
// Transform documents from version 0 to version 1
1: (oldDoc) => {
return {
...oldDoc,
status: oldDoc.published ? 'published' : 'draft',
rating: oldDoc.rating ?? 0
};
}
}
}
});
```
When the database opens with a higher schema version, RxDB runs the migration automatically on the local data. The backend schema is independent of the client schema, so client migrations do not require a coordinated backend deployment.
### No Reactive Queries
DataStore provides a subscription API to observe model changes:
```ts
const subscription = DataStore.observe(Post).subscribe(msg => {
console.log(msg.opType, msg.element);
});
```
However, this notifies you that something changed, not what the current query results are. You must re-query after each notification to get the updated result set. There is no equivalent of a live query that re-emits the full current result on every relevant change.
RxDB queries are observable by default. Every query exposes a `$` property that emits the current result set and re-emits automatically whenever the underlying data changes, without polling and without a separate re-query step:
```ts
// This observable emits immediately with current results,
// then re-emits whenever matching posts change
db.posts.find({
selector: { status: 'published' },
sort: [{ rating: 'desc' }]
}).$.subscribe(posts => {
// posts is always the current, up-to-date result set
renderUI(posts);
});
```
RxDB uses the [event-reduce](https://github.com/pubkey/event-reduce) algorithm internally. When a document write occurs, RxDB checks whether the existing query result can be updated by applying the change event directly, without re-executing the full query against storage. This keeps reactive updates fast even in write-heavy workloads.
---
Both projects are under active development. As of July 30, 2026, [Amplify](https://github.com/aws-amplify/amplify-js) has 9,555 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## How RxDB Covers the DataStore Use Case
### Replicating with AWS AppSync via GraphQL
If your existing backend is AWS AppSync, RxDB can replicate with it directly using the [GraphQL replication plugin](../../replication-graphql.md). You keep AppSync as your sync backend but replace the DataStore client with RxDB:
```ts
import { replicateGraphQL } from 'rxdb/plugins/replication-graphql';
const replicationState = await replicateGraphQL({
collection: db.posts,
url: {
http:
'https://your-appsync-endpoint.appsync-api.us-east-1.amazonaws.com' +
'/graphql'
},
headers: {
'x-api-key': 'your-api-key'
},
pull: {
queryBuilder: (checkpoint, limit) => ({
query: `
query SyncPosts($lastSync: AWSTimestamp, $limit: Int) {
syncPosts(lastSync: $lastSync, limit: $limit) {
items { id title body status rating _deleted _lastChangedAt }
nextToken
}
}
`,
variables: {
lastSync: checkpoint?.updatedAt ?? 0,
limit
}
}),
responseModifier: (response) => {
return {
documents: response.data.syncPosts.items,
checkpoint: { updatedAt: Date.now() }
};
}
},
push: {
queryBuilder: (rows) => ({
query: `
mutation CreateOrUpdatePost($input: CreatePostInput!) {
createPost(input: $input) { id title body status rating }
}
`,
variables: { input: rows[0].newDocumentState }
})
},
live: true
});
```
This means you can migrate from DataStore to RxDB incrementally: keep AppSync running, replace the client SDK, and gain RxDB's reactive queries, flexible storage, and observable replication state without touching the backend.
### Pluggable Storage for Any Environment
DataStore used SQLite on mobile and IndexedDB on web. RxDB supports both of these and adds more options:
| Environment | Storage Option |
|---|---|
| Browser (standard) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high-throughput) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multi-tab browsers | [SharedWorker](../../rx-storage-shared-worker.md) |
| Testing / CI | [Memory](../../rx-storage-memory.md) |
Switching storage is a single parameter change:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
```
The OPFS storage option is worth highlighting for web applications. The [Origin Private File System](../../rx-storage-opfs.md) is a modern browser API that gives web pages access to a private file system with significantly faster read and write throughput than IndexedDB. For applications that previously experienced DataStore's startup performance problems with large local datasets, OPFS offers a meaningful improvement.
### Conflict Resolution You Own
DataStore handled conflicts on the server using Auto Merge, Optimistic Concurrency, or a Lambda function. The client had no direct role in conflict resolution.
RxDB runs conflict resolution on the client. When the pull handler returns a document that conflicts with a locally modified version, RxDB calls your conflict handler synchronously:
```ts
await db.addCollections({
posts: {
schema: postSchema,
conflictHandler: async ({ newDocumentState, realMasterState }) => {
// Example: keep whichever version was updated more recently
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For applications where users edit the same documents from multiple devices simultaneously, RxDB also supports [CRDT-based conflict resolution](../../crdt.md). CRDTs merge concurrent edits automatically and deterministically, without requiring a server-side Lambda or custom conflict handler:
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const postSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
status: { type: 'string' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDTs, two users editing the same post while offline will have their changes merged field-by-field when they reconnect, rather than one edit overwriting the other.
### Multi-Tab Support in the Browser
DataStore on the web stored data in IndexedDB per-tab. Multiple browser tabs each had their own DataStore instance, and keeping them in sync required additional subscription logic.
RxDB solves this with its [SharedWorker storage](../../rx-storage-shared-worker.md). All browser tabs share a single database instance running in a SharedWorker, so writes from any tab are immediately visible in all others with no extra code:
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
### Encryption at Rest
RxDB includes a built-in [encryption plugin](../../encryption.md) that encrypts individual document fields before they are written to the local storage. This is relevant for applications that store user data locally and need to comply with data protection requirements:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'your-encryption-passphrase'
});
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
token: { type: 'string' },
email: { type: 'string' }
},
encrypted: ['token', 'email'] // These fields are stored as ciphertext
};
```
### Full TypeScript and JSON Schema Validation
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before it is written. Invalid documents are rejected at the database level:
```ts
try {
await db.posts.insert({
id: 'post-001',
// 'title' is required but missing
status: 'published',
updatedAt: Date.now()
});
} catch (err) {
console.error(err); // Schema validation error
}
```
RxDB also generates TypeScript types from the schema automatically, giving you IDE autocompletion and type checking for all collection operations. DataStore's code generation produced TypeScript classes, but the types came from the Amplify CLI rather than a portable JSON Schema definition, making them harder to share or validate outside the Amplify toolchain.
---
## Migrating from DataStore to RxDB
Teams using DataStore in Amplify Gen 1 applications face a migration before May 2027. The migration path to RxDB involves three steps:
**1. Replace the client data model definitions.**
DataStore used GraphQL schemas processed by the Amplify CLI. RxDB uses JSON Schema defined in TypeScript:
```ts
// DataStore model (generated from GraphQL)
import { Post } from './models';
// RxDB equivalent
const postSchema = {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
status: { type: 'string', enum: ['PUBLISHED', 'DRAFT'] },
rating: { type: 'number' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'updatedAt'],
indexes: ['updatedAt', 'status']
};
```
**2. Replace DataStore reads and writes with RxDB collection operations.**
```ts
// DataStore
await DataStore.save(
new Post({ title: 'Hello', status: PostStatus.DRAFT, rating: 0 })
);
const posts = await DataStore.query(Post, c => c.status('eq', PostStatus.PUBLISHED));
// RxDB equivalent
await db.posts.insert({
id: uuid(),
title: 'Hello',
status: 'DRAFT',
rating: 0,
updatedAt: Date.now()
});
const posts = await db.posts.find({ selector: { status: 'PUBLISHED' } }).exec();
```
**3. Replace DataStore subscriptions with RxDB reactive queries.**
```ts
// DataStore
const sub = DataStore.observe(Post).subscribe(msg => {
refetchPosts(); // Manual re-query needed
});
// RxDB equivalent: result set updates automatically
db.posts.find({ selector: { status: 'PUBLISHED' } }).$.subscribe(posts => {
updateUI(posts); // posts is always current
});
```
The backend can remain AppSync during migration. Point the RxDB GraphQL replication plugin at the same AppSync endpoint and the data keeps flowing while you replace the client layer.
---
## Getting Started with RxDB
Install RxDB and RxJS:
```bash
npm install rxdb rxjs
```
Create a database with a collection and start using reactive queries:
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
addRxPlugin(RxDBDevModePlugin);
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
status: { type: 'string' },
rating: { type: 'number' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'updatedAt'],
indexes: ['updatedAt', 'status', 'rating']
}
}
});
// Write data
await db.posts.insert({
id: 'post-001',
title: 'Getting started with RxDB',
status: 'PUBLISHED',
rating: 5,
updatedAt: Date.now()
});
// Reactive query: subscribes to published posts sorted by rating
db.posts.find({
selector: { status: 'PUBLISHED' },
sort: [{ rating: 'desc' }]
}).$.subscribe(posts => {
console.log('Current published posts:', posts.map(p => p.title));
});
```
---
## Comparison Summary
| Aspect | AWS Amplify DataStore | RxDB |
|---|---|---|
| **Current status** | Deprecated (Gen 1 EOL: May 2027) | Actively maintained since 2016 |
| **Backend requirement** | AWS AppSync + DynamoDB only | Any backend or no backend |
| **Vendor lock-in** | High (AWS ecosystem) | None (open source, pluggable) |
| **Query language** | Function-based predicates (limited nesting) | Mango/MongoDB-style JSON (full $or/$and nesting) |
| **Reactive queries** | Change notifications only (no live result sets) | Full live queries via RxJS Observables |
| **Conflict resolution** | Server-side via AppSync (Auto Merge, Lambda) | Client-side configurable handler or CRDTs |
| **Sync observability** | Black box; limited error exposure | Full observable state (active$, error$, sent$, received$) |
| **Browser storage** | IndexedDB | IndexedDB, OPFS (faster) |
| **Mobile storage** | SQLite | SQLite (expo-sqlite, op-sqlite) |
| **Multi-tab support** | No (separate IndexedDB instances per tab) | SharedWorker (shared instance across tabs) |
| **Schema migration** | Amplify CLI + backend deployment | Client-side migration strategies |
| **Encryption at rest** | Not built-in | Built-in encryption plugin |
| **Schema validation** | None at runtime | JSON Schema enforced on every write |
| **TypeScript** | Generated classes from CLI | Auto-generated types from JSON Schema |
| **Local development** | Amplify mock (no real-time support) | Full functionality, memory storage for tests |
| **Framework support** | React, React Native, iOS, Android | Any JS framework + React Native + Electron |
| **License** | Apache 2.0 (client SDK) | Apache 2.0 |
---
## FAQ
Yes. RxDB's [GraphQL replication plugin](../../replication-graphql.md) can connect to any GraphQL endpoint, including AWS AppSync. You configure the pull and push query builders to match your AppSync schema, and RxDB handles the sync loop, checkpoint tracking, and conflict resolution. This means you can keep AppSync as your backend while replacing the DataStore client with RxDB.
Yes. RxDB's replication pull and push handlers are plain async functions, so you can include authentication headers (JWT, API key, Cognito tokens) in each request. The local database works without authentication; only the replication to the remote backend requires it. If a user's session expires, replication pauses and resumes once valid credentials are available again.
All reads and writes go to the local storage (IndexedDB or OPFS) first. The application works fully offline. When network connectivity is available, replication runs in the background and syncs local changes to the server. When the user goes offline again, the local database continues to work normally and RxDB queues any changes for the next sync. See the [offline-first documentation](../../offline-first.md) for details.
AWS Amplify Gen 2 does not include a DataStore replacement. AWS recommends building offline-first features manually using a local storage library and a GraphQL client like Apollo that connects directly to AppSync. RxDB fills that gap: it provides the local database and the sync engine that Gen 2 does not include.
DataStore's startup performance degrades with large local datasets because it performs a full reconciliation scan on initialization. RxDB starts by loading no data; collections are queried on demand. The [OPFS storage](../../rx-storage-opfs.md) option provides significantly faster bulk read and write throughput compared to IndexedDB, which addresses the performance issues many DataStore users experienced with growing local datasets.
---
## RxDB as a Cloudant Alternative for the JavaScript Client
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Cloudant Alternative for the JavaScript Client
Teams that adopted **Cloudant** usually picked it because they wanted CouchDB semantics without running their own cluster. The replication protocol, the JSON document model, and the ability to sync with [PouchDB](../../replication-couchdb.md) in the browser made Cloudant a popular choice for offline-capable web and mobile apps. Over time, many of those teams ran into the same set of issues on the client side: PouchDB struggles with large datasets, IBM Cloud pricing is hard to predict, and the developer experience around schemas and reactive UIs feels dated.
This page explains how **RxDB** fits as a Cloudant alternative on the JavaScript client. You can keep your existing Cloudant backend and replace only the client database, or move to a different sync target entirely.
## A Short History of Cloudant
Cloudant Inc. was founded in 2008 by former MIT physicists who wanted a distributed version of Apache CouchDB. The product was built on top of **BigCouch**, an internal fork that added clustering, sharding, and quorum reads and writes to plain CouchDB. BigCouch was later merged back into the upstream CouchDB project, which is why modern CouchDB clusters look very similar to what Cloudant offered from day one.
In 2014, **IBM acquired Cloudant** and integrated it into the IBM Cloud portfolio. The service kept the CouchDB API surface, including `_changes` feeds, MapReduce views, Mango queries, and the standard replication protocol. In 2018, the **Cloudant Shared Plan was retired** and existing customers were migrated to dedicated IBM Cloud accounts. Today, Cloudant is positioned as part of **IBM Cloud Databases**, billed mostly through provisioned throughput capacity and storage.
The protocol stayed open, which is the important part for this article: anything that can replicate with CouchDB can replicate with Cloudant.
## What is RxDB?
[RxDB](https://rxdb.info/) is a [local-first](../../articles/local-first-future.md) NoSQL database for JavaScript. It runs in the browser, in [Node.js](../../nodejs-database.md), in React Native, in Electron, and in most other JavaScript runtimes. Documents live on the client, queries run locally against indexed storage, and a [replication layer](../../replication.md) keeps the local state in sync with a remote endpoint.
Two design choices matter for Cloudant users:
- The storage layer is swappable. You can pick [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), in-memory, SQLite, and others depending on the runtime.
- The replication layer is pluggable. There is a dedicated [CouchDB replication plugin](../../replication-couchdb.md) that speaks the same protocol Cloudant uses, plus generic [HTTP replication](../../replication-http.md) for custom endpoints.
## Where Cloudant with PouchDB Falls Short on the Client
Cloudant on the server is solid. The friction shows up in the browser, where most teams pair it with PouchDB.
### PouchDB Performance with Large Datasets
PouchDB stores a full **revision tree** for every document to stay protocol-compatible with CouchDB. On top of [IndexedDB](../../slow-indexeddb.md), this design pays a cost for every read and write:
- Each document update appends to a per-document revision tree, which inflates storage size.
- Bulk inserts trigger many IndexedDB transactions because of the way revision metadata is written.
- Initial replication of tens of thousands of documents can take minutes and freeze the UI.
- Query performance degrades because secondary indexes are built on top of slow IndexedDB key ranges.
Once a collection grows past a few thousand active documents, users notice the slowdowns.
### IBM Lock-In
Even though Cloudant uses an open protocol, the operational side is tied to IBM Cloud. Identity and Access Management, billing, support tickets, and monitoring all live inside the IBM ecosystem. Moving away requires migrating accounts, IAM policies, and integration glue, not just data.
### Billing Complexity
Cloudant pricing is based on provisioned throughput capacity for reads, writes, and queries, plus storage. Spikes in client activity translate directly into throughput overruns. Teams often over-provision to be safe, which makes the bill larger than expected.
### Limited Client-Side Features
PouchDB ships a small query engine and basic change events. There is no schema validation, no typed collections, no reactive query results out of the box, and no first-class hooks for migrations or encryption. Application code has to fill those gaps.
The numbers reflect this. As of July 30, 2026, [the official Cloudant Node.js client](https://github.com/cloudant/nodejs-cloudant) has 255 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `@cloudant/cloudant` package was downloaded 15,925 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/@cloudant/cloudant-vs-rxdb)). The last commit to the [nodejs-cloudant repository](https://github.com/cloudant/nodejs-cloudant) was in January 2022, , and the repository was archived by its owner in March 2022, so it is read-only. The maintained replacement, [`@ibm-cloud/cloudant`](https://github.com/IBM/cloudant-node-sdk) (29 GitHub stars, 54,167 npm downloads in the last 30 days), is a REST SDK for the hosted service and not a client-side database, so it does not cover the offline case at all.
## Why RxDB Works as a Cloudant Alternative
RxDB keeps the parts that made Cloudant attractive and replaces the parts that hurt on the client.
### 1. Still Talks to Any CouchDB-Compatible Server
The [CouchDB replication plugin](../../replication-couchdb.md) implements the standard CouchDB replication protocol, so it works against Cloudant, Apache CouchDB, and any compatible service. You keep your existing backend, your existing documents, and your existing access control.
### 2. Faster Client Storage Options
RxDB does not force a single storage engine. For browsers you can choose:
- [IndexedDB storage](../../rx-storage-indexeddb.md) for broad compatibility.
- [OPFS storage](../../rx-storage-opfs.md) for the fastest persistent storage in modern browsers.
- [Dexie storage](../../rx-storage-dexie.md) when you want a battle-tested IndexedDB wrapper.
Because RxDB does not maintain a full CouchDB-style revision tree on disk, write throughput and initial sync are noticeably faster than PouchDB on the same hardware.
### 3. MongoDB-Style Queries
[RxQuery](../../rx-query.md) supports a Mango-like syntax with selectors, sort, skip, limit, and indexes defined at the schema level. The query planner uses your indexes directly against the underlying storage, so equality, range, and compound queries stay fast as the dataset grows.
### 4. Observable Queries
Every query and document in RxDB is [reactive](../../reactivity.md). A query returns an observable that emits a new result whenever a matching document changes, locally or through replication. UI frameworks like React, Vue, Svelte, and Angular bind to those observables directly, which removes a lot of glue code that PouchDB users normally write by hand.
### 5. Schemas, Migrations, and Plugins
[Collections](../../rx-collection.md) are defined with JSON schemas, which gives you validation, typed documents, schema versioning with migration strategies, encryption, attachments, and a long list of optional plugins. Cloudant on its own does not enforce a schema, and PouchDB does not either. Adding RxDB on the client gives you that structure without changing the backend.
## Code Sample: Replicating with a Cloudant CouchDB Endpoint
The CouchDB replication plugin points at any CouchDB-compatible URL, including a Cloudant database URL. The example below uses a basic auth token, but you can plug in IAM-issued session cookies or API keys the same way you would with PouchDB.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageDexie()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'string', format: 'date-time' }
},
required: ['id', 'title', 'done']
}
}
});
const replicationState = replicateCouchDB({
replicationIdentifier: 'cloudant-todos-replication',
collection: db.todos,
url: 'https://USER:PASSWORD@my-account.cloudantnosqldb.appdomain.cloud/todos/',
live: true,
pull: {},
push: {}
});
replicationState.error$.subscribe(err => {
console.error('Cloudant sync error', err);
});
```
The replication is bidirectional, runs continuously when `live: true`, and survives offline periods. When the client comes back online, RxDB replays local changes against Cloudant and pulls down any new revisions.
## Code Sample: Subscribing to a Query in the Browser
Once data is local, queries no longer hit the network. The same query that would cost a Cloudant read now runs against IndexedDB or OPFS and updates automatically when documents change.
```ts
const openTodos$ = db.todos.find({
selector: { done: false },
sort: [{ updatedAt: 'desc' }]
}).$;
const subscription = openTodos$.subscribe(todos => {
// Re-render the list whenever the result set changes
renderTodoList(todos);
});
// Writing a document triggers the subscription above
await db.todos.insert({
id: 'todo-1',
title: 'Try RxDB with Cloudant',
done: false,
updatedAt: new Date().toISOString()
});
```
There is no separate change feed wiring, no manual diffing, and no extra Cloudant reads.
## Keep Cloudant as the Backend, Swap the Client
You do not have to leave Cloudant to fix the client. A common migration path looks like this:
1. Keep the existing Cloudant database, indexes, and security configuration.
2. Replace PouchDB on the client with RxDB plus the [CouchDB replication plugin](../../replication-couchdb.md).
3. Pick a storage adapter that fits the target runtime, for example [OPFS](../../rx-storage-opfs.md) for desktop browsers and [IndexedDB](../../rx-storage-indexeddb.md) for older ones.
4. Define RxDB schemas that mirror your existing document shapes and add validations gradually.
5. Roll the new client out behind a feature flag so existing PouchDB users keep working until they switch over.
Because RxDB speaks the CouchDB replication protocol, the server does not know or care whether the client is PouchDB or RxDB. You can run both at the same time during the transition.
If you later decide to leave IBM Cloud entirely, you can repoint the [CouchDB replication](../../replication-couchdb.md) at a self-hosted CouchDB cluster, or switch to [generic HTTP replication](../../replication-http.md) against your own API. The client code does not change.
## FAQ
Yes. Cloudant exposes the standard CouchDB replication protocol, and RxDB ships an official [CouchDB replication plugin](../../replication-couchdb.md) that targets that protocol. Point the plugin at your Cloudant database URL and authenticate the same way you would with any other CouchDB client. Both pull and push are supported, including continuous live replication.
Cloudant is still offered as a managed service inside IBM Cloud Databases. The Cloudant Shared Plan was retired in 2018, and current deployments run on dedicated IBM Cloud capacity with throughput-based billing. The CouchDB-compatible API is still the supported way to talk to the service, so client tooling built for CouchDB keeps working against modern Cloudant.
PouchDB stores a full per-document revision tree to mirror CouchDB on disk, which adds overhead to every read and write on top of [slow IndexedDB](../../slow-indexeddb.md). RxDB separates the storage engine from the replication protocol, so the on-disk format is optimized for the client and replication metadata is kept compact. RxDB also supports faster storage backends like [OPFS](../../rx-storage-opfs.md) and uses event reduction to avoid recomputing observable queries on every change.
Install RxDB and the [CouchDB replication plugin](../../replication-couchdb.md), define schemas for your existing collections, and start replication against the same Cloudant URL you used with PouchDB. RxDB will pull the documents into local storage on first run. You can keep the old PouchDB code path during a rollout window and remove it after users have synced. No server-side changes are required.
## Comparison Table
| Capability | Cloudant + PouchDB | RxDB |
| --- | --- | --- |
| Client storage | IndexedDB via PouchDB only | IndexedDB, OPFS, Dexie, in-memory, SQLite, more |
| Replication protocol | CouchDB | CouchDB, HTTP, GraphQL, WebRTC, P2P, Firestore, others |
| Backend choice | IBM Cloud Cloudant | Cloudant, self-hosted CouchDB, custom servers |
| Query language | Mango on PouchDB | Mango-like [RxQuery](../../rx-query.md) with indexes |
| Reactive queries | Manual via change feed | Built-in observable queries |
| Schema validation | None on client | JSON schema per collection |
| Schema migrations | Manual | Built-in versioned migrations |
| Encryption | Manual | Optional plugin |
| Conflict handling | Revision-based, manual resolution | Pluggable conflict handler per collection |
| Offline-first | Yes, with PouchDB caveats | Yes, [offline-first](../../offline-first.md) by design |
| Vendor lock-in | IBM Cloud account and billing | None, replace replication target at any time |
## Follow Up
If Cloudant works for your backend but PouchDB is holding back your client, RxDB is a drop-in upgrade path. You keep the open replication protocol, switch to faster storage, and gain schemas, reactive queries, and a plugin ecosystem.
More resources:
- [RxDB Sync Engine](../../replication.md)
- [CouchDB Replication Plugin](../../replication-couchdb.md)
- [HTTP Replication](../../replication-http.md)
- [RxDB GitHub Repository](/code/)
---
## RxDB as a Couchbase Alternative - Local-First, Backend-Agnostic, and Framework-Ready
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Couchbase Alternative
Developers building offline-capable JavaScript applications often evaluate Couchbase Mobile as a data layer because of its embedded Couchbase Lite client and its Sync Gateway component for backend replication. In practice, the stack introduces significant infrastructure requirements, a proprietary synchronization protocol, and a tight coupling to the Couchbase Server ecosystem. [RxDB](https://rxdb.info) is a local-first JavaScript database that handles offline storage, reactive queries, and replication without any of that infrastructure dependency.
This page covers what Couchbase is, how its mobile and web stack works, where it creates friction for JavaScript developers, and how RxDB provides a simpler path to the same offline-first goals.
---
## What is Couchbase?
Couchbase has a long history in the NoSQL space. The company was formed in February 2011 through the merger of **Membase** (a key-value store derived from NorthScale, founded 2009) and **CouchOne** (a company built around Apache CouchDB development). The merger combined the high-performance, memcached-compatible architecture of Membase with the document-model concepts from CouchDB, producing a platform aimed at enterprise-scale, low-latency applications.
Over the years, Couchbase has expanded from a pure document database into a multi-model platform. The key additions:
- **N1QL (SQL++ query language)**: A SQL-like query language added to make Couchbase more approachable for developers familiar with relational databases.
- **Full-text search**: Integrated search engine for text queries within the document store.
- **Analytics**: Columnar analytics via the Couchbase Analytics Service.
- **Couchbase Capella**: The managed cloud Database-as-a-Service launched in 2020.
- **Vector search**: Added as part of Couchbase 8.0 (2025) for AI workloads.
- **Couchbase Lite**: An embeddable, lightweight client-side database for mobile and edge devices.
- **Sync Gateway**: The server-side component that handles synchronization between Couchbase Lite clients and Couchbase Server.
In 2025, the company was acquired by Haveli Investments in a deal valued at approximately $1.5 billion, shifting it further toward enterprise and AI-focused positioning.
### Couchbase Mobile: The Client-Side Stack
For developers building offline-capable applications, the relevant part of Couchbase is the mobile stack:
- **Couchbase Lite**: An embedded NoSQL JSON database that runs locally on iOS, Android, Windows, Linux, macOS, and (as of 2025) in browser-based JavaScript applications. It supports CRUD operations and SQL++ queries.
- **Sync Gateway**: A backend server process that sits between Couchbase Lite clients and Couchbase Server. It manages bidirectional replication, access control via "channels", conflict resolution, and authentication.
- **Capella App Services**: The managed cloud version of Sync Gateway, provided as part of the Couchbase Capella offering.
This is a capable stack for native mobile applications. For JavaScript and web development, however, it introduces a set of constraints that make it a poor fit compared to purpose-built JavaScript-native databases like RxDB.
---
## The Infrastructure Requirements of Couchbase Mobile
### The Three-Tier Dependency Chain
Using Couchbase Mobile for a JavaScript application requires:
1. **Couchbase Server** (or Couchbase Capella): The primary backend database, a cluster-based system requiring significant infrastructure and operational knowledge.
2. **Sync Gateway** (or Capella App Services): A separate server process that must be deployed, configured, versioned, and maintained alongside Couchbase Server.
3. **Couchbase Lite** (client): The embedded client library on the device.
All three tiers must be version-compatible with each other. Updating any tier requires checking compatibility matrices across the stack. Sync Gateway alone requires configuration of JSON config files, RBAC roles in Couchbase Server, network port management, TLS configuration, and a custom sync function written in JavaScript that governs how documents are routed to users.
RxDB requires none of this. It runs in the browser or in Node.js directly, stores data locally, and replicates with any backend that exposes a minimal HTTP, GraphQL, or WebSocket interface. You do not need to deploy or maintain any Couchbase-specific infrastructure.
### Sync Gateway Configuration Overhead
The Sync Gateway sync function is a JavaScript function that runs on the server and determines which documents each user can access. While flexible, it requires careful design and testing. An incorrect sync function can expose data to the wrong users or block replication entirely, and debugging it requires server-side log analysis.
A typical Sync Gateway configuration involves:
- Defining database bucket mappings between Couchbase Server and Sync Gateway
- Creating RBAC roles and users in Couchbase Server with specific permissions
- Writing a sync function that maps documents to channels and assigns channel access to users
- Configuring CORS if browser clients need to connect
- Setting up TLS and load balancing for production deployments
None of this is unreasonable for a dedicated native mobile team, but for a JavaScript application team that wants to add offline support to a web app, this is a substantial operational investment before writing a single line of application code.
---
## Couchbase Lite for JavaScript: Capabilities and Limitations
Couchbase added JavaScript/browser support to Couchbase Lite in 2025. The library stores data locally using the browser's IndexedDB and can synchronize with Sync Gateway or Capella App Services. This is a meaningful addition for web developers, but several constraints remain:
### No Multi-Tab Support
Couchbase Lite for JavaScript does not support running the same database across multiple browser tabs simultaneously. If a user opens the application in two tabs, the behavior is undefined and may produce data inconsistencies. There is no cross-tab synchronization or locking mechanism.
RxDB handles this natively with its [SharedWorker storage](../../rx-storage-shared-worker.md). All tabs share a single database instance running in a Web Worker. A write in one tab propagates to reactive queries in all other tabs automatically:
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
### No Full-Text Search in the Browser
The SQL++ `MATCH()` function for full-text search is not available in Couchbase Lite for JavaScript. Applications that require in-browser text search must implement their own solution outside of the database.
### No Peer-to-Peer Sync
Couchbase Lite for JavaScript requires a Sync Gateway or Capella App Services backend for replication. There is no peer-to-peer synchronization between browser tabs or devices without going through the server.
RxDB supports [WebRTC replication](../../replication-webrtc.md) for direct peer-to-peer sync between browser clients:
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
const replicationPool = await replicateWebRTC({
collection: db.items,
topic: 'my-collaboration-room',
connectionHandlerCreator: getConnectionHandlerSimplePeer({
signalingServerUrl: 'wss://signaling.example.com'
}),
pull: {},
push: {}
});
```
### Requires Sync Gateway Even for Simple Use Cases
There is no lightweight path to synchronization. Even a simple two-user application that needs to share a handful of documents must deploy and operate a Sync Gateway instance. The Couchbase Lite protocol is proprietary, so there is no way to write a compatible backend yourself without using Couchbase's own infrastructure.
RxDB's [HTTP replication](../../replication-http.md) works with any backend that supports two endpoints: one to fetch changed documents since a checkpoint, and one to accept pushed documents. You can implement this in any language on any infrastructure:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
replicationIdentifier: 'my-http-replication',
collection: db.items,
pull: {
async handler(checkpointOrNull, batchSize) {
const since = checkpointOrNull ? checkpointOrNull.updatedAt : 0;
const response = await fetch(
`/api/items?since=${since}&limit=${batchSize}`
);
const data = await response.json();
return {
documents: data.items,
checkpoint: data.checkpoint
};
}
},
push: {
async handler(rows) {
const response = await fetch('/api/items/bulk', {
method: 'POST',
body: JSON.stringify(rows)
});
return response.json(); // conflicts
}
},
live: true,
retryTime: 5000
});
```
---
Couchbase Lite is a native-first product, and its JavaScript footprint is small. As of July 30, 2026, [couchbase-lite-core](https://github.com/couchbase/couchbase-lite-core) has 265 GitHub stars and the biggest per-platform SDK, [couchbase-lite-ios](https://github.com/couchbase/couchbase-lite-ios), has 1,663, while [RxDB](https://github.com/pubkey/rxdb) has 23,296. On npm, the React Native binding `cbl-reactnative` was downloaded 99 times in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/cbl-reactnative-vs-rxdb)). Most Couchbase Lite installs happen through the native iOS, Android and .NET SDKs, so these numbers say nothing about the product's total install base. But they do describe the JavaScript ecosystem around it, and that ecosystem decides how many code examples, blog posts and answered questions exist for the stack you build on.
## How RxDB Approaches the Same Problems
### Local-First Storage Without Infrastructure Dependencies
[RxDB](https://rxdb.info) is a local-first JavaScript database. All reads and writes go to local storage on the device. Replication is optional and does not affect the basic functionality of the database. An application built on RxDB works fully offline without any server connection. Sync runs in the background and applies changes when connectivity is available.
This model is identical in intent to what Couchbase Mobile aims for with Couchbase Lite. The difference is the implementation path. RxDB is a JavaScript-native library that integrates directly with the JavaScript ecosystem without requiring any server-side Couchbase components.
### Pluggable Storage Backends
RxDB separates its query engine from the storage layer via the [RxStorage interface](../../rx-storage.md). You choose the storage backend that fits your platform:
| Environment | RxDB Storage Option |
|---|---|
| Browser (general use) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high throughput) | [OPFS](../../rx-storage-opfs.md) |
| React Native | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multiple browser tabs | [SharedWorker](../../rx-storage-shared-worker.md) |
| Testing | [Memory](../../rx-storage-memory.md) |
Switching storage is a single-line change in the database creation call. The rest of the application code remains identical:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
// swap for any other storage without changing application code:
// import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
// import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
```
Couchbase Lite for JavaScript only supports IndexedDB as its browser storage backend. There is no way to switch to OPFS for better performance or to SQLite for native environments without switching to a different Couchbase Lite SDK entirely.
### Reactive Queries with RxJS Observables
A key architectural difference between RxDB and Couchbase Lite is how they expose data changes to the application.
Couchbase Lite provides a change listener API that fires a callback when documents change. The application must then re-query the database to get the current state and reconcile the changes with its local state.
RxDB builds on [RxJS](https://rxjs.dev) to provide reactive queries as a first-class feature. Every query result is a live observable. When documents matching the query change (from a local write or from an incoming replication), the observable emits the updated result automatically:
```ts
// This subscription remains live and re-emits whenever matching documents change
db.items.find({
selector: { status: 'active' },
sort: [{ createdAt: 'asc' }]
}).$.subscribe(activeItems => {
// Called immediately with current results, then again on any relevant change
renderList(activeItems);
});
```
You can also subscribe to individual documents or to specific fields within a document:
```ts
const item = await db.items.findOne('item-001').exec();
// Fires only when the 'status' field changes
item.get$('status').subscribe(newStatus => {
updateStatusBadge(newStatus);
});
```
RxDB uses the [event-reduce algorithm](https://github.com/pubkey/event-reduce) to determine whether a document change affects a query's result set without re-running the full query against storage. For most write operations, the updated result is calculated from the change event itself, making reactive queries efficient even when many subscriptions are active simultaneously.
### Schema Validation and Full TypeScript Support
RxDB enforces a [JSON Schema](../../rx-schema.md) on every document before it is written to storage. Documents that do not match the schema are rejected with a typed error:
```ts
await db.addCollections({
items: {
schema: {
title: 'item schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
status: { type: 'string', enum: ['active', 'done', 'archived'] },
priority: { type: 'number', minimum: 0, maximum: 10 },
createdAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'priority', 'createdAt'],
indexes: ['createdAt', ['status', 'priority']]
}
}
});
```
RxDB also infers TypeScript types from the schema automatically. Accessing a document field gives you the exact TypeScript type without any manual type annotation:
```ts
const item = await db.items.findOne('item-001').exec();
if (item) {
// TypeScript knows: title is string, status is 'active' | 'done' | 'archived'
console.log(item.title);
console.log(item.status);
// Compile-time error if you access a field that is not in the schema
}
```
Couchbase Lite for JavaScript handles documents as untyped JSON objects. There is no schema declaration, no validation before writes, and no compile-time type checking on document access. All field access returns `any`, which removes TypeScript's ability to catch data model mismatches at development time.
### Replication with Any Backend
RxDB's replication system is built around a protocol-agnostic pull/push model. Any backend that supports a checkpoint-based sync API can work with RxDB. Built-in plugins cover the most common cases:
- **[HTTP replication](../../replication-http.md)**: Sync with any REST API via pull and push handlers
- **[GraphQL replication](../../replication-graphql.md)**: Sync with any GraphQL endpoint
- **[WebSocket replication](../../replication-websocket.md)**: Real-time bidirectional sync over WebSocket
- **[CouchDB replication](../../replication-couchdb.md)**: Sync with any CouchDB-compatible server
- **[Supabase replication](../../replication-supabase.md)**: Sync with a Supabase PostgreSQL backend
- **[Firestore replication](../../replication-firestore.md)**: Sync with Firebase Cloud Firestore
- **[WebRTC replication](../../replication-webrtc.md)**: Peer-to-peer sync between browser clients
This means that if your backend is already PostgreSQL, MongoDB, a REST API, or any other existing system, you can add RxDB on the client without changing the backend architecture. There is no requirement to deploy Couchbase infrastructure.
Couchbase Lite replicates exclusively using the Couchbase Lite replication protocol, which requires Sync Gateway (version 3.3.1+ or 4.0.1+) or Capella App Services on the server side. No custom backend implementation is possible. Migrating away from Couchbase on the backend means replacing the client-side database as well.
### Automatic Schema Migrations
When a data model changes, RxDB handles migration automatically. You increment the schema version and provide a migration strategy for each version step. The migration runs when the database opens with the new schema version:
```ts
await db.addCollections({
items: {
schema: {
version: 1, // was 0
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
status: { type: 'string', enum: ['active', 'done', 'archived'] },
priority: { type: 'number', minimum: 0, maximum: 10 },
tags: { type: 'array', items: { type: 'string' } }, // new field
createdAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'priority', 'tags', 'createdAt']
},
migrationStrategies: {
1: (oldDoc) => {
// Existing documents get an empty tags array
oldDoc.tags = [];
return oldDoc;
}
}
}
});
```
All locally stored documents from version 0 are automatically migrated to version 1 before the application starts using the collection. Couchbase Lite has no built-in migration system. Schema changes require manual update scripts executed in application code, with no guarantee that all client devices will run them in the correct order.
### Conflict Resolution
RxDB provides a configurable conflict handler per collection. When a local document and a remote document conflict during replication, the handler receives both versions and returns the winning state:
```ts
await db.addCollections({
items: {
schema: itemSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Last-write-wins based on updatedAt timestamp
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative applications where concurrent writes from multiple users should be merged automatically, RxDB supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md):
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const itemSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDTs enabled, concurrent writes are automatically merged when clients sync, with no manual conflict resolution code required for the common case.
### Encryption at Rest
RxDB includes a [field-level encryption plugin](../../encryption.md) that encrypts specific document fields before they are written to the local storage engine. The raw IndexedDB, OPFS, or SQLite contents contain ciphertext for encrypted fields:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'user-provided-passphrase'
});
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
sensitiveField: { type: 'string' }
},
encrypted: ['sensitiveField']
};
```
Without the passphrase, the raw storage contents are unreadable. This protects user data on shared or lost devices without requiring the application to implement custom encryption logic.
### Performance: OPFS Storage
For browser applications that need high write throughput, RxDB supports the [Origin Private File System (OPFS)](../../rx-storage-opfs.md) as a storage backend. OPFS gives JavaScript direct filesystem access within the browser's sandboxed origin, bypassing IndexedDB's transaction model entirely:
```ts
import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageOPFS()
});
```
OPFS provides significantly higher write throughput than IndexedDB for bulk operations because it avoids the overhead of IndexedDB transactions. Couchbase Lite for JavaScript only supports IndexedDB as its browser storage layer.
---
## Vendor Lock-In and Backend Flexibility
One of the practical differences between Couchbase Mobile and RxDB is what happens when requirements change.
With Couchbase Mobile:
- Replication only works with Sync Gateway or Capella App Services.
- Sync Gateway only works with Couchbase Server or Capella.
- Migrating the backend to a different database (PostgreSQL, MongoDB, etc.) requires also replacing the client-side library and rewriting all sync logic.
- The replication protocol between Couchbase Lite and Sync Gateway is proprietary and not documented for third-party implementation.
With RxDB:
- Replication works with any backend that exposes a compatible HTTP, GraphQL, or WebSocket endpoint.
- Changing the backend database (from PostgreSQL to MongoDB, or from a REST API to GraphQL) requires only updating the replication plugin configuration on the client.
- The client-side schema, queries, and application code remain unchanged when the backend changes.
- You can replicate with Firebase, Supabase, CouchDB, a custom REST API, or any other system using the appropriate plugin.
This backend flexibility is meaningful for long-lived applications where requirements evolve. A team that starts with Supabase and later moves to a custom backend does not need to replace their entire offline-first stack.
---
## Positioning Against Couchbase: When RxDB Fits Better
Couchbase Mobile is designed for enterprise teams building native mobile applications where the full Couchbase stack (Couchbase Server, Sync Gateway, Couchbase Lite for iOS/Android) is already established or justified by organizational requirements. The JavaScript/web support for Couchbase Lite is recent and still has constraints around multi-tab access, full-text search, and peer-to-peer sync.
RxDB is the better fit when:
- The application is built primarily in JavaScript or TypeScript for web, React Native, Electron, or Node.js.
- The team does not have existing Couchbase infrastructure and does not want to adopt it.
- The backend is PostgreSQL, MongoDB, a REST API, GraphQL, Firebase, or any non-Couchbase system.
- Reactive queries are needed (UI components that update automatically when data changes).
- Multiple browser tabs must share the same database state.
- The team wants schema validation and TypeScript type inference from the data model.
- Budget or operational capacity does not support running Couchbase Server and Sync Gateway.
---
## Comparison Table
| Feature | Couchbase Lite (JS) | RxDB |
|---|---|---|
| **Offline-first** | Yes | Yes |
| **Local storage in browser** | IndexedDB only | IndexedDB, OPFS, Memory |
| **Local storage in React Native** | Native CBL SDK (separate) | SQLite via expo-sqlite or op-sqlite |
| **Reactive queries** | Change listener callbacks | RxJS Observables (live query subscriptions) |
| **Schema validation** | No | Yes (JSON Schema) |
| **TypeScript inference** | No (untyped documents) | Full (inferred from schema) |
| **Multi-tab browser support** | Not recommended | Yes (SharedWorker storage) |
| **Replication targets** | Sync Gateway / Capella App Services only | Any backend (HTTP, GraphQL, WebSocket, CouchDB, Supabase, Firestore, WebRTC) |
| **Backend infrastructure required** | Couchbase Server + Sync Gateway | None (any existing backend works) |
| **Replication protocol** | Proprietary (Couchbase Lite protocol) | Open (HTTP, WebSocket, GraphQL) |
| **Full-text search in browser** | Not supported | Via external index or RxDB plugin |
| **Peer-to-peer sync** | Not supported in browser | Yes (WebRTC replication plugin) |
| **Conflict resolution** | Automatic (last-write-wins) | Configurable handler or CRDT plugin |
| **Schema migrations** | Manual | Automatic via versioned strategies |
| **Encryption at rest** | Enterprise feature (Couchbase EE) | Built-in field-level encryption plugin |
| **Open source** | Community Edition only | Yes (core + premium plugins) |
| **Backend flexibility** | Couchbase ecosystem only | Any backend |
| **Vendor lock-in** | High (Couchbase protocol) | None |
| **Setup complexity** | High (Server + Sync Gateway + CORS + channels) | Low (npm install, no server-side Couchbase components) |
| **JavaScript-native** | Recent addition (2025), limited features | Yes, built from the start for JavaScript |
| **Active development** | Enterprise-focused, acquired 2025 | Active, independent, commercially supported |
---
## FAQ
No. RxDB is a client-side database, not a server-side database. It runs in the browser, in React Native, in Electron, or in Node.js as an embedded database. It does not replace Couchbase Server or any other backend database. What it replaces is the client-side Couchbase Lite layer and the synchronization requirement for Sync Gateway. The backend can remain any system that exposes an API RxDB can replicate with.
No. RxDB stores all data locally and operates fully offline without any backend connection. Replication is optional and runs in the background when connectivity is available. An application can be built with RxDB that never replicates with any backend and still has full offline-first functionality.
RxDB uses a configurable conflict handler per collection. During replication, when a locally modified document conflicts with a version from the server, the conflict handler receives both document states and returns the winning state. Common strategies include last-write-wins (based on a timestamp field) and merge-based strategies (combining fields from both versions). For text or structured data that requires automatic merging, the [CRDT plugin](../../crdt.md) provides conflict-free replicated data types that merge concurrent changes without any custom handler code.
Yes. RxDB works in React Native using the [SQLite storage plugin](../../rx-storage-sqlite.md), which wraps either `expo-sqlite` or `op-sqlite`. The same schema definitions, queries, reactive subscriptions, and replication configuration that work in a browser also work in React Native. There is no separate SDK or separate configuration required.
Yes. Couchbase Lite stores data as JSON documents, and RxDB stores data as JSON documents. A migration involves reading documents from the Couchbase Lite database (using the Couchbase Lite query API), defining a matching schema in RxDB, and inserting the documents into RxDB. The primary work is defining the schema and setting up RxDB's replication to replace the Sync Gateway connection. If the backend is Couchbase Server, you would also add a new API layer (HTTP, GraphQL, or WebSocket) that RxDB can replicate with.
Couchbase has a Community Edition (open source) and an Enterprise Edition (commercial). Some features, including advanced encryption and certain enterprise security options, are only available in the Enterprise Edition. Sync Gateway has its own licensing terms. RxDB's core is open source. Advanced storage plugins like IndexedDB, OPFS, and SQLite are available under a commercial [premium license](https://rxdb.info/premium/). The premium license is a one-time purchase and does not require ongoing cloud service fees.
---
## RxDB as a CouchDB Alternative - Client-Side, Offline-First, Reactive Queries
import {Faq, FaqItem} from '@site/src/components/faq';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a CouchDB Alternative
Apache CouchDB is a well-known, server-side document database recognized for its multi-master replication protocol and HTTP-based API. Many developers searching for a "CouchDB alternative" are not looking for a different server-side database. They want something that runs on the client, inside a browser or a mobile application, with offline support and reactive queries. That is what [RxDB](https://rxdb.info) is designed to do.
This page explains what CouchDB is, what its limitations are when it comes to client-side application development, and how RxDB solves those problems while optionally using CouchDB as a backend.
---
## What is CouchDB?
Apache CouchDB is a server-side, document-oriented NoSQL database that has been in active development since 2005. It was created by Damien Katz, who published the initial design in a blog post in 2005 and open-sourced the code in 2008. The Apache Software Foundation adopted CouchDB as a top-level project in 2008.
CouchDB stores data as JSON documents inside named databases. It exposes a full HTTP REST API, meaning every operation (reads, writes, queries, replication) is performed via standard HTTP requests. Documents have a revision field (`_rev`) that tracks changes and forms the basis of its conflict detection system.
The most widely known feature of CouchDB is its **Couch Replication Protocol**: a bidirectional, incremental sync protocol that allows any number of CouchDB servers to replicate with each other, including over unreliable networks. This "multi-master" replication is designed to handle cases where nodes go offline and come back later, making it a natural fit for distributed applications.
CouchDB also provides a **changes feed** (`/_changes` endpoint) that streams a log of all document modifications. This is used by replication clients and by real-time notification systems.
### A Brief Timeline
- **2005** - Damien Katz begins development and publishes initial concepts
- **2008** - CouchDB open-sourced; joins the Apache Software Foundation
- **2010** - Version 1.0 released; widespread adoption begins
- **2012** - Cloudant (a hosted CouchDB service) grows significantly; IBM later acquires it in 2014
- **2013** - BigCouch clustering code merged into Apache CouchDB 2.0 development
- **2016** - CouchDB 2.0 released with native clustering support
- **2017** - CouchDB 2.1 released with Mango query improvements
- **2022** - CouchDB 3.x branch brings performance improvements and security updates
- **2025** - CouchDB 3.5.1 released; the project remains maintained under the Apache Software Foundation
CouchDB has a stable, dedicated user base, particularly in humanitarian and academic contexts where reliable data synchronization across disconnected field locations is critical. It is not a fast-growing technology in terms of raw market share, but it is not abandoned either. It occupies a specific niche: server-side distributed document storage with built-in replication.
### How CouchDB Replication Works
CouchDB replication is a sequence-based protocol. Each database maintains a sequence counter that increments with every document change. When two CouchDB instances replicate, the replication process reads the source's changes feed since the last known sequence, fetches each changed document, and writes it to the target. Conflicts are detected via the revision tree (`_rev` field) and stored as alternate branches. Applications must read and resolve conflicts explicitly.
The protocol was designed for server-to-server replication where both sides are CouchDB instances. This design choice has significant implications for client-side use.
---
## Why CouchDB Is Not a Client-Side Database
The most fundamental limitation of CouchDB for modern web and mobile development is that it is a **server process**. It cannot run inside a browser tab or natively inside a React Native application. To build an offline-first app using CouchDB, developers historically paired it with PouchDB, a JavaScript library that implements the Couch Replication Protocol in the browser.
This pairing introduced its own set of problems, described in detail below.
### The PouchDB Overhead Problem
PouchDB was created to bring the CouchDB protocol to the browser. To stay compatible with CouchDB replication, PouchDB must store the **entire revision tree** of every document on the client. In CouchDB, a document's revision history is a tree structure that records every version ever written, including conflicting branches. This tree is required for the protocol to detect which revisions the other side already has.
Storing the revision tree for every document causes two problems:
1. **Storage bloat**: The client stores far more data than the current document state requires.
2. **Slow queries**: IndexedDB queries must navigate around the revision storage layout, which is optimized for replication correctness, not for read performance.
RxDB was originally built on top of PouchDB. As the project grew, these limitations became clear. In [RxDB version 10.0.0](../../releases/10.0.0.md), the storage layer was fully abstracted away. RxDB no longer uses PouchDB internally. Instead, it uses a pluggable [RxStorage](../../rx-storage.md) interface that can be backed by IndexedDB, OPFS, SQLite, or any other storage engine, without the revision-tree overhead required by the CouchDB protocol.
### No Reactive Query System
CouchDB provides a changes feed at the server level, but there is no client-side reactive query system built around CouchDB. When a document changes on the server, the client receives a change event from the changes feed. It is up to the application to determine which queries are affected, re-run them, and update the UI.
Implementing this correctly requires significant custom code. Race conditions between incoming change events and in-flight queries are common sources of bugs.
### MapReduce Queries Are Predefined
CouchDB uses design documents with MapReduce views for indexed queries. These views must be defined in advance and stored on the server. Ad-hoc queries against large datasets are slow because they run against the raw document storage without an index. The Mango query interface (added in CouchDB 2.x) provides a more familiar query syntax, but it still requires explicit index creation and has significant limitations compared to the MongoDB-style query language supported by RxDB.
### No JavaScript Query Language on the Client
When using CouchDB from a browser, queries go over the network to the server. There is no local query engine. For every read, the browser sends an HTTP request and waits for a response. This means:
- Reads are subject to network latency
- The application breaks entirely when the user is offline
- There is no way to cache query results reactively
---
Both projects are under active development. As of July 30, 2026, [the CouchDB server](https://github.com/apache/couchdb) has 6,933 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## How RxDB Solves These Problems
[RxDB](https://rxdb.info) is a local-first JavaScript database designed to run on the client. All reads and writes go to local storage. Network replication happens in the background. RxDB includes a [CouchDB replication plugin](../../replication-couchdb.md) so you can use CouchDB as a backend while gaining all the client-side benefits of RxDB.
### Local-First Architecture
With RxDB, the client has a full database instance. Every query runs against local storage, so reads are fast and the application works offline by design:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'done', 'updatedAt'],
indexes: ['updatedAt']
}
}
});
// This runs against local IndexedDB. No network required.
const openTodos = await db.todos.find({
selector: { done: false },
sort: [{ updatedAt: 'desc' }]
}).exec();
```
Writes made while offline are stored locally and automatically pushed to CouchDB when the connection returns.
### Replication with CouchDB
RxDB provides a dedicated [CouchDB replication plugin](../../replication-couchdb.md). This plugin does **not** use the official CouchDB replication protocol. Instead, it uses RxDB's own sync engine on top of the CouchDB HTTP API. This design choice avoids the revision-tree overhead while still replicating correctly.
```ts
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB({
replicationIdentifier: 'my-couchdb-replication',
collection: db.todos,
url: 'http://example.com/db/todos',
live: true,
pull: {
batchSize: 60
},
push: {
batchSize: 60
}
});
// Wait for the first sync to complete
await replicationState.awaitInitialReplication();
// Monitor errors
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
```
When authentication is required, you can provide a custom `fetch` method:
```ts
import {
replicateCouchDB,
getFetchWithCouchDBAuthorization
} from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB({
replicationIdentifier: 'my-couchdb-replication',
collection: db.todos,
url: 'http://example.com/db/todos',
fetch: getFetchWithCouchDBAuthorization('myUsername', 'myPassword'),
live: true,
pull: { batchSize: 60 },
push: { batchSize: 60 }
});
```
The token can also be updated dynamically while the replication is running:
```ts
replicationState.fetch = getFetchWithCouchDBAuthorization(
'myUsername',
'newPassword'
);
```
### Benefits Over the Traditional CouchDB + PouchDB Approach
| Aspect | CouchDB + PouchDB | RxDB + CouchDB |
|---|---|---|
| **Revision tree storage** | Full tree stored on client | Only current revision stored |
| **Initial replication speed** | Slow (one HTTP request per document) | Fast (batched pull) |
| **Storage engines** | IndexedDB only (in browser) | IndexedDB, OPFS, SQLite, Memory |
| **Query language** | CouchDB map/reduce or Mango | MongoDB-style with indexes |
| **Reactive queries** | Not built-in | RxJS Observables, auto-updating |
| **TypeScript support** | Limited | Full inference from JSON Schema |
| **Multi-tab support** | Not built-in | SharedWorker storage available |
### Reactive Observable Queries
One of RxDB's most significant advantages over CouchDB (and PouchDB) is its reactive query system. Every query can be subscribed to as an RxJS Observable. The query result re-emits automatically whenever the matching documents change, whether the change came from a local write or from a remote sync:
```ts
// Subscribe to open todos, sorted by most recently updated
db.todos.find({
selector: { done: false },
sort: [{ updatedAt: 'desc' }]
}).$.subscribe(todos => {
console.log('Open todos:', todos.length);
renderTodoList(todos);
});
```
When the CouchDB replication plugin pulls a new document from the server, the observable emits the updated results automatically. No polling, no manual re-fetch, no separate state management.
RxDB uses the [event-reduce algorithm](https://github.com/pubkey/event-reduce) to determine whether a document change affects the current query result. For most changes, the updated result can be computed without re-running the query against storage. This makes reactive queries fast even when many subscriptions are active.
You can also subscribe to individual documents or specific document fields:
```ts
const doc = await db.todos.findOne('todo-001').exec();
// React to a single field changing
doc.get$('title').subscribe(newTitle => {
console.log('Title updated:', newTitle);
});
// Watch the raw change stream of the collection
db.todos.$.subscribe(changeEvent => {
console.log(changeEvent.operation, changeEvent.documentId);
});
```
### Pluggable Storage Backends
Unlike PouchDB, which is tied to IndexedDB in the browser, RxDB separates the query engine from the storage layer. You choose the storage engine based on platform and performance requirements:
| Environment | Storage Option |
|---|---|
| Browser (general use) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (write-heavy workloads) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multiple browser tabs | [SharedWorker](../../rx-storage-shared-worker.md) |
| Tests | [Memory](../../rx-storage-memory.md) |
Switching storage is a one-line change in database creation. The rest of the application (queries, replication, schema) remains unchanged:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
// Swap this import to change storage
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
// import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
// import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
```
The [OPFS storage](../../rx-storage-opfs.md) option is particularly useful for write-heavy applications. OPFS gives browsers direct access to a private file system, bypassing the IndexedDB transaction overhead. Benchmarks show OPFS significantly outperforming IndexedDB for bulk write operations.
### Multi-Tab Consistency
When a user opens a web application in multiple browser tabs, each tab has its own JavaScript process. Without coordination, writes from one tab would not appear reactively in other tabs.
RxDB solves this with the [SharedWorker storage](../../rx-storage-shared-worker.md):
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
All tabs share a single database instance running in the SharedWorker. A write from tab A appears in tab B's reactive queries immediately, with no additional coordination code.
### Conflict Resolution
CouchDB stores conflicting revisions as alternate branches in the document revision tree. Resolving conflicts requires fetching the conflicting revisions, comparing them, and posting the winner as the current revision. This process is explicit and manual.
RxDB handles conflicts during replication using a configurable conflict handler on each collection:
```ts
await db.addCollections({
todos: {
schema: todoSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Last-write-wins by timestamp
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative applications where changes from multiple users should be merged rather than discarded, RxDB supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md):
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const todoSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDTs, concurrent writes are merged deterministically when clients sync. The revision-tree approach that CouchDB uses requires manual conflict resolution code on every read path where conflicts can occur.
### Schema Validation and TypeScript
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before writing it to storage. Invalid documents are rejected before they reach storage:
```ts
try {
await db.todos.insert({
id: 'todo-002',
// 'title' is required but missing
done: false,
updatedAt: Date.now()
});
} catch (err) {
// Rejected: document does not match schema
console.error(err.message);
}
```
RxDB also infers TypeScript types from the schema automatically, giving compile-time type checking and IDE autocompletion for all collection operations:
```ts
// TypeScript knows the shape of this document
const todo = await db.todos.findOne('todo-001').exec();
if (todo) {
console.log(todo.title); // string
console.log(todo.done); // boolean
}
```
CouchDB has no client-side schema validation. Documents are freeform JSON. Enforcing a schema requires either a middleware layer on the server or manual validation code in the client application.
### Schema Migrations
When your data model changes, RxDB's [migration system](../../migration-schema.md) handles the transition automatically. You increment the version number and provide a migration strategy:
```ts
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 1, // incremented from 0
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
priority: { type: 'number' }, // new field
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'done', 'priority', 'updatedAt']
},
migrationStrategies: {
1: (oldDoc) => {
// Set a default priority for all existing documents
oldDoc.priority = 0;
return oldDoc;
}
}
}
});
```
When the database opens with the new schema version, RxDB migrates all existing local documents before the application starts. CouchDB has no client-side migration system; schema changes must be handled manually through update scripts.
### Encryption at Rest
RxDB includes a [built-in encryption plugin](../../encryption.md) for encrypting document fields before writing them to local storage:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'user-specific-passphrase'
});
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
sensitiveNote: { type: 'string' }
},
encrypted: ['sensitiveNote']
};
```
Fields marked as `encrypted` are stored as ciphertext in IndexedDB. Without the passphrase, the raw storage is unreadable.
---
## RxDB and CouchDB Together
RxDB and CouchDB are not mutually exclusive. CouchDB is a capable server-side database with robust replication features. RxDB is a capable client-side database with a reactive query system. They work well as a stack:
- **CouchDB** runs on the server: stores documents, exposes the changes feed, handles server-side replication between nodes
- **RxDB** runs on the client: stores documents locally, queries without network, syncs with CouchDB in the background
```
[User Device]
RxDB (IndexedDB / SQLite)
|
| CouchDB Replication Plugin
| (HTTP pull/push + changes feed)
|
[Your Server]
CouchDB
|
| CouchDB-to-CouchDB replication (optional)
|
[Other Servers or Clients]
```
This architecture gives you offline-capable clients with reactive queries and the battle-tested CouchDB replication protocol at the server tier.
You can also switch away from CouchDB later without changing any application code. RxDB supports [HTTP replication](../../replication-http.md), [GraphQL replication](../../replication-graphql.md), [Supabase replication](../../replication-supabase.md), [WebSocket replication](../../replication-websocket.md), and [WebRTC peer-to-peer replication](../../replication-webrtc.md). The application logic that works against the local RxDB collection does not change when the backend changes.
---
## CouchDB Connection Limit in Browsers
One practical limitation of using CouchDB directly from a browser (via PouchDB or the CouchDB replication plugin) is that CouchDB uses HTTP long polling for its changes feed. Browsers limit the number of concurrent HTTP/1.1 connections to the same host to six. This means a maximum of six active CouchDB sync connections per tab.
If your application needs more than six synchronized collections, solutions include:
- Using a single CouchDB database with a `type` field per document to combine collections
- Using multiple subdomains with at most six active connections each
- Placing a proxy like nginx or HAProxy in front of CouchDB and enabling HTTP/2, which multiplexes requests over a single connection
Example nginx configuration:
```
server {
http2 on;
location /db {
rewrite /db/(.*) /$1 break;
proxy_pass http://127.0.0.1:5984;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded;
proxy_set_header Connection "keep_alive";
}
}
```
---
## Comparison Summary
| Aspect | CouchDB (alone) | RxDB + CouchDB |
|---|---|---|
| **Runs on client** | No (server only) | Yes (browser, React Native, Electron) |
| **Offline-first** | No (server goes offline, clients break) | Yes (all reads/writes local) |
| **Reactive queries** | Not built-in | RxJS Observables, auto-updating |
| **Query language** | MapReduce views or Mango | MongoDB-style with indexes |
| **Storage on client** | IndexedDB via PouchDB (with revision tree overhead) | IndexedDB, OPFS, SQLite (no revision tree) |
| **Initial sync speed** | Slow (one request per document) | Fast (batched) |
| **Multi-tab consistency** | Not built-in | SharedWorker storage available |
| **Conflict handling** | Revision tree branches (manual resolution) | Configurable handler or CRDT plugin |
| **Schema validation** | None client-side | JSON Schema enforced on every write |
| **TypeScript support** | None built-in | Inferred types from JSON Schema |
| **Encryption at rest** | None client-side | Per-field encryption plugin |
| **Schema migrations** | Manual | Automatic via versioned strategies |
| **Backend flexibility** | CouchDB protocol only | CouchDB, Supabase, GraphQL, HTTP, WebRTC |
---
## FAQ
No. RxDB is a client-side database. It runs in the browser or on a mobile device. CouchDB is a server-side database. They serve different roles. RxDB can sync with CouchDB using the [CouchDB replication plugin](../../replication-couchdb.md), making them complementary parts of an offline-first application stack.
For initial replication and local queries, yes. PouchDB must store the full document revision tree on the client to stay compatible with the CouchDB replication protocol. RxDB's CouchDB plugin uses a different sync approach that only stores the current document version, reducing storage usage and speeding up reads and initial sync. RxDB also supports storage engines like OPFS that are significantly faster than IndexedDB for write-heavy workloads.
Yes. CouchDB is one of many backends RxDB can replicate with. You can use RxDB with a custom HTTP endpoint, a GraphQL server, Supabase, or no backend at all. The [replication protocol](../../replication.md) is designed to be backend-agnostic. If you already have a CouchDB server, the CouchDB replication plugin is a straightforward way to add offline-first capabilities to your client application.
CouchDB stores all conflicting revisions as branches in the document revision tree. Reading a conflicted document requires fetching the winning and losing revisions, comparing them, and resolving the conflict by deleting the unwanted branch. This logic runs on the application side after the conflict is detected.
RxDB resolves conflicts during replication. When a push is rejected because the server has a newer version, the conflict handler on the collection is called. You define the resolution strategy (last-write-wins, field merge, server-wins, etc.) once per collection. For complex collaborative scenarios, the [CRDT plugin](../../crdt.md) can merge changes from multiple clients automatically.
Yes. The CouchDB replication plugin takes a URL pointing to your CouchDB database. It works with any CouchDB-compatible endpoint, whether hosted on your own server, in a Docker container, or in the cloud. Authentication is handled via a custom `fetch` method that you can configure per-request.
Yes. Your application code reads and writes against the local RxDB collection. The replication configuration is separate. If you replace CouchDB with a different backend, you change only the replication plugin configuration. The schema, queries, and UI code remain unchanged.
---
## RxDB as a Dexie.js Alternative with Mango Queries and Replication
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Dexie.js Alternative with Mango Queries and Replication
Teams reach for [Dexie.js](https://dexie.org/) when they want a friendlier API on top of IndexedDB without giving up the speed of a native browser store. Dexie does a fine job at that single goal. The trouble starts when an app outgrows simple key range lookups and needs MongoDB-style queries, [strict schemas](../../rx-schema.md), [reactive results](../../reactivity.md) across tabs, or [replication with a backend](../../replication.md). At that point, most projects either build those features from scratch on top of Dexie or move to a database that already provides them. RxDB sits in the second category and can even run on top of Dexie internally through the [Dexie RxStorage](../../rx-storage-dexie.md).
## A Short History of Dexie.js
Dexie.js was started in 2014 by David Fahlander as a thin wrapper around IndexedDB. The IndexedDB API itself is verbose, callback-heavy, and easy to misuse, so a higher level library filled an obvious gap. Over the years Dexie grew batched transactions, a query builder, hooks, and a small reactivity layer. It became the de-facto IndexedDB library in the JavaScript ecosystem and is used in production by WhatsApp Web, Microsoft To Do, and GitHub Desktop, among many others.
Later, the maintainers added Dexie Cloud, a paid service that provides sync, authentication, and access control on top of Dexie databases. Dexie Cloud filled a real need, since IndexedDB has no built-in replication, but it ties the sync layer to a single hosted backend. For apps that want to own their server, use an existing API, or sync peer-to-peer, Dexie Cloud is not always a good fit.
## What is RxDB
RxDB (Reactive Database) is a [local-first](../../articles/local-first-future.md) NoSQL database for JavaScript. It runs in the browser, in Node.js, in React Native, in Electron, and in any other JavaScript runtime. Data is stored locally first and then replicated to one or many backends through a generic [replication protocol](../../replication.md). Queries return [observables](../../reactivity.md) that update whenever the underlying data changes, including changes made in other browser tabs.
RxDB is storage-agnostic. It can run on [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), in-memory, SQLite, and on top of [Dexie.js](../../rx-storage-dexie.md) itself. Picking the Dexie storage means you keep all the work Dexie does inside IndexedDB while gaining schemas, queries, replication, and reactivity from RxDB on top.
## Where Dexie.js Stops Short
Dexie is a small library with a clear scope. That scope leaves a few gaps once an application becomes non-trivial.
### Queries Limited to Index Ranges
Dexie queries are built around IndexedDB cursors and key ranges. Filtering by anything other than an indexed field requires a full scan with `.filter(fn)`, which loads documents into memory and runs a JavaScript predicate on each one. Multi-field conditions, `$or` branches, and nested property matches need manual composition or hand-written index keys. Anyone who has tried to express a query like "all open tasks assigned to user X with priority above 3, sorted by due date" against Dexie knows how much glue code is involved.
### No MongoDB-Style Operators
Operators such as `$gt`, `$in`, `$elemMatch`, `$regex`, and `$or` are not part of Dexie. The query builder covers `equals`, `above`, `below`, `between`, `startsWith`, and a handful of variants. For richer matching the application has to fall back to in-memory filtering, which throws away the index advantage.
### Loose Schemas
Dexie stores arbitrary JavaScript objects. Field types, required keys, and value ranges are not validated. A typo in a field name or a wrong type from an external API silently lands in the database and surfaces later as a runtime bug. There is no built-in migration framework either, so schema changes are handled with version callbacks that the developer writes by hand.
### Replication Is a Paid Add-On
Plain Dexie has no replication. The official answer is Dexie Cloud, a hosted service with its own pricing and its own server. There is no built-in HTTP, GraphQL, CouchDB, or WebRTC replication. Bringing your own backend means writing the sync engine yourself, including change tracking, checkpointing, retry logic, and conflict detection.
### No Built-In CRDT or Conflict Handler
When two tabs or two devices write to the same document, Dexie itself does not resolve the conflict. The application code has to detect it, decide which write wins, and apply the result. There is no [CRDT plugin](../../crdt.md) and no pluggable [conflict handler](../../transactions-conflicts-revisions.md). For a single-user, single-device app this rarely matters. For collaborative or multi-device apps it becomes the central problem.
Both projects are under active development. As of July 30, 2026, [Dexie.js](https://github.com/dexie/Dexie.js) has 14,507 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## What RxDB Adds on Top
RxDB was designed around the gaps above.
### MongoDB-Style Queries
[RxQuery](../../rx-query.md) accepts the full Mango query syntax. Operators like `$gt`, `$lt`, `$in`, `$or`, `$and`, `$regex`, and `$elemMatch` are first-class. The query planner picks an index automatically and falls back to a scan only when no index fits.
### JSON Schema Validation
Every [RxCollection](../../rx-collection.md) is defined by a [JSON Schema](../../rx-schema.md). Inserts and updates are validated against the schema, primary keys are enforced, and indexes are declared once in the schema instead of being scattered across migration callbacks. Schema versions and migration strategies are part of the API.
### Observable Queries and Multi-Tab Sync
Query results are RxJS observables. When a document changes, every subscriber receives the new result set. The same mechanism works across browser tabs through a leader election protocol, so a write in tab A immediately updates a list rendered in tab B without extra code.
### Replication Primitives for Any Backend
RxDB ships replication plugins for [HTTP](../../replication-http.md), [GraphQL](../../replication-graphql.md), [CouchDB](../../replication-couchdb.md), WebRTC, Firestore, NATS, and more. All of them are built on the same generic [replication protocol](../../replication.md), so a custom backend only needs to implement a pull and a push handler. There is no required hosted service and no per-document fee.
### CRDT and Custom Conflict Handlers
For collaborative workloads RxDB provides a [CRDT plugin](../../crdt.md) and a pluggable [conflict handler](../../transactions-conflicts-revisions.md) per collection. Conflicts are detected by revision, passed to the handler, and resolved deterministically on every device.
## Code Sample: Dexie Query vs RxDB Query
A query like "open tasks with priority above 3 or tagged as urgent" is awkward in Dexie because it mixes a range condition with an `$or` branch on a different field.
Dexie:
```ts
import Dexie from 'dexie';
const db = new Dexie('tasks-db');
db.version(1).stores({
tasks: '++id, done, priority, tag'
});
const result = await db.tasks
.where('done').equals(0)
.and(task => task.priority > 3 || task.tag === 'urgent')
.toArray();
```
The `and(fn)` part runs in JavaScript over every non-done task, so the `priority` index is not used.
RxDB:
```ts
const result = await db.tasks.find({
selector: {
done: false,
$or: [
{ priority: { $gt: 3 } },
{ tag: 'urgent' }
]
}
}).exec();
```
The query planner inspects the selector, picks an index, and returns the matching documents. The same query can be observed:
```ts
db.tasks.find({
selector: {
done: false,
$or: [{ priority: { $gt: 3 } }, { tag: 'urgent' }]
}
}).$.subscribe(docs => render(docs));
```
## Code Sample: RxDB on Dexie Storage with HTTP Replication
The Dexie [RxStorage](../../rx-storage-dexie.md) lets RxDB use Dexie under the hood. The application code stays on the RxDB API and gains queries, schemas, and replication.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import { replicateRxCollection } from 'rxdb/plugins/replication';
const db = await createRxDatabase({
name: 'tasksdb',
storage: getRxStorageDexie(),
multiInstance: true,
eventReduce: true
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
priority: { type: 'number' },
tag: { type: 'string', maxLength: 50 },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'done', 'updatedAt'],
indexes: ['priority', 'tag', 'updatedAt']
}
}
});
replicateRxCollection({
collection: db.tasks,
replicationIdentifier: 'tasks-http',
live: true,
pull: {
handler: async (checkpoint, batchSize) => {
const checkpointQuery = encodeURIComponent(
JSON.stringify(checkpoint || {})
);
const url =
`https://api.example.com/tasks/pull?checkpoint=${checkpointQuery}` +
`&limit=${batchSize}`;
const res = await fetch(url);
return await res.json();
}
},
push: {
handler: async (changeRows) => {
const res = await fetch('https://api.example.com/tasks/push', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(changeRows)
});
return await res.json();
}
}
});
```
This setup keeps Dexie as the storage engine, so writes still hit IndexedDB through Dexie. RxDB adds the schema, the Mango query layer, the observables, the multi-tab support, and the HTTP replication.
## Use Dexie Inside RxDB
The [Dexie RxStorage](../../rx-storage-dexie.md) wraps Dexie as a storage backend for RxDB. It is a good pick when:
- The team already trusts Dexie and wants to keep it as the IndexedDB layer.
- The browser is the main target and IndexedDB is the most compatible option.
- The app needs schemas, observable queries, and replication that Dexie alone does not provide.
The migration path is small. Documents stored by Dexie use the same IndexedDB databases that the Dexie RxStorage reads, but the RxDB layer adds its own metadata for revisions and replication checkpoints. Most projects copy the data from the legacy Dexie database into a new RxCollection on first start. For larger datasets see the notes on [slow IndexedDB writes](../../slow-indexeddb.md) which apply to both Dexie and the Dexie RxStorage.
## FAQ
Yes. RxDB ships the [Dexie RxStorage](../../rx-storage-dexie.md) which uses Dexie.js as the underlying engine. Pass `getRxStorageDexie()` as the `storage` option when creating the database and Dexie handles the IndexedDB calls while RxDB provides schemas, queries, reactivity, and replication on top.
The core RxDB library and the open source storages, including the Dexie RxStorage, are free under the Apache 2.0 license. Some advanced plugins are part of the [Premium](/premium/) package for commercial projects. The [replication protocol](../../replication.md) itself is open source, so syncing with your own backend never requires a paid service.
Use RxDB on top of an IndexedDB-based storage like the [Dexie RxStorage](../../rx-storage-dexie.md), the [IndexedDB RxStorage](../../rx-storage-indexeddb.md), or [OPFS](../../rx-storage-opfs.md). RxDB exposes a Mango query API with operators like `$gt`, `$in`, `$or`, and `$elemMatch`, plans the query against the declared indexes, and returns either a snapshot or a live observable.
RxDB on the Dexie storage adds a thin layer over Dexie, so raw single-document reads are close to plain Dexie. For multi-condition queries RxDB is often faster because the query planner uses indexes that a hand-written Dexie query would skip. For very large bulk inserts the underlying IndexedDB is the bottleneck for both libraries, see [slow IndexedDB](../../slow-indexeddb.md).
Yes. The common pattern is to keep the existing Dexie database read-only on first start, create an RxCollection with a matching schema, and copy the documents over in batches. After the copy step the app talks to RxDB only. Because the Dexie RxStorage also uses IndexedDB, the data stays inside the same browser storage area.
## Comparison Table
| Feature | Dexie.js | RxDB |
| --- | --- | --- |
| Underlying storage | IndexedDB only | Dexie, IndexedDB, OPFS, SQLite, memory, more |
| Query language | Index ranges plus JS filter | MongoDB-style Mango queries |
| Operators like `$gt`, `$or`, `$in` | Manual JS filter | Built in |
| Schema validation | None | JSON Schema per collection |
| Schema migrations | Manual version callbacks | Declarative migration strategies |
| Observable queries | Limited via liveQuery | First-class RxJS observables |
| Multi-tab sync | Manual | Leader election built in |
| Replication with custom backend | Not included | HTTP, GraphQL, CouchDB, WebRTC, more |
| Hosted sync option | Dexie Cloud (paid) | Optional, any backend works |
| CRDT support | None | [CRDT plugin](../../crdt.md) |
| Conflict handler | Application code | Pluggable per collection |
| Runtime targets | Browser only | Browser, Node.js, React Native, Electron |
| License | Apache 2.0 | Apache 2.0 (Premium plugins separate) |
## Follow Up
If Dexie covers the current requirements, it is a solid choice for IndexedDB access. Once the app needs Mango queries, JSON Schema, replication with an arbitrary backend, or deterministic conflict resolution, RxDB fills those gaps and can still keep Dexie as the storage engine through the [Dexie RxStorage](../../rx-storage-dexie.md).
More resources:
- [RxDB Replication Engine](../../replication.md)
- [RxQuery and Mango Selectors](../../rx-query.md)
- [Reactivity in RxDB](../../reactivity.md)
- [Dexie RxStorage](../../rx-storage-dexie.md)
- [Local-First Future](../../articles/local-first-future.md)
- [RxDB GitHub Repository](/code/)
---
## RxDB as an ElectricSQL Alternative for Local-First JavaScript Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as an ElectricSQL Alternative for Local-First JavaScript Apps
[ElectricSQL](https://electric-sql.com/) is in the middle of a major rewrite. The original prototype combined SQLite, Postgres, and CRDT-based bidirectional sync. The new direction, often called Electric Next, drops most of that and focuses on partial sync of "shapes" from a Postgres source database to TypeScript or Elixir clients. The write path is not yet implemented and client-side reactivity is incomplete. Teams that want to ship a [local-first](../../offline-first.md) JavaScript application today need a stable alternative that already supports reads, writes, queries, and live updates.
[RxDB](https://rxdb.info/) is a [local-first](../../articles/local-first-future.md) NoSQL database for JavaScript that has been in production since 2016. It runs in the browser, in Node.js, in Electron, and in React Native. It ships a full bidirectional [replication protocol](../../replication.md), [reactive queries](../../reactivity.md), and pluggable storages including SQLite, IndexedDB, OPFS, and in-memory. This page explains what ElectricSQL offers today, where it falls short, and how RxDB fills the same role with fewer surprises.
## A Short History of ElectricSQL
The first version of ElectricSQL targeted a specific architecture. Each client embedded SQLite. A backend process replicated rows from Postgres into those SQLite databases and back, using CRDT-based merge logic to resolve conflicts. The pitch was a SQL database on the client that stayed in sync with a SQL database on the server.
In 2024 the team announced a rewrite. The new branch, Electric Next, narrows the scope significantly:
- Sync is built around "shapes". A shape is a filtered subset of a Postgres table that the client subscribes to. Internally this looks closer to a document store than a relational system, since the client receives JSON rows over an HTTP stream.
- The backend is written in Elixir and runs as a service in front of Postgres.
- The client library is TypeScript and JavaScript, with an Elixir client also in scope.
- The write path is not yet implemented in the new architecture. Clients can read shapes but cannot send changes back through the same protocol.
- Client-side reactivity, the ability to subscribe to a query and receive updates as the local data changes, is not yet feature complete.
The shape-based read model is interesting for some workloads, but it is a partial product. Most apps need both reads and writes, and most apps need observable queries that the UI can bind to.
## What is RxDB?
RxDB is a JavaScript database that stores data on the client and syncs with any backend you choose. Documents are validated against a JSON schema, queries follow a MongoDB-style syntax, and every query can be observed as an [RxJS observable](../../reactivity.md). The same code runs in browsers, Node.js, Electron, React Native, Capacitor, and other JavaScript runtimes.
The storage layer is pluggable. You can use [IndexedDB](../../rx-storage-indexeddb.md), OPFS, an in-memory store, or a [SQLite-backed storage](../../rx-storage-sqlite.md) when you want a SQL engine under the hood. The replication layer is also pluggable. RxDB ships handlers for [generic HTTP/REST](../../replication-http.md), GraphQL, CouchDB, WebRTC, Firestore, NATS, and others. Because the [replication protocol](../../replication.md) is documented and minimal, you can implement it on top of any backend, including a Postgres database fronted by a small REST or HTTP service.
## ElectricSQL Limitations Today
These are the practical issues a team hits when evaluating ElectricSQL Next for a production app.
### 1. The Rewrite is in Flight
The product is being redesigned in public. Documentation, APIs, and feature scope keep shifting. Building on a moving target is risky for any application that needs a stable contract over the next few years.
### 2. No Write Path
Electric Next streams shapes from Postgres to the client. It does not provide a built-in mechanism to push local writes back through the same channel. Teams have to build their own write API on the side and reconcile it with the shape stream. This is the core feature most local-first apps need, and it is missing.
### 3. Incomplete Client Reactivity
A local-first app usually binds the UI to live query results. When the underlying data changes, the view updates. Electric Next does not yet offer a complete reactive query layer on the client. You receive shape updates, but wiring them into queries with filters, sorts, and joins is left to the application.
### 4. Elixir Backend Dependency
The sync service is written in Elixir and runs as its own process in front of Postgres. Teams that do not already operate Elixir services take on a new runtime, new deployment story, and new monitoring surface. For shops standardized on Node.js, Go, Python, or Rust, this is real overhead.
### 5. Postgres-Centric
ElectricSQL assumes Postgres is the source of truth. If your backend uses MongoDB, MySQL, DynamoDB, a custom service, or a mix of stores, ElectricSQL is the wrong fit. The shape model is tied to Postgres replication internals.
Both projects are under active development. As of July 30, 2026, [ElectricSQL](https://github.com/electric-sql/electric) has 10,286 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## RxDB Advantages
### 1. Stable and Production Tested
RxDB has been published since 2016 and is used in production across web, desktop, and mobile apps. The API surface, schema model, and replication protocol are stable.
### 2. Full Read and Write Replication
The [RxDB replication protocol](../../replication.md) handles pull, push, and live updates. Writes made on the client flow back to the server through the push handler, with conflict detection based on document revisions. This works out of the box, not as a future roadmap item.
### 3. Multiple Storage Engines
You pick the storage that fits the runtime. IndexedDB and OPFS for browsers, [SQLite](../../rx-storage-sqlite.md) for Node.js, Electron, React Native, and Capacitor, and in-memory for tests. The collection and query API stay the same across all of them.
### 4. MongoDB-Style Queries
RxDB queries use the [Mango query syntax](../../rx-query.md). You can filter, sort, limit, and skip without writing SQL, and the query planner picks indexes you defined in the schema.
### 5. Observable Queries
Every query is an observable. Subscribe to it once and the subscription emits new results whenever matching data changes, locally or via replication. UI bindings for React, Vue, Svelte, Angular, and Solid are documented in the [reactivity guide](../../reactivity.md).
### 6. Bring Your Own Backend
RxDB does not require a specific backend. You can sync against Postgres through a REST or HTTP service, against MongoDB, against a GraphQL gateway, against CouchDB, or against peer clients over WebRTC. The [HTTP replication guide](../../replication-http.md) shows the standard pattern.
## Code Sample: Defining a Collection
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'shop',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
products: {
schema: {
title: 'product schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
price: { type: 'number' },
updatedAt: { type: 'number' }
},
required: ['id', 'name', 'price', 'updatedAt'],
indexes: ['updatedAt']
}
}
});
// Observable query that updates on every change
const subscription = db.products
.find({ selector: { price: { $lt: 100 } }, sort: [{ updatedAt: 'desc' }] })
.$.subscribe(results => {
console.log('current cheap products:', results.length);
});
```
See the [RxCollection guide](../../rx-collection.md) for the full collection API.
## Code Sample: HTTP Replication Against a Postgres Backend
The example below replicates an RxDB collection with a REST endpoint that reads from and writes to a Postgres database. The server side is any framework you already use, Express, Fastify, NestJS, Hono, or anything that can speak HTTP and run a SQL query.
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
collection: db.products,
replicationIdentifier: 'products-postgres-rest',
live: true,
pull: {
handler: async (checkpoint, batchSize) => {
const updatedAt = checkpoint ? checkpoint.updatedAt : 0;
const id = checkpoint ? checkpoint.id : '';
const url = `https://api.example.com/products/pull` +
`?updatedAt=${updatedAt}&id=${encodeURIComponent(id)}&limit=${batchSize}`;
const res = await fetch(url);
const data = await res.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
handler: async (changeRows) => {
const res = await fetch('https://api.example.com/products/push', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(changeRows)
});
const conflicts = await res.json();
return conflicts;
}
}
});
replicationState.error$.subscribe(err => console.error('sync error', err));
```
The pull handler returns documents newer than the checkpoint and a new checkpoint for the next batch. The push handler sends local writes to the server and returns any conflicting server documents. The full contract is described in the [HTTP replication docs](../../replication-http.md).
## Use Postgres as the Source of Truth
The standard pattern for replacing ElectricSQL with RxDB looks like this.
1. Keep Postgres as the canonical store on the server.
2. Add an `updated_at` column and a stable primary key on every table you want to sync.
3. Expose two HTTP endpoints per synced table, one for pull and one for push.
4. The pull endpoint accepts a checkpoint with `updated_at` and `id`, and returns rows ordered by `updated_at, id` with a new checkpoint.
5. The push endpoint accepts a batch of change rows, applies them inside a transaction, and returns rows that lost a conflict so the client can reconcile.
6. Add an event stream, server-sent events or a WebSocket, that emits a notification when any row in a table changes. The RxDB replication uses that signal to trigger a new pull. This is the live channel described in the [realtime database article](../../articles/realtime-database.md).
This setup mirrors what ElectricSQL provides on the read side, adds the write path that ElectricSQL Next is missing, and runs on whatever language your team already uses. There is no Elixir service to operate.
## FAQ
The original ElectricSQL is no longer the active product. Electric Next is under active development, the write path is not implemented, and client reactivity is incomplete. For a production deployment that needs both reads and writes today, it is too early.
Yes. Expose a small pull and push HTTP API in front of Postgres and use the [HTTP replication plugin](../../replication-http.md). The server can be Node.js, Go, Rust, Python, or anything else that speaks HTTP and SQL. RxDB does not require a specific backend runtime.
RxDB can use SQLite as a storage backend through the [SQLite RxStorage](../../rx-storage-sqlite.md). It also supports IndexedDB, OPFS, in-memory, and other engines. The choice is per database, and the rest of the API stays the same.
RxDB supports partial replication. The pull handler can filter on the server side based on user, tenant, region, or any other dimension. You can also run multiple replications per collection with different filters, which gives the same outcome as subscribing to several shapes.
In theory yes. You could let ElectricSQL stream shapes into a service that then feeds an RxDB pull endpoint. In practice this adds two systems to maintain. Most teams pick one. If RxDB covers the read and write path on its own, the simpler choice is to drop the extra layer.
## Comparison Table
| Feature | ElectricSQL Next | RxDB |
| --- | --- | --- |
| Status | Rewrite in progress | Stable since 2016 |
| Read sync | Yes, via shapes | Yes, via pull handler |
| Write sync | Not yet implemented | Yes, via push handler |
| Client reactivity | Incomplete | Observable queries on every collection |
| Backend runtime | Elixir service in front of Postgres | Any HTTP server, any language |
| Source database | Postgres only | Postgres, MongoDB, MySQL, CouchDB, custom, P2P |
| Client storage | Internal, JSON over HTTP | IndexedDB, OPFS, SQLite, in-memory, more |
| Query language | Shape filters | MongoDB-style Mango queries |
| Conflict handling | Application defined | Pluggable conflict handler with revisions |
| Mobile support | Limited | React Native, Capacitor, Expo, Electron |
| Offline-first | Read-only today | Full offline reads and writes |
## Next Steps
If you were waiting for ElectricSQL Next to ship a complete read and write path with reactive queries, RxDB already covers that ground. Start with the [RxDB replication guide](../../replication.md), wire up an [HTTP replication](../../replication-http.md) against your Postgres backend, and bind your UI to [observable queries](../../reactivity.md).
More resources:
- [RxDB Sync Engine](../../replication.md)
- [HTTP Replication](../../replication-http.md)
- [RxQuery](../../rx-query.md)
- [Reactivity](../../reactivity.md)
- [SQLite RxStorage](../../rx-storage-sqlite.md)
- [Local-First Future](../../articles/local-first-future.md)
- [Realtime Database](../../articles/realtime-database.md)
---
## RxDB as a GUN (gundb) Alternative for JavaScript Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a GUN (gundb) Alternative for JavaScript Apps
Developers who reach for [GUN](https://gun.eco/) usually want one thing: a JavaScript database that syncs data peer-to-peer without depending on a central backend. GUN delivers on that promise, and it pairs the graph model with extras like the SEA module for cryptography and authentication. The trouble starts once you move past the first demo. Getting basic features running often takes days of trial and error, the schema story is informal, and the source code is dense enough that tracking down a sync bug can stall a project for a week.
This guide walks through where GUN came from, where it falls short for production JavaScript apps, and how [RxDB](https://rxdb.info/) covers the same [offline-first](../../offline-first.md) and peer-to-peer use cases with a typed API, JSON Schema validation, and well-documented [replication](../../replication.md) plugins.
## A Short History of GUN
GUN was started around 2014 by Mark Nadal as an experiment in building a fully decentralized graph database for the web. The library is dual licensed under ZLIB and Apache 2.0 and ships as a small JavaScript module that runs in browsers, Node.js, and React Native. Peers connect through WebSocket relays or WebRTC and exchange small graph deltas, which the library merges using a conflict resolution scheme based on a Hypothetical Amnesia Machine algorithm.
On top of the core graph, the project ships SEA (Security, Encryption, Authorization), a module that adds public key identities, signed updates, and end-to-end encryption. The community around GUN has stayed active on GitHub and Discord, with a steady stream of issues and a smaller pool of regular contributors than larger database projects. Maintenance is concentrated around a single primary author, which is part of why some long standing issues stay open for a long time.
## What is RxDB?
[RxDB](https://rxdb.info/) (Reactive Database) is a [local-first](../../articles/local-first-future.md) NoSQL database for JavaScript. It runs in the browser, in Node.js, in Electron, and in React Native, persists data through a pluggable storage layer, and exposes documents and queries as RxJS observables for [reactivity](../../reactivity.md). The query language follows the MongoDB style and validates documents against [JSON Schema](../../rx-schema.md). Replication is handled by a small generic protocol that already has plugins for HTTP, GraphQL, CouchDB, Firestore, [WebRTC](../../replication-webrtc.md).
## Where GUN Falls Short
GUN solves a hard problem and gets a lot right at the protocol level. The pain points show up once an application grows beyond a small prototype.
### Hard to Debug Source Code
The core source files use terse variable names, heavy use of nested callbacks, and unconventional control flow. When sync breaks or a write does not propagate, stepping through the code to find the cause is slow even for experienced JavaScript developers. Stack traces often point at internal callbacks rather than user code, which makes issue reports hard to write and harder to fix.
### Opaque CRDT Internals
GUN merges concurrent writes using its own algorithm rather than a documented CRDT family like LWW-Element-Set or RGA. The behavior is deterministic in many cases, but the rules around tombstones, deletion, and graph traversal are not described in a way that maps cleanly onto a formal model. Teams that need to reason about merge outcomes for compliance or correctness checks end up reading source code instead of specifications.
### Weak Schema and Types
Documents in GUN are loose JSON graphs with no enforced shape. There is no schema validation, no required fields, and no migration tooling. A typo in a property name silently writes a new field rather than failing fast. For larger codebases this turns into shape drift across clients and versions.
### Weak Query Language
GUN exposes a chainable graph traversal API. It works for fetching nodes by key and walking edges, but it does not support range queries, sorting, compound indexes, or aggregation. Anything that resembles a SQL `WHERE` with multiple conditions has to be implemented by hand on top of `.map()` and manual filtering.
### Limited Tooling
There is no official devtools panel, no schema explorer, and no migration runner. Logging is verbose by default and hard to filter. Test setups for sync code usually involve spinning up real relay peers, which slows feedback loops.
### No First-Class TypeScript Story
GUN ships informal type definitions through community packages. The graph traversal API is dynamic enough that type inference rarely catches mistakes. Developers used to typed end-to-end pipelines lose that safety net the moment they touch GUN code.
The numbers reflect this. As of July 30, 2026, [GUN](https://github.com/amark/gun) has 19,086 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `gun` package was downloaded 211,905 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/gun-vs-rxdb)).
## Where RxDB Helps
RxDB targets the same set of use cases (offline reads, real time updates, peer-to-peer sync) and addresses the points above directly.
- **Typed API**: All collections, documents, and queries are typed. The schema feeds TypeScript types so query results infer correctly.
- **JSON Schema validation**: Documents are checked against a [JSON Schema](../../rx-schema.md). Required fields, enums, and string lengths are enforced at write time.
- **MongoDB-style queries**: Use `$gt`, `$in`, `$regex`, sorting, and compound indexes through the [RxQuery](../../rx-query.md) API. Queries return observables that re-emit when matching data changes.
- **CRDT plugin**: For collaborative apps that need formal merge semantics, the [CRDT plugin](../../crdt.md) provides documented operations on counters, sets, and lists.
- **Encryption**: The [encryption plugin](../../encryption.md) encrypts selected fields at rest using AES.
- **WebRTC P2P replication**: The [WebRTC replication plugin](../../replication-webrtc.md) syncs collections directly between browser peers without a central data server.
- **Conflict handling**: Custom conflict resolution is configured per collection through the [revisions and conflict handler API](../../transactions-conflicts-revisions.md).
## Code Sample: Schema and Reactive Query
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'notesdb',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
notes: {
schema: {
title: 'note schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
tags: { type: 'array', items: { type: 'string' } },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'updatedAt']
}
}
});
// Reactive query: re-emits whenever matching documents change.
const recent$ = db.notes
.find({
selector: { tags: { $in: ['inbox'] } },
sort: [{ updatedAt: 'desc' }],
limit: 20
})
.$;
recent$.subscribe(notes => {
console.log('inbox notes:', notes.map(n => n.title));
});
```
The schema enforces shape at write time, and the query result is a stream that any UI layer can subscribe to.
## Code Sample: Peer-to-Peer Replication via WebRTC
The [WebRTC replication plugin](../../replication-webrtc.md) gives you the same serverless P2P sync that draws people to GUN, with an explicit configuration and clear error events.
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer,
createSimplePeerWrtc
} from 'rxdb/plugins/replication-webrtc';
const replicationPool = await replicateWebRTC({
collection: db.notes,
topic: 'notes-room-42', // peers sharing a topic sync with each other
connectionHandlerCreator: getConnectionHandlerSimplePeer({
signalingServerUrl: 'wss://signaling.rxdb.info/',
wrtc: createSimplePeerWrtc(),
}),
pull: {},
push: {}
});
replicationPool.error$.subscribe(err => {
console.error('P2P sync error:', err);
});
```
Peers join a topic, the signaling server pairs them, and from there the data exchange runs directly between browsers. For a transport that does not require running your own signaling server, the Nostr replication plugin routes updates through public Nostr relays.
## FAQ
Yes. The [WebRTC replication plugin](../../replication-webrtc.md) syncs collections directly between browser peers. Both run without a central data server, and both reuse the same RxDB sync protocol used for HTTP and GraphQL backends.
Yes. RxDB stores data locally in IndexedDB, OPFS, SQLite, or memory, and any [replication](../../replication.md) is optional. With the WebRTC or Nostr plugins, multiple clients can sync directly with each other and never contact a backend you operate.
Export your GUN graph to a JSON file by walking the root nodes you care about and serializing each subgraph. Then map the flat documents onto an RxDB collection schema and bulk insert them with `collection.bulkInsert(docs)`. Because GUN graphs use references between nodes, denormalize linked nodes into embedded fields or split them across collections that match your query patterns.
RxDB does not bundle a full identity module. It pairs with any auth system you already use (JWT, OAuth, custom tokens) by passing credentials into the replication handler headers. For data confidentiality, the [encryption plugin](../../encryption.md) encrypts selected fields with AES, and signed payloads can be added on top in the replication layer when needed.
Each collection has a conflict handler. The default keeps the newer revision, and you can replace it with a custom function that merges fields, picks a winner based on metadata, or runs CRDT operations through the [CRDT plugin](../../crdt.md). The full model is described in [transactions, conflicts and revisions](../../transactions-conflicts-revisions.md).
## Comparison Table
| Topic | GUN (gundb) | RxDB |
| ------------------------ | ---------------------------------------- | -------------------------------------------------------------------- |
| Data model | JSON graph of linked nodes | JSON documents organized into typed collections |
| Schema | None, fields are free-form | [JSON Schema](../../rx-schema.md) with validation and migrations |
| TypeScript | Community types, dynamic API | First-class types inferred from schemas |
| Query language | Chainable graph traversal | [MongoDB-style queries](../../rx-query.md) with sort, limit, and indexes |
| Reactivity | Subscriptions on nodes | RxJS observables on documents and queries |
| Conflict resolution | Built-in HAM merge, opaque rules | Pluggable handler plus optional [CRDT plugin](../../crdt.md) |
| Encryption | SEA module | [Encryption plugin](../../encryption.md), AES on selected fields |
| P2P transport | Built-in WebSocket and WebRTC peers | [WebRTC](../../replication-webrtc.md) plugins |
| Server-based sync | Optional relay peers | HTTP, GraphQL, CouchDB, Firestore, and custom backends |
| Storage backends | IndexedDB, file, in-memory | IndexedDB, OPFS, SQLite, Dexie, LocalStorage, Memory, and more |
| Tooling | Minimal, source-level debugging | Devtools, logger, schema validator, migration runner |
| License | ZLIB and Apache 2.0 | Apache 2.0 with paid premium plugins |
## Follow Up
If GUN attracted you because of peer-to-peer sync but the debugging cost is slowing the project down, RxDB covers the same ground with a typed schema, documented merge semantics, and dedicated plugins for [WebRTC](../../replication-webrtc.md) replication. Start with the [RxDB Quickstart](../../quickstart.md), pick a storage that fits your runtime, and add a replication plugin once your local data model is stable.
More resources:
- [RxDB Sync Engine](../../replication.md)
- [WebRTC Replication](../../replication-webrtc.md)
- [CRDT Plugin](../../crdt.md)
- [Encryption Plugin](../../encryption.md)
- [Conflicts and Revisions](../../transactions-conflicts-revisions.md)
- [The Local-First Future](../../articles/local-first-future.md)
---
## RxDB as a Hoodie Alternative for Offline-First JavaScript Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Hoodie Alternative for Offline-First JavaScript Apps
If you built a project on **Hoodie**, you chose it for a clear reason: ship an [offline-first](../../offline-first.md) JavaScript application without writing a backend. The "noBackend" promise meant a single API call to store and sync data, with CouchDB and PouchDB doing the heavy lifting underneath. That promise still matters today, but the Hoodie project itself has been inactive for years, the `hood.ie` website is offline, and the GitHub repository has not received a meaningful commit in a long time.
Teams maintaining Hoodie applications now face a hard question: how do you keep the offline-first developer experience while moving to a stack that is actively maintained, typed, and supported across modern JavaScript runtimes? **RxDB** is a direct answer to that question. It keeps the local-first model, supports CouchDB replication out of the box, and adds reactive queries, multi-tab synchronization, and conflict handling that Hoodie never offered.
## A Short History of Hoodie
Hoodie started around 2012, founded by Caolan McMahon and a small team behind the **noBackend** movement. The idea was simple: frontend developers should not have to assemble servers, authentication systems, and databases just to store user data. Instead, a single client-side library would expose a small API (`hoodie.store.add(...)`, `hoodie.account.signUp(...)`) and handle persistence, sync, and accounts behind the scenes.
The technical foundation was:
- **PouchDB** in the browser as the local store.
- **CouchDB** on the server as the sync target.
- A small Node.js server wrapping CouchDB and providing account management.
For a few years Hoodie was a popular choice for offline-first prototypes, hackathons, and progressive web apps. Around 2018 to 2019 development slowed to near zero. The website at `hood.ie` is no longer reachable, and the GitHub organization shows the project as effectively unmaintained. Anyone running Hoodie in production today is running on frozen dependencies, including older versions of PouchDB and CouchDB clients that no longer receive security or performance updates.
## What is RxDB?
RxDB (Reactive Database) is a local-first, NoSQL database for JavaScript. It runs in browsers, Node.js, Electron, React Native, and any other JavaScript runtime. Data is stored locally through a pluggable storage layer (IndexedDB, OPFS, SQLite, in-memory, and more) and can be replicated to any backend through the [RxDB Sync Engine](../../replication.md).
Compared to a raw PouchDB setup, RxDB adds:
- A schema layer with JSON Schema validation and migrations.
- [Reactive queries](../../rx-query.md) that emit new results whenever the underlying data changes.
- [Multi-tab support](../../reactivity.md) so several browser tabs share a single consistent state.
- A pluggable [conflict handler](../../transactions-conflicts-revisions.md) instead of a fixed last-write-wins rule.
- First-class TypeScript types for collections, documents, and queries.
## Where Hoodie Falls Short Today
Hoodie's design was solid for its time, but the gap between Hoodie and a modern JavaScript stack has grown wide.
### 1. The Project is Unmaintained
The clearest issue is also the most important. Hoodie does not get bug fixes, security patches, or compatibility updates. New Node.js versions, new browsers, and new build tools can break a Hoodie setup with no upstream fix in sight.
### 2. CouchDB-Only Backend
Hoodie was tightly coupled to CouchDB. If your team wants to sync to PostgreSQL, a custom REST API, GraphQL, Firestore, or a peer-to-peer mesh, Hoodie does not help. RxDB treats the backend as a plugin choice and ships replication adapters for [CouchDB](../../replication-couchdb.md), [HTTP](../../replication-http.md), and several other targets.
### 3. Dated APIs and No TypeScript
Hoodie's client API predates modern JavaScript patterns. There is no first-class TypeScript support, no observable query API, and no integration with frameworks like React, Vue, Svelte, or Angular beyond plain callbacks. RxDB ships full type definitions and integrates cleanly with reactive UI frameworks through RxJS.
### 4. PouchDB Performance Limits
Hoodie depends on PouchDB for local storage. PouchDB works, but its IndexedDB adapter has well-known performance issues with large datasets, range queries, and bulk writes. RxDB lets you pick the storage that fits your workload, including the OPFS storage for high-throughput browser apps and the SQLite storage for native runtimes.
### 5. No Built-In Conflict Strategy Beyond CouchDB Defaults
Hoodie inherits CouchDB's conflict model, which surfaces conflicts but leaves resolution entirely to the application. RxDB lets you supply a [custom conflict handler](../../transactions-conflicts-revisions.md) per collection, so merges happen automatically and consistently across clients.
The numbers reflect this. As of July 30, 2026, [Hoodie](https://github.com/hoodiehq/hoodie) has 4,454 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `hoodie` package was downloaded 996 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/hoodie-vs-rxdb)). The last commit to the [Hoodie repository](https://github.com/hoodiehq/hoodie) was in January 2021, .
## Why RxDB is a Strong Hoodie Replacement
### Actively Maintained
RxDB sees regular releases, security updates, and an active community. Bugs get fixed and new platforms (React Native New Architecture, OPFS, modern Node versions) are supported as they appear.
### Keep CouchDB if You Want
If you already run a CouchDB cluster for your Hoodie deployment, you do not have to throw it away. The [CouchDB replication plugin](../../replication-couchdb.md) lets RxDB sync directly with CouchDB or any CouchDB-compatible server.
### Modern TypeScript and Reactive APIs
Every collection, document, and query is fully typed. Queries return RxJS observables, so your UI updates automatically when data changes locally or arrives from the server.
### Multi-Tab and Cross-Process Coordination
RxDB coordinates state across browser tabs, web workers, and Electron processes through a leader election and event broadcast system. Hoodie has no equivalent.
### Pluggable Storage and Backends
Pick the storage that fits each runtime, and pick the replication target that fits your infrastructure. Switching from CouchDB to a custom REST endpoint is a configuration change, not a rewrite.
## Code Sample: RxDB Collection with CouchDB Replication
This is the closest match to a Hoodie setup. A local collection backed by IndexedDB, replicating with a CouchDB server.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
const db = await createRxDatabase({
name: 'hoodie_migration',
storage: getRxStorageDexie(),
multiInstance: true,
eventReduce: true
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'string', format: 'date-time' }
},
required: ['id', 'title', 'done']
}
}
});
const replicationState = replicateCouchDB({
replicationIdentifier: 'todos-couchdb-replication',
collection: db.todos,
url: 'http://localhost:5984/todos/',
fetch: (url, options) => fetch(url, {
...options,
credentials: 'include'
}),
live: true,
pull: {},
push: {}
});
replicationState.error$.subscribe(err => console.error('Sync error', err));
```
## Code Sample: Reactive Query
Where Hoodie returned plain promises, RxDB returns observables that re-emit whenever the result set changes. This works for local writes and for documents pulled from the server.
```ts
import { Subscription } from 'rxjs';
const query = db.todos.find({
selector: { done: false },
sort: [{ updatedAt: 'desc' }]
});
const sub: Subscription = query.$.subscribe(openTodos => {
renderTodoList(openTodos);
});
// Adding a document anywhere triggers the subscription above
await db.todos.insert({
id: 't-1',
title: 'Migrate from Hoodie',
done: false,
updatedAt: new Date().toISOString()
});
```
The same query updates when a remote replication pulls in a new document, when another tab writes to the database, or when a peer pushes a change over WebRTC.
## Migration Notes from Hoodie and PouchDB to RxDB
A typical Hoodie migration follows these steps:
1. **Inventory your stores.** Each Hoodie store maps to an [RxCollection](../../rx-collection.md). Define a JSON Schema for each one, including the primary key.
2. **Keep CouchDB during the transition.** Point RxDB at the existing CouchDB databases using the [CouchDB replication plugin](../../replication-couchdb.md). Existing documents flow into the local RxDB store on first sync.
3. **Translate Hoodie queries.** `hoodie.store.findAll(...)` and filter callbacks become [RxQuery](../../rx-query.md) selectors with proper indexes.
4. **Replace event listeners.** `store.on('add', ...)` becomes a subscription to `collection.$` or to a query's `.$` observable. See [Reactivity](../../reactivity.md).
5. **Plan account migration.** Hoodie shipped its own account system. With RxDB you choose your auth provider and pass credentials into the replication `fetch` function.
6. **Decide on a long-term backend.** Many teams keep CouchDB. Others move to a custom HTTP endpoint using the [generic replication](../../replication-http.md) plugin so they can drop CouchDB entirely.
If you currently use PouchDB directly (with or without Hoodie), the migration is even smaller: replace the PouchDB instance with an RxDB collection, keep the same CouchDB server, and gain schemas, observables, and conflict handling on top.
## FAQ
No. The Hoodie project has not seen meaningful commits for several years, the `hood.ie` website is offline, and the GitHub organization is effectively dormant. Running Hoodie today means relying on frozen dependencies with no security or compatibility updates.
Yes. RxDB ships a [CouchDB replication plugin](../../replication-couchdb.md) that syncs directly with any CouchDB-compatible server. You can migrate the client without touching the server, then decide later whether to keep CouchDB or switch to a different backend.
RxDB gives you a local-first experience where the client is the source of truth. You still need a sync target, but it can be an existing CouchDB cluster, a managed service, a small custom HTTP endpoint, or even peer-to-peer WebRTC sync. The frontend code stays focused on data, queries, and UI, much like Hoodie's noBackend ideal. See [Local-First](../../articles/local-first-future.md) for the broader pattern.
Point RxDB at your existing CouchDB databases through the [CouchDB replication plugin](../../replication-couchdb.md). On first run, RxDB pulls documents into the local store, validates them against your new JSON Schema, and keeps syncing on every change. For Hoodie account data, export the relevant `_users` and per-user databases the same way you would back up any CouchDB instance.
## Hoodie vs RxDB Comparison Table
| Feature | Hoodie | RxDB |
| --- | --- | --- |
| Project status | Unmaintained since around 2018 to 2019 | Actively maintained |
| Local storage | PouchDB only | Pluggable: IndexedDB, OPFS, SQLite, in-memory, and more |
| Backend | CouchDB only | CouchDB, HTTP, GraphQL, Firestore, WebRTC, custom |
| Schema validation | Optional, manual | Built-in JSON Schema with migrations |
| Reactive queries | No | Yes, RxJS observables |
| TypeScript support | Limited | First-class types |
| Multi-tab coordination | No | Yes, leader election and broadcast |
| Conflict handling | CouchDB default surface only | Custom conflict handler per collection |
| Mobile runtimes | Browser focused | Browser, Node.js, Electron, React Native |
| Account system | Built-in | Bring your own auth |
| Website / domain | `hood.ie` offline | `rxdb.info` active |
## Follow Up
If you maintain a Hoodie application and want a path forward that keeps the offline-first model, RxDB is the closest direct replacement. You can keep CouchDB during the transition, gain reactive queries and TypeScript on day one, and choose a long-term backend that fits your infrastructure.
More resources:
- [RxDB Sync Engine](../../replication.md)
- [CouchDB Replication](../../replication-couchdb.md)
- [HTTP Replication](../../replication-http.md)
- [Reactive Queries](../../rx-query.md)
- [Conflict Handling](../../transactions-conflicts-revisions.md)
- [Local-First Future](../../articles/local-first-future.md)
---
## RxDB as a Horizon Alternative - Offline-First, Client-Side Reactive Database
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
import {ComparisonTable} from '@site/src/components/comparison-table';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a Horizon Alternative
Horizon was the official client-side library for RethinkDB, launched in 2016 to let developers build realtime JavaScript applications without writing server code. It offered a clean API for subscribing to live data, built-in authentication, and a permission system. But Horizon's life was short. The company behind RethinkDB shut down just months after Horizon launched, and offline support was never implemented. The project is now archived and receives no maintenance.
This page explains what Horizon was, where it failed to deliver for modern client-side applications, and why [RxDB](https://rxdb.info) is a strong replacement for teams that want the reactive, data-subscription model Horizon promised, with the offline-first architecture it never had.
---
## What Was Horizon?
Horizon (also written as horizon.io) was an open-source backend platform built on top of RethinkDB. It was launched by the RethinkDB company in May 2016. Its goal was to give frontend JavaScript developers a direct path to realtime data without needing to write or maintain a traditional REST or GraphQL API.
The Horizon client library connected directly to a Horizon server process, which in turn communicated with RethinkDB. Developers interacted with data through collections and a fluent API that returned RxJS Observables. A basic Horizon application looked like this:
```javascript
// Horizon: connect to the backend
const horizon = Horizon();
// Get a reference to a collection (table in RethinkDB)
const messages = horizon('messages');
// Store a document
messages.store({ id: 'msg-1', text: 'Hello, Horizon!', createdAt: new Date() });
// Watch for realtime changes - returns an RxJS Observable
messages.watch().subscribe(allMessages => {
renderMessages(allMessages);
});
// Fetch once (not live)
messages.fetch().subscribe(allMessages => {
console.log(allMessages);
});
// Order and limit
messages.order('createdAt', 'descending').limit(20).watch().subscribe(recent => {
renderRecentMessages(recent);
});
```
The `watch()` method was the centerpiece: it connected to RethinkDB changefeeds and pushed updated result sets to the client whenever data changed. This was compelling in 2016, when building a realtime app without polling required significant infrastructure work.
Horizon also provided:
- **Authentication** via local username/password, GitHub OAuth, Google OAuth, and other providers.
- **A permission system** with rules that controlled which users could read or write which documents.
- **An `hz` command-line tool** for scaffolding projects and running a local development server.
- **Embedding into existing Node.js apps** for teams that needed custom server logic alongside Horizon's data layer.
### Horizon's Timeline
- **May 2016** - Horizon launches publicly with its `hz` CLI and the Horizon client library for JavaScript.
- **October 2016** - RethinkDB Inc. announces it is shutting down. The company failed to build a sustainable business competing against hosted databases and cloud services.
- **February 2017** - The Linux Foundation (via the Cloud Native Computing Foundation) acquires RethinkDB and relicenses it as Apache 2.0. Horizon does not receive the same treatment.
- **2016-present** - Horizon receives no meaningful updates. The GitHub repository is effectively archived. The `rethinkdb/horizon` repository shows no significant activity after 2016.
The shutdown happened almost immediately after launch. Horizon never had a chance to mature. Key features that were planned but never shipped included offline support, which was requested by the community in an [open GitHub issue from 2016](https://github.com/rethinkdb/horizon/issues/58) and never resolved before the project was abandoned.
### What Horizon Did Well
For the narrow use case of building a connected, online-only realtime web application, Horizon reduced the amount of code developers needed to write. Subscribing to a collection and rendering the result directly in the UI, without writing any server routes, was genuinely useful. The Observable-based API was ahead of its time in bringing reactive programming idioms to the client-server data layer.
---
## Where Horizon Falls Short
### No Offline Support
Horizon was never built for offline scenarios. Every query, read, and write required a live network connection to the Horizon server. When the connection dropped, the Observable streams from `watch()` would stop emitting, and any attempt to call `store()`, `update()`, or `remove()` would fail silently or throw.
The offline support issue was one of the most-requested features on the Horizon GitHub repository. The [issue thread](https://github.com/rethinkdb/horizon/issues/58) collected significant community discussion, but the feature was never designed, let alone implemented. When the company shut down, the issue was closed without resolution.
For most modern applications, offline capability is not a nice-to-have. Users expect applications to work in areas with poor connectivity, on public transport, in buildings with unreliable Wi-Fi, and in regions where mobile data is expensive. An application that fails when the network drops is a broken application.
### The Project Is Abandoned
Horizon is not maintained. There are no security updates, no compatibility fixes for modern Node.js versions, no TypeScript type definitions, and no response to open issues or pull requests. Installing Horizon in a new project in 2025 requires working around dependency conflicts with modern tooling.
The underlying RethinkDB project still receives occasional community maintenance releases, but Horizon is separate and does not benefit from that work. Any team that built on Horizon faced a migration problem at some point, and most already completed that migration years ago.
### Server-Side Architecture, No Client-Side Storage
Horizon did not store any data on the client. Every piece of data lived in RethinkDB on the server. The client library was a subscription mechanism, not a database. This means:
- The application cannot serve any data when offline.
- There is no local query cache. Changing the query parameters means a new network request.
- There is no way to write data locally and sync it later.
- Closing and reopening the application requires re-fetching all data from the server.
This architecture is fundamentally incompatible with the offline-first pattern, where the application treats local storage as the primary source of truth and treats the server as a sync target rather than a mandatory dependency.
### No Conflict Resolution
Horizon provided no mechanism for resolving conflicts when two users modified the same document concurrently. The server-authoritative model meant that RethinkDB's last-write-wins behavior determined the result. In practice, when two users modified the same document, one of their changes was silently overwritten without any notification to either user.
For collaborative applications, this is a significant limitation. The developer had no API to detect a conflict, inspect both versions, or apply a merge strategy.
### Highly Opinionated Structure
Horizon required a specific project structure, a specific server process (`hz serve`), and its own authentication system. Integrating Horizon into an existing backend, a non-Node.js server, or a project with an existing authentication layer required significant workarounds. The permission system was designed around Horizon's own user model and could not be easily adapted to existing user databases or role systems.
---
The numbers reflect this. As of July 30, 2026, [Horizon](https://github.com/rethinkdb/horizon) has 6,735 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `@horizon/client` package was downloaded 513 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/@horizon/client-vs-rxdb)). The last commit to the [Horizon repository](https://github.com/rethinkdb/horizon) was in January 2021, .
## How RxDB Covers the Same Ground (and More)
[RxDB](https://rxdb.info) shares Horizon's core idea: data changes should propagate automatically to the UI through a reactive, Observable-based API. But RxDB implements this on the client side, in a full local database, rather than as a subscription mechanism to a remote server.
### Reactive Queries on the Client
In RxDB, every query is Observable. When you subscribe to a query, you receive the current result set immediately, and the Observable emits again whenever the data changes, whether from a local write or from a replication event.
```typescript
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
messages: {
schema: {
title: 'message schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
roomId: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'text', 'roomId', 'createdAt'],
indexes: ['createdAt']
}
}
});
// Subscribe to messages in a room, ordered by creation time
db.messages.find({
selector: { roomId: 'room-42' },
sort: [{ createdAt: 'asc' }]
}).$.subscribe(messages => {
renderMessages(messages); // called immediately and on every change
});
```
This is equivalent to Horizon's `watch()` API, but the data comes from local IndexedDB storage, not a remote server. The UI works the same way whether the user is online or offline.
RxDB uses the [event-reduce algorithm](https://github.com/pubkey/event-reduce) to make reactive updates efficient. When a document changes, RxDB checks whether the existing query result can be updated by applying the change event directly without re-running the full query against storage. This keeps reactive UI updates fast even in write-heavy workloads.
### Full Offline-First Operation
When a user opens an RxDB application without network access, every feature works normally. Reads come from local storage. Writes go to local storage. There are no errors, no loading spinners waiting for a server, and no missing data.
When connectivity returns, RxDB's replication plugins synchronize local changes with the remote backend automatically. The application seamlessly transitions between offline and online states without any code change required for individual features.
This is the [offline-first architecture](../../offline-first.md). RxDB treats local storage as the primary source of truth. The server is a sync target, not a dependency for normal operation. Horizon's architecture was exactly the opposite: the server was the only source of data, and offline operation was not possible.
### Multi-Tab Support
Horizon ran as a single connection per browser window. If a user opened two tabs of the same application, each tab would maintain its own WebSocket connection to the server, and local state between tabs could diverge.
RxDB provides a [SharedWorker storage mode](../../rx-storage-shared-worker.md) that runs a single database instance in a shared worker. All tabs share that single instance, so a write in one tab is immediately reflected in reactive queries in all other tabs without any additional code:
```typescript
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
For background tasks that only one tab should perform (such as running replication), RxDB includes a [leader election plugin](../../leader-election.md). One tab is elected leader and handles background work. If the leader tab is closed, another tab takes over automatically:
```typescript
import { RxDBLeaderElectionPlugin } from 'rxdb/plugins/leader-election';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBLeaderElectionPlugin);
// Only the leader tab runs replication
await db.waitForLeadership();
startReplication(db);
```
### Flexible Replication to Any Backend
Horizon required a Horizon server, which required RethinkDB. The entire stack was prescribed and non-negotiable. RxDB separates storage from replication and makes both independently configurable.
RxDB stores data locally and replicates to a backend using a plugin system. You can replicate to any backend your application already uses:
```typescript
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: db.messages,
replicationIdentifier: 'messages-http-v1',
pull: {
handler: async (checkpoint, batchSize) => {
const since = checkpoint?.updatedAt ?? 0;
const response = await fetch(
`/api/messages?since=${since}&limit=${batchSize}`
);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
handler: async (rows) => {
const response = await fetch('/api/messages', {
method: 'POST',
body: JSON.stringify(rows),
headers: { 'Content-Type': 'application/json' }
});
return response.json(); // returns conflicting docs or []
}
},
live: true,
retryTime: 5000
});
// Observable replication state
replicationState.active$.subscribe(active => console.log('Syncing:', active));
replicationState.error$.subscribe(err => console.error('Sync error:', err));
```
The pull handler fetches changes from the server since a checkpoint. The push handler sends local changes to the server. The replication protocol is simple enough that any backend language can implement the server side. There is no requirement to run a specific server process, use a specific database, or adopt a specific permission model.
For common backend setups, RxDB provides ready-made plugins:
| Plugin | Use case |
|---|---|
| [HTTP replication](../../replication-http.md) | Any REST API endpoint |
| [GraphQL replication](../../replication-graphql.md) | GraphQL APIs including AWS AppSync |
| [WebSocket replication](../../replication-websocket.md) | Low-latency server push |
| [CouchDB replication](../../replication-couchdb.md) | CouchDB or PouchDB server |
| [Firestore replication](../../replication-firestore.md) | Google Cloud Firestore |
### Pluggable Storage Backends
RxDB's storage layer is separate from its query and replication logic. The same application code runs with different storage engines depending on the environment:
| Environment | Storage Option |
|---|---|
| Browser (standard) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high-throughput) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multi-tab browsers | [SharedWorker](../../rx-storage-shared-worker.md) |
| Testing / CI | [Memory](../../rx-storage-memory.md) |
Switching storage is a one-line change:
```typescript
import { getRxStorageOpfs } from 'rxdb/plugins/storage-opfs';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageOpfs() // use OPFS for higher throughput in the browser
});
```
Horizon offered no equivalent. Data always lived in RethinkDB on the server. There was no local storage, no storage abstraction, and no way to run the application without a running RethinkDB instance.
### Conflict Resolution
Horizon inherited RethinkDB's last-write-wins conflict model. Two concurrent writes to the same document produced a silent overwrite. The developer had no mechanism to detect, inspect, or merge conflicting versions.
RxDB provides a configurable conflict handler per collection. When two versions of the same document arrive during replication, the conflict handler is called with both versions and returns the resolved document:
```typescript
await db.addCollections({
messages: {
schema: messageSchema,
conflictHandler: async ({ newDocumentState, realMasterState }) => {
// Keep whichever version was updated more recently
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative editing where documents can be modified concurrently by multiple users, RxDB supports [CRDT-based conflict resolution](../../crdt.md). CRDTs merge concurrent edits deterministically without requiring a central authority to decide the winner:
```typescript
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const messageSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
roomId: { type: 'string' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
### Schema Validation and TypeScript Support
Horizon stored and returned plain JavaScript objects with no validation. If a client stored a document with the wrong field name or the wrong data type, RethinkDB accepted it without complaint, and that corrupted document propagated to all other clients through changefeeds.
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before writing it to storage. Invalid documents are rejected at the write step, before they can reach the local store or propagate through replication:
```typescript
try {
await db.messages.insert({
id: 'msg-001',
// 'text' field is required but missing
roomId: 'room-42',
createdAt: Date.now()
});
} catch (err) {
console.error(err); // Schema validation error: missing 'text'
}
```
RxDB generates TypeScript types automatically from the schema. Collection methods like `find()`, `insert()`, and `upsert()` are fully typed, giving you IDE autocompletion and compile-time safety for all database operations.
### Schema Migration
As an application evolves, the data model changes. Adding new required fields, renaming properties, or restructuring nested objects all require updating existing stored documents.
Horizon provided no migration system. If you changed the shape of your data, you were responsible for writing a migration script that connected to RethinkDB and updated every document manually, with no help from the framework.
RxDB has a built-in [schema migration system](../../migration-schema.md). When the schema version number increases, RxDB automatically runs migration strategies on all locally stored documents before making the database available to the application:
```typescript
await db.addCollections({
messages: {
schema: messageSchemaV2, // version: 1
migrationStrategies: {
1: (oldDoc) => {
// Migrate from version 0: add 'threadId' with a default
return {
...oldDoc,
threadId: oldDoc.threadId ?? 'main'
};
}
}
}
});
```
Migrations run locally on each client independently. They do not require a coordinated server deployment or a manual database update script.
### Encryption at Rest
Horizon sent data between the browser and RethinkDB in plaintext (over WebSocket). Data in RethinkDB was stored as-is. If a user's device was compromised, the IndexedDB data from the browser session would be readable without decryption.
RxDB includes a built-in [encryption plugin](../../encryption.md) that encrypts individual document fields before writing them to local storage. The data is decrypted on read, so the application sees plaintext, but the underlying storage contains only ciphertext:
```typescript
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'your-encryption-passphrase'
});
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
sensitiveData: { type: 'string' }
},
encrypted: ['sensitiveData'] // stored as ciphertext in IndexedDB
};
```
### Observable Change Streams
Horizon's `watch()` method pushed complete updated result sets from the server. RxDB provides observable change streams at both the collection and document level, giving fine-grained access to change events locally:
```typescript
// Subscribe to all changes in the messages collection
db.messages.$.subscribe(changeEvent => {
console.log('Operation:', changeEvent.operation); // INSERT, UPDATE, DELETE
console.log('Document ID:', changeEvent.documentId);
console.log('Document data:', changeEvent.documentData);
});
// Subscribe to changes on a specific document
const doc = await db.messages.findOne('msg-001').exec();
doc.$.subscribe(updatedDoc => {
console.log('Document updated:', updatedDoc?.text);
});
```
These events originate from the local database. They fire for writes made locally and for documents that arrive through replication. They fire while the user is offline. There is no server connection required.
---
## Getting Started with RxDB
Install RxDB and RxJS:
```bash
npm install rxdb rxjs
```
Create a database, add a collection, write documents, and subscribe to reactive queries:
```typescript
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
addRxPlugin(RxDBDevModePlugin);
const db = await createRxDatabase({
name: 'chatapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
messages: {
schema: {
title: 'message schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
roomId: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'text', 'roomId', 'createdAt'],
indexes: ['roomId', 'createdAt']
}
}
});
// Write a message
await db.messages.insert({
id: 'msg-001',
text: 'Hello from RxDB!',
roomId: 'room-42',
createdAt: Date.now()
});
// Reactive query: emits current state immediately and on every change
db.messages.find({
selector: { roomId: 'room-42' },
sort: [{ createdAt: 'asc' }]
}).$.subscribe(messages => {
console.log('Current messages:', messages.map(m => m.text));
});
```
This works entirely offline. Connect a replication plugin when you need server sync.
---
## Comparison Summary
| Aspect | Horizon | RxDB |
|---|---|---|
| **Project status** | Abandoned since 2016 | Actively maintained since 2016 |
| **Where it runs** | Client connects to server | Full database on the client |
| **Offline support** | None (network required for all operations) | Full offline-first operation |
| **Reactive queries** | `watch()` pushes from RethinkDB | Observable queries from local storage |
| **Data location** | Remote RethinkDB server | Local storage (IndexedDB, OPFS, SQLite) |
| **Backend dependency** | Must run Horizon + RethinkDB | Any backend or no backend |
| **Replication** | Horizon-proprietary protocol | Pluggable (HTTP, WebSocket, CouchDB, GraphQL, custom) |
| **Conflict resolution** | Last-write-wins (silent overwrite) | Configurable handler or CRDT-based merging |
| **Schema validation** | None | JSON Schema enforced on every write |
| **Schema migration** | Manual scripts | Built-in versioned migration strategies |
| **Multi-tab support** | Separate connections per tab | SharedWorker (shared state across all tabs) |
| **Encryption at rest** | None | Built-in field-level encryption plugin |
| **TypeScript support** | None (JavaScript only) | Auto-generated types from schema |
| **Authentication** | Built-in OAuth providers (now outdated) | Handled by your existing backend; no lock-in |
| **Permissions** | Horizon-specific rule system | Handled by your existing backend |
| **Security updates** | None (abandoned) | Ongoing with active development |
| **License** | Apache 2.0 | Apache 2.0 |
---
## FAQ
Yes. RxDB can take over the client-side data layer. You keep RethinkDB on the server and build a thin API (REST or WebSocket) in front of it. Then use RxDB's [custom replication](../../replication.md) or [WebSocket replication](../../replication-websocket.md) plugin to sync data between RxDB on the client and RethinkDB on the server.
The main difference is that Horizon was the entire client-server protocol, while with RxDB you own the API layer. That gives you full control over authentication, rate limiting, and data access rules, instead of depending on Horizon's specific permission model.
Horizon's `watch()` connected to RethinkDB's changefeed system and pushed updated result sets to the client. When any document in a collection changed, Horizon sent the entire updated array to the subscriber.
RxDB works similarly at the API level: subscribing to a query gives you the current result set immediately, and the Observable re-emits the updated array whenever relevant data changes. The difference is where the data comes from. Horizon pulled from a remote server, so `watch()` required a live connection. RxDB emits from local storage, so reactive queries work identically online and offline.
RxDB uses the [event-reduce algorithm](https://github.com/pubkey/event-reduce) to compute result set updates efficiently without re-running the full query on every change, keeping reactive updates fast even in write-heavy applications.
Yes. RxDB is framework-agnostic. Its reactive queries return RxJS Observables, which integrate with any framework. RxDB provides convenience hooks for React (`useRxQuery`, `useRxDocument`) that wrap Observable subscriptions in React's state model. For Angular, RxJS Observables can be used directly with the async pipe. For Vue, plain Observable subscriptions work with `ref` and `reactive`.
Horizon was also framework-agnostic at the API level, but the outdated state of its dependencies makes integration with modern framework versions difficult without forking the library.
RxDB's replication plugins run continuously with automatic retry. When the network is unavailable, the pull and push handlers fail, and RxDB waits for `retryTime` milliseconds before retrying. When the network returns, replication resumes from the last successful checkpoint. No writes are lost: documents written while offline are stored locally and pushed to the server when the connection is re-established. The reactive queries in the UI stay up to date throughout, reflecting local writes immediately without waiting for server confirmation.
RxDB has been in active development since 2016 and is used by companies in production applications. It has a working business model through [premium plugins](/premium/), which funds ongoing maintenance. The project has close to zero open bugs and receives regular releases. Unlike Horizon, which has had no maintenance since 2016, RxDB continues to add new storage backends, fix browser compatibility issues, and improve performance.
---
## RxDB as an InstantDB Alternative with Custom Backends
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as an InstantDB Alternative with Custom Backends
If you arrived here looking for an **InstantDB alternative**, you most likely want one of three things:
- A real [offline-first](../../offline-first.md) local database, not just a network cache that flushes when storage gets tight.
- The freedom to plug your own backend into the sync layer instead of relying on a hosted Datalog service.
- An open source stack that you can audit, fork, and self-host without depending on a single vendor.
**RxDB** (Reactive Database) is a client-side NoSQL database for JavaScript that stores data locally in the browser, mobile runtimes, Electron, or Node.js, and replicates with any backend you control. It is fully open source and has been used in production for offline-first apps for years.
## A Short History of InstantDB
InstantDB launched in 2023 out of Y Combinator with the goal of making real-time, collaborative apps feel as easy to build as a static page. The product centers on a hosted sync service written in Clojure that exposes a Datalog-style query language to JavaScript clients. Clients write through optimistic updates, the server stores the canonical state, and changes are streamed back to every connected device.
The design works well for prototypes and small collaborative tools. Queries are expressive once you know Datalog, and the SDK handles subscriptions and rollbacks automatically. The trade-off is that the sync engine and storage are tightly coupled to the InstantDB cloud, and the local layer is a query cache rather than a full database.
## What Is RxDB?
RxDB is an embeddable JavaScript database built around three ideas:
- **Local-first storage**: every read and write hits a local engine first. The app keeps working when the network is gone. See [offline-first](../../offline-first.md) and [local-first future](../../articles/local-first-future.md).
- **Reactive queries**: any [RxQuery](../../rx-query.md) returns an observable that re-emits when matching data changes, locally or from a remote push. See [reactivity](../../reactivity.md).
- **Pluggable replication**: the [Sync Engine](../../replication.md) talks to any HTTP endpoint, GraphQL server, CouchDB, Firestore, or peer over WebRTC. The protocol is documented and you can implement it on any backend with [HTTP replication](../../replication-http.md).
Storage is also pluggable. The same collection can run on IndexedDB, OPFS, SQLite, in-memory, or LocalStorage without changing application code.
## Limitations of InstantDB
InstantDB is a good fit for many apps, but a few constraints push teams to look for an alternative.
### 1. The Sync Service Is Hosted Only
InstantDB requires its managed backend. There is no open source server you can run on your own infrastructure for production workloads. If your app must run on customer hardware, in regulated environments, or in regions where the InstantDB cloud is not available, you are stuck.
### 2. Datalog Has a Learning Curve
InstantDB queries use a Datalog-style triple syntax. The model is expressive once it clicks, but new contributors have to learn it before they can ship a feature. RxDB uses a JSON Mango query format that is familiar to anyone who has touched MongoDB, PouchDB, or Supabase.
### 3. The Local Layer Is a Cache
InstantDB stores results of subscribed queries on the client so the UI stays responsive during short network drops. It is not a full local database. You cannot run arbitrary queries that were never subscribed, and the cache can be evicted. RxDB stores every document of every replicated collection on disk and lets you query any field at any time.
### 4. Few Storage Adapter Options
The InstantDB client picks its own persistence layer. RxDB exposes a [storage interface](../../rx-collection.md) with adapters for IndexedDB, OPFS, SQLite, Memory, LocalStorage, Dexie, and more. You can swap storage per platform or per collection.
### 5. Limited Conflict Resolution Knobs
InstantDB resolves write conflicts internally. You get optimistic updates and rollback, but the resolution policy is mostly fixed. RxDB lets you write a [custom conflict handler](../../transactions-conflicts-revisions.md) per collection, or opt into [CRDTs](../../crdt.md) when you need automatic merging without server arbitration.
Both projects are under active development. As of July 30, 2026, [InstantDB](https://github.com/instantdb/instant) has 10,366 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## Why Teams Pick RxDB Instead
### Bring Your Own Backend
RxDB does not ship with a mandatory cloud. You connect a collection to whatever you already run:
- A REST API over [HTTP replication](../../replication-http.md).
- A GraphQL endpoint with subscriptions.
- A CouchDB or Firestore instance using the official replication plugins.
- WebRTC peers for direct device-to-device sync.
The sync protocol is a simple pull plus push plus event stream contract. Anything that can implement those three calls can be an RxDB backend.
### Open Source
RxDB is Apache 2.0 on the core and the storage adapters. You can read every line of the engine, fork it, and run it without sending data to a third party.
### JSON Mango Queries
Queries look like the rest of the JavaScript ecosystem. See [RxQuery](../../rx-query.md) for the full operator list.
### Observable Queries Out of the Box
Every query is an RxJS observable. UI frameworks subscribe once and re-render when data changes. See [reactivity](../../reactivity.md) and [optimistic UI](../../articles/optimistic-ui.md).
### Real Conflict Resolution
Plug in a [custom conflict handler](../../transactions-conflicts-revisions.md) per collection, or enable [CRDTs](../../crdt.md) when you want commutative merges without writing your own merge logic.
### Multi-Storage and Multi-Tab
The same code runs on IndexedDB in the browser, OPFS for higher write throughput, SQLite on React Native, or in-memory for tests. The [multi-tab](../../rx-collection.md) layer makes sure two browser tabs see the same state through a shared worker or BroadcastChannel.
## Code Sample: Rewriting an InstantDB Query in RxDB
A typical InstantDB query that fetches open todos for a user looks roughly like this:
```ts
// InstantDB
const { data } = db.useQuery({
todos: {
$: { where: { ownerId: userId, done: false } }
}
});
```
The same query in RxDB uses the [Mango query format](../../rx-query.md) and returns an observable:
```ts
// RxDB
const query = db.todos.find({
selector: {
ownerId: userId,
done: false
},
sort: [{ createdAt: 'desc' }]
});
const todos$ = query.$; // Observable
```
`todos$` re-emits whenever a matching document is inserted, updated, or deleted, whether the change came from a local write, another tab, or the replication stream.
## Code Sample: Driving a React UI From an Observable
A small React component that mirrors the InstantDB pattern of subscribing to a query and writing optimistically:
```tsx
import { useEffect, useState } from 'react';
import { db } from './db';
export function TodoList({ userId }: { userId: string }) {
const [todos, setTodos] = useState([]);
useEffect(() => {
const sub = db.todos
.find({ selector: { ownerId: userId, done: false } })
.$.subscribe(setTodos);
return () => sub.unsubscribe();
}, [userId]);
async function addTodo(title: string) {
// Optimistic write: the local store updates first,
// replication pushes it to the backend in the background.
await db.todos.insert({
id: crypto.randomUUID(),
ownerId: userId,
title,
done: false,
createdAt: new Date().toISOString()
});
}
return (
{todos.map(t => {t.title})}
);
}
```
The local insert resolves immediately. The replication layer streams the write to the server when the network is available, and any conflict is funneled through the configured conflict handler. See [optimistic UI](../../articles/optimistic-ui.md) for the full pattern.
## Hosting: Managed Sync vs Self-Hosted
| Concern | InstantDB | RxDB |
| --- | --- | --- |
| Sync service | Hosted by InstantDB | Self-hosted on any stack |
| Storage location | InstantDB cloud | Your servers, your database |
| Pricing model | Per-app hosted plan | Cost of your own infrastructure |
| Data residency | InstantDB regions | Wherever you deploy |
| Open source server | No | Yes, the protocol is open |
If you want to ship fast and a hosted backend is acceptable, InstantDB removes a lot of work. If your team needs to own the data path end to end, RxDB lets you put the sync server next to the rest of your services and reuse your existing auth, logging, and backups.
## FAQ
The InstantDB sync service is a hosted product. There is no supported way to run the production server on your own infrastructure today. If self-hosting is a hard requirement, RxDB plus a backend you already operate is a closer fit.
No. RxDB queries use a JSON Mango syntax similar to MongoDB and PouchDB. The format is documented in [RxQuery](../../rx-query.md) and supports selectors, sorting, indexes, and limits without a separate query language.
Every write goes to the local storage first and resolves immediately. Observable queries re-emit with the new state, so the UI updates without waiting for the server. The [Sync Engine](../../replication.md) pushes the change in the background, and conflicts are routed through the collection's [conflict handler](../../transactions-conflicts-revisions.md). See [optimistic UI](../../articles/optimistic-ui.md) for an end-to-end example.
Yes. RxDB has a storage interface with adapters for IndexedDB, OPFS, SQLite, in-memory, LocalStorage, and Dexie. You pick the adapter when you create the database, and the rest of the API stays the same. See [RxCollection](../../rx-collection.md) for how storage plugs into a collection.
RxDB streams changes from the server through the replication event channel and from other browser tabs through BroadcastChannel. Combined with [observable queries](../../reactivity.md), the UI updates as soon as a remote change lands. For peer-to-peer setups, the WebRTC plugin lets devices sync directly. See [realtime database](../../articles/realtime-database.md) for the architecture.
## Comparison Table
| Feature | InstantDB | RxDB |
| --- | --- | --- |
| License | Proprietary client, hosted backend | Apache 2.0 core |
| Backend | Hosted Clojure sync service | Any HTTP, GraphQL, CouchDB, Firestore, or WebRTC peer |
| Query language | Datalog-style triples | JSON Mango selectors |
| Local layer | Query cache | Full local database |
| Storage adapters | Built in, fixed | IndexedDB, OPFS, SQLite, Memory, LocalStorage, Dexie |
| Offline writes | Yes, queued | Yes, persisted to local storage |
| Conflict resolution | Built-in, limited knobs | Custom handler per collection or [CRDT](../../crdt.md) |
| Reactivity | Subscriptions on queries | RxJS observables on every query |
| Multi-tab sync | Yes | Yes, via BroadcastChannel or shared worker |
| P2P sync | No | Yes, via WebRTC plugin |
| Self-hosting | Not supported | Standard |
## Follow Up
If you want the developer experience of InstantDB but with a backend you control, an open source codebase, and a real local database underneath, RxDB is worth a look. Start with the [Sync Engine](../../replication.md) docs to see how a collection connects to your existing API, then read the [reactivity](../../reactivity.md) guide for how the UI stays in sync.
More resources:
- [RxDB Sync Engine](../../replication.md)
- [HTTP Replication](../../replication-http.md)
- [Custom Conflict Resolution](../../transactions-conflicts-revisions.md)
- [CRDT Support](../../crdt.md)
- [Local-First Future](../../articles/local-first-future.md)
- [Optimistic UI Patterns](../../articles/optimistic-ui.md)
- [Realtime Database](../../articles/realtime-database.md)
---
## RxDB as a localForage Alternative for Real Database Features
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a localForage Alternative for Real Database Features
[localForage](https://localforage.github.io/localForage/) gives JavaScript developers a clean promise based wrapper around browser storage. It is a thin key-value layer that picks the best available backend, usually IndexedDB, with fallbacks to WebSQL or localStorage. Teams reach for it when they want a simple `setItem` and `getItem` API that works across browsers without writing IndexedDB transaction code by hand.
The trouble starts when an app grows past simple caching. As soon as you need indexed queries, schema validation, change subscriptions, replication with a backend, or coordination across browser tabs, you end up rebuilding most of a database on top of localForage. That is when [RxDB](https://rxdb.info/) becomes a better fit.
## A Short History of localForage
localForage was started around 2014 by developers at Mozilla as part of efforts to make offline web apps more practical. The motivation was straightforward: IndexedDB had a verbose, event based API, WebSQL was deprecated in some browsers, and localStorage was synchronous and capped at a few megabytes. localForage hid those differences behind a single API modeled on `localStorage`.
The library settled into a stable shape early. Recent commit activity on the main repository has been low, and the feature set has stayed close to its original scope: get, set, remove, clear, keys, length, and a few iteration helpers. It does what it set out to do, and nothing more.
That focus is the point. localForage is a storage compatibility layer, not a database. When the requirements list grows beyond key-value reads and writes, the gap between what localForage offers and what an application needs widens.
## What RxDB Is
RxDB is a [local-first](../../articles/local-first-future.md), NoSQL database that runs inside JavaScript runtimes. It stores documents in [collections](../../rx-collection.md), validates them against a schema, runs MongoDB style [queries](../../rx-query.md) with indexes, and emits changes through RxJS observables. RxDB sits on top of a pluggable storage layer, so the same database code can run on [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), in memory, in Node.js, in React Native, or in Electron.
On top of local storage, RxDB ships a [replication protocol](../../replication.md) that syncs collections with any backend that can speak HTTP, GraphQL, WebRTC, CouchDB, or Firestore. Reads stay local and fast, while writes flow to the server in the background.
## Where localForage Stops
The places where localForage runs out of road are predictable once you list them:
- **Key-value only.** Every value is opaque to the library. There is no notion of fields, types, or relationships.
- **No indexes.** You cannot ask for "all todos where `done = false` and `dueDate < tomorrow`". You either keep your own index keys by hand or scan all entries.
- **No queries.** There is no query language, no filtering, no sorting, no pagination beyond what you build yourself.
- **No schema.** Data shape drifts over time and migrations become custom scripts.
- **No reactivity.** When a value changes, other parts of the app do not find out unless you wire your own event bus on top.
- **No multi-tab coordination.** Two tabs writing to the same key can clobber each other without warning.
- **No replication.** Syncing with a server is left entirely to the application.
- **Limited debugging.** There are no dev tools that understand collections, queries, or change streams because those concepts do not exist in the library.
For a cache of avatars or a saved form draft, none of this matters. For an app that wants to feel like a product with offline support, all of it matters.
The numbers reflect this. As of July 30, 2026, the last commit to the [localForage repository](https://github.com/localForage/localForage) was in July 2024, .
## What RxDB Adds On Top
RxDB treats the browser like a real database environment.
- **Documents and collections.** Data is stored in typed [collections](../../rx-collection.md) with a JSON schema that validates every write.
- **Indexed queries.** Define indexes in the schema and run [MongoDB style queries](../../rx-query.md) such as `find`, `findOne`, `where`, `gt`, `in`, `sort`, `skip`, and `limit`.
- **Reactive results.** Every query returns an [observable](../../reactivity.md) that re-emits when matching documents change, so UI components stay in sync without manual refetching.
- **Multi-tab.** Open the same app in three tabs and they share one database state, with leader election and cross-tab change propagation handled by RxDB.
- **Replication primitives.** A [pull/push checkpoint protocol](../../replication.md) keeps local data in sync with any backend you choose, including custom REST APIs.
- **Storage choice.** Swap [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), memory, or a server side storage without changing application code.
- **Conflict handling.** Each document carries a revision, and a custom conflict handler decides how concurrent edits merge.
## Code Sample: Reading a Single Record
A typical localForage read by key:
```ts
import localforage from 'localforage';
const todo = await localforage.getItem('todo-42');
if (todo) {
console.log(todo.title);
}
```
The same lookup in RxDB, by primary key, with a typed result and a schema validated value behind it:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
dueDate: { type: 'string', format: 'date-time' }
},
required: ['id', 'title', 'done'],
indexes: ['done', 'dueDate']
}
}
});
const todo = await db.todos.findOne('todo-42').exec();
console.log(todo?.title);
```
The RxDB version is longer at setup time, but the database now knows the shape of a todo, can index `done` and `dueDate`, and can stream changes to the rest of the app.
## Code Sample: Indexed Range Query With a Subscription
In localForage, finding open todos due before tomorrow means iterating every entry:
```ts
const openTodos: Todo[] = [];
await localforage.iterate((value) => {
if (!value.done && value.dueDate < tomorrow) {
openTodos.push(value);
}
});
```
The same query in RxDB uses the index and returns an observable that re-fires on every change:
```ts
const query = db.todos.find({
selector: {
done: false,
dueDate: { $lt: tomorrow }
},
sort: [{ dueDate: 'asc' }]
});
const subscription = query.$.subscribe((openTodos) => {
render(openTodos);
});
```
Insert a new todo, mark one as done, or sync a remote change in another tab, and the subscriber receives the new result set without writing extra code.
## Storage Layer Notes
localForage and RxDB both end up writing to similar browser primitives, but the way they use them differs.
- localForage selects one backend per database and stores opaque blobs at string keys.
- RxDB writes structured documents through a pluggable [storage interface](../../rx-storage-indexeddb.md). For modern browsers, the [OPFS storage](../../rx-storage-opfs.md) avoids many [IndexedDB performance issues](../../slow-indexeddb.md) by writing to the Origin Private File System. The [Dexie storage](../../rx-storage-dexie.md) is a thin layer over IndexedDB for cases where Dexie is already in use.
Because storages are swappable, the same RxDB schemas and queries run unchanged across these backends, including in Node.js or React Native where IndexedDB is not the right fit.
## FAQ
For a flat cache of API responses keyed by URL, localForage is fine and has a smaller footprint. Reach for RxDB when the cache needs queries, indexes, expiry rules expressed as fields, change subscriptions for the UI, or replication back to a server. See the [reactivity guide](../../reactivity.md) for how observable queries replace manual cache invalidation.
Yes. RxDB ships a localStorage based [storage adapter](../../articles/localstorage.md) for small datasets, and IndexedDB or OPFS adapters for larger ones. Unlike raw localStorage, RxDB gives you schemas, queries, and async APIs that do not block the main thread.
Yes. With `multiInstance: true`, RxDB coordinates across tabs of the same origin. Writes in one tab are visible to queries in other tabs, leader election picks one tab to run replication, and change events propagate over a BroadcastChannel.
RxDB has been used with hundreds of thousands of documents per collection in IndexedDB. For larger datasets or write heavy workloads, the [OPFS storage](../../rx-storage-opfs.md) sidesteps many of the [slow IndexedDB](../../slow-indexeddb.md) bottlenecks and keeps query latency low.
## Comparison Table
| Feature | localForage | RxDB |
| --- | --- | --- |
| Data model | Key-value blobs | Documents in collections |
| Schema validation | None | JSON Schema per collection |
| Queries | None, manual iteration | MongoDB style with indexes |
| Indexes | Not supported | Declared in schema |
| Reactivity | None | Observable queries and documents |
| Multi-tab sync | Not handled | Built in via BroadcastChannel |
| Replication | Not included | Pull/push protocol with many plugins |
| Conflict handling | Not applicable | Per document revisions and custom handlers |
| Storage backends | IndexedDB, WebSQL, localStorage | IndexedDB, OPFS, Dexie, memory, Node.js, React Native, more |
| Encryption | Not built in | Plugin available |
| Migrations | Manual | Schema versioning with migration strategies |
| Offline first | Storage only | Full [offline first](../../offline-first.md) stack |
| Active development | Low | Active |
## When to Pick Which
Choose localForage when the job is "store a few values in the browser without thinking about IndexedDB". It is small, well understood, and stays out of the way.
Choose RxDB when the app needs a real client side database: typed [collections](../../rx-collection.md), indexed [queries](../../rx-query.md), [reactive results](../../reactivity.md), [multi-tab](../../rx-storage-indexeddb.md) coordination, and [replication](../../replication.md) with a backend. RxDB takes more setup at first, then pays it back as features are added on top of the same data layer.
More resources:
- [RxDB Quickstart](../../quickstart.md)
- [RxDB Sync Engine](../../replication.md)
- [RxDB Storage Plugins](../../rx-storage-indexeddb.md)
- [Local First Future](../../articles/local-first-future.md)
---
## RxDB as a LokiJS Alternative for Persistent JavaScript Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a LokiJS Alternative for Persistent JavaScript Apps
Developers reach for **LokiJS** when they want a small, MongoDB-like JavaScript database that lives in memory and feels instant. The trade-off shows up later: writes only become durable when an adapter flushes them on a timer or before the process exits, replication across tabs and devices is missing, and the project itself has slowed down. Teams that ship real applications eventually look for a database that keeps the same ergonomic API but stores data safely, syncs across clients, and is still under active development.
This page explains how **RxDB** fits that role and how it covers both the in-memory speed scenarios that drew people to LokiJS and the persistence and sync gaps that pushed them away.
## A Short History of LokiJS
LokiJS was started around 2014 by Joe Minichino as an embeddable JavaScript document store with a MongoDB-style query API. It kept the entire dataset in a JavaScript object graph, which is what made queries and mutations feel fast: there was no IO on the hot path. To survive a page reload, LokiJS shipped persistence adapters for IndexedDB, the file system in Node.js, and other targets. Those adapters serialize the in-memory state and write it out, either after a configurable autosave interval or when the process is asked to shut down.
Around 2020 the pace of releases dropped sharply. The project is closer to "feature complete" than to "actively developed", and unresolved issues have piled up. For new projects the question is no longer "is LokiJS fast enough" but "will the database I pick today still be maintained and safe to use in three years".
## What RxDB Is
RxDB is a [local-first](../../offline-first.md), reactive, NoSQL database for JavaScript. It runs in the browser, in Node.js, in Electron, and in React Native. Documents are validated against a JSON schema, queries are MongoDB-style, and every [RxQuery](../../rx-query.md) is observable so the UI can re-render when data changes. The storage layer is pluggable, so the same application code can run on top of [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), [SQLite](../../rx-storage-sqlite.md), or a pure [in-memory](../../rx-storage-memory.md) store.
On top of that, RxDB ships a [replication protocol](../../replication.md) that keeps clients in sync with each other and with a backend, with proper [conflict handling](../../transactions-conflicts-revisions.md).
## Where LokiJS Falls Short
The shortcomings below are the ones that show up most often when teams move off LokiJS:
- **In-memory primary store.** The authoritative copy of the data is the JavaScript object graph. If the tab crashes, the device loses power, or the user closes the browser between autosave intervals, every write since the last flush is gone.
- **Persistence is a side effect.** Adapters write a serialized snapshot of the database. That makes durability coarse and slow on larger datasets, since saving means re-emitting the whole collection or large parts of it.
- **No real replication.** LokiJS has no built-in protocol to sync changes between tabs, devices, or a server. Multi-tab usage in the browser is fragile because two tabs can both load the same database file and overwrite each other on save.
- **No conflict resolution.** Without revisions or a sync engine, there is no defined behavior when two writers touch the same document.
- **Dated codebase and low activity.** New runtimes, new browser storage APIs (OPFS, modern IndexedDB usage patterns), and new bundler conventions are not being adopted.
- **Maintenance mode.** Bug fixes and security patches arrive slowly, if at all.
The numbers reflect this. As of July 30, 2026, [LokiJS](https://github.com/techfort/LokiJS) has 6,830 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296. The last commit to the [LokiJS repository](https://github.com/techfort/LokiJS) was in March 2022, .
## What RxDB Gives You Instead
RxDB was designed around the assumption that the database has to outlive a tab crash and stay consistent across many clients.
- **Durable storages by default.** Pick the engine that matches the runtime: [IndexedDB](../../rx-storage-indexeddb.md) and [OPFS](../../rx-storage-opfs.md) in the browser, [Dexie](../../rx-storage-dexie.md) as a thin wrapper over IndexedDB, [SQLite](../../rx-storage-sqlite.md) for Node.js, Electron, and React Native. Each write is persisted by the underlying engine, not by a periodic snapshot of the whole dataset.
- **In-memory mode when you want it.** The [Memory RxStorage](../../rx-storage-memory.md) keeps everything in RAM for cache-style use cases, ephemeral test runs, and hot-path workloads where you do not need persistence.
- **Real replication.** The [RxDB sync engine](../../replication.md) handles pull, push, and live updates against any backend you point it at, including HTTP, GraphQL, CouchDB, WebRTC peers, and Firestore.
- **MongoDB-style queries with indexes.** Define indexes on the fields you query and run rich selectors locally. Queries are observable, so subscribing to a result set is a single call.
- **Reactive UI integration.** Each [RxQuery](../../rx-query.md) emits the latest result whenever the underlying data changes, which is the [reactivity](../../reactivity.md) story LokiJS never fully had.
- **Multi-tab safety.** RxDB coordinates writes across tabs and refuses to corrupt itself when two tabs of the same origin open the same database.
- **Schema and migrations.** Documents are validated, and schema upgrades are first-class instead of ad-hoc.
## Defining a Schema and Subscribing to Changes
The example below mirrors what a LokiJS user would write to insert a document and observe a query, but with RxDB on top of IndexedDB so writes survive a reload.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'appdb',
storage: getRxStorageIndexedDB(),
multiInstance: true,
eventReduce: true
});
await db.addCollections({
notes: {
schema: {
title: 'notes schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'updatedAt'],
indexes: ['updatedAt']
}
}
});
await db.notes.insert({
id: 'note-1',
title: 'First note',
body: 'Hello RxDB',
updatedAt: Date.now()
});
// Observe a live query: the subscription fires on every change
const query = db.notes
.find()
.sort({ updatedAt: 'desc' });
query.$.subscribe(notes => {
console.log('current notes:', notes.map(n => n.title));
});
```
Compared to LokiJS, the meaningful difference is not the API surface, it is what happens under the hood: each insert is durable in IndexedDB, the query is reactive through RxJS, and other tabs of the same origin see the change automatically.
## Using In-Memory Storage for LokiJS-Style Speed
When the workload truly is "load some data, query it many times, throw it away", swap the storage for the [Memory RxStorage](../../rx-storage-memory.md). The rest of the code stays the same.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
const cache = await createRxDatabase({
name: 'cache',
storage: getRxStorageMemory()
});
await cache.addCollections({
products: {
schema: {
title: 'products schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
price: { type: 'number' }
}
}
}
});
await cache.products.bulkInsert([
{ id: 'p1', name: 'Pen', price: 2 },
{ id: 'p2', name: 'Notebook', price: 6 }
]);
const cheap = await cache.products
.find({ selector: { price: { $lt: 5 } } })
.exec();
```
The Memory storage is also useful as a fast tier in front of a persistent [RxCollection](../../rx-collection.md) when you want both speed and durability.
## Why the LokiJS RxStorage Was Removed in RxDB v16
Earlier RxDB versions shipped a `lokijs` RxStorage that wrapped LokiJS as a backing store. It was removed in RxDB version 16. Two reasons drove the decision:
1. LokiJS itself stopped getting fixes for problems that surfaced through RxDB usage, especially around larger datasets and edge cases in the persistence adapters.
2. The use cases the LokiJS storage covered are now served by other built-in storages. For "everything in RAM" there is the [Memory RxStorage](../../rx-storage-memory.md). For "persistent in the browser" there is the [IndexedDB RxStorage](../../rx-storage-indexeddb.md), the [OPFS RxStorage](../../rx-storage-opfs.md), and the [Dexie RxStorage](../../rx-storage-dexie.md). Each of these is faster, safer, and actively maintained.
If you previously used the LokiJS RxStorage, the migration is to pick whichever of those storages matches your durability needs and switch the `storage` option of `createRxDatabase`. Schemas and collection definitions stay the same.
## FAQ
LokiJS is no longer actively maintained, and bugs that affected RxDB users were not getting fixed upstream. RxDB v16 removed the LokiJS storage and points users at the Memory storage for in-memory use and at IndexedDB, OPFS, or Dexie for persistent browser storage.
Yes. The [Memory RxStorage](../../rx-storage-memory.md) keeps the dataset in RAM and runs queries against in-memory indexes, which gives the same query latency profile as LokiJS without the broken persistence story.
Activity on the project has been minimal since around 2020. New issues and pull requests sit for long periods. For new projects, treating it as feature-frozen is the safer assumption.
Each write goes through the configured RxStorage, and storages like [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), and [SQLite](../../rx-storage-sqlite.md) persist that write before acknowledging it. There is no autosave interval that can drop committed data when the tab is closed.
Define an [RxCollection](../../rx-collection.md) with a JSON schema that matches your LokiJS collection, read the existing LokiJS data once on startup, and `bulkInsert` it into the RxDB collection. From that point on, write through RxDB and use a [replication plugin](../../replication.md) if you also need to sync the data with a server.
## RxDB vs LokiJS at a Glance
| Capability | LokiJS | RxDB |
| ------------------------- | -------------------------------------------- | -------------------------------------------------------------- |
| Primary storage model | In-memory object graph | Pluggable RxStorage (IndexedDB, OPFS, Dexie, SQLite, Memory) |
| Durability of writes | Periodic snapshot via adapter | Per-write durability through the underlying engine |
| In-memory mode | Default | Optional via [Memory RxStorage](../../rx-storage-memory.md) |
| Query API | MongoDB-style | MongoDB-style with observable [RxQuery](../../rx-query.md) |
| Reactivity | Events, no observable queries | RxJS observables on every query and document |
| Multi-tab support | Fragile, snapshot collisions | Coordinated writes across tabs |
| Replication / sync | Not built in | Built-in [sync engine](../../replication.md), HTTP, GraphQL, CouchDB, WebRTC, Firestore |
| Conflict resolution | None | Custom [conflict handlers](../../transactions-conflicts-revisions.md) with revisions |
| Schema and migrations | Optional, ad-hoc | JSON schema with versioned migrations |
| Project activity | Low since around 2020 | Actively maintained |
## Follow Up
If LokiJS got you most of the way and stopped being enough once persistence, multi-tab safety, or sync entered the picture, RxDB is the natural next step. It keeps the document-store ergonomics, adds reactive queries, lets you choose between durable and in-memory storages per use case, and gives you a real replication protocol for the day your app stops being a single-device toy.
For more on the broader direction RxDB is built around, see [the local-first future](../../articles/local-first-future.md).
---
## RxDB as a LowDB Alternative for Node.js and Beyond
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a LowDB Alternative for Node.js and Beyond
[LowDB](https://github.com/typicode/lowdb) is a small JSON file database that fits a specific niche: tiny CLIs, prototypes, configuration stores, and demo servers. The API is straightforward, the data sits in a single `db.json` file, and the whole library is a few kilobytes. For projects where the data set stays small and the access pattern is single-process and synchronous-feeling, LowDB does the job.
Teams tend to outgrow LowDB once the application requires any of the following:
- Reactive UI updates when data changes.
- Replication between clients, servers, or peers.
- Schema validation enforced at write time.
- Concurrent access from multiple processes or threads.
- Storage that scales past a JSON blob loaded fully into memory.
This page walks through where LowDB fits, where it does not, and how [RxDB](https://rxdb.info/) covers the same ground while adding the features needed for production-sized Node.js, Electron, and web applications.
## A Short History of LowDB
LowDB was created by Typicode, the author of the popular `json-server` project. It was released in 2014 as a way to give Node.js scripts a database-like API without running a separate process. The original versions used [Lodash](https://lodash.com/) chains as the query language: developers wrote `db.get('posts').find({ id: 1 }).value()` and Lodash handled the filtering against an in-memory copy of the JSON file.
Over the years LowDB picked up adapters for browsers, atomic file writes, and TypeScript types. Version 5 dropped CommonJS support and shipped as ESM only, which aligned the project with modern Node.js but cut off some legacy users. The data model stayed the same: load a JSON document into memory, mutate it through a JavaScript proxy, and write it back to disk on save.
That model is the source of both LowDB's appeal and its limits. It is easy to read and easy to debug because the database is a plain file. It is also bound by the size of process memory, the cost of full-file writes, and the absence of any change feed.
## What is RxDB?
[RxDB](https://rxdb.info/) is a [local-first](../../articles/local-first-future.md) database for JavaScript. It runs in browsers, Node.js, Electron, React Native, and Deno, and stores data through a pluggable storage layer that can sit on top of IndexedDB, OPFS, SQLite, MongoDB, or in-memory engines. Queries are reactive by default: a query observable emits a new result set whenever a matching document changes, no matter which tab, process, or remote peer wrote the change.
The feature set centers on three ideas:
- A typed [collection](../../rx-collection.md) backed by a [JSON schema](../../rx-schema.md).
- A [reactive query](../../rx-query.md) layer powered by [RxJS](../../reactivity.md).
- A pluggable [replication](../../replication.md) protocol that can sync with HTTP, GraphQL, WebRTC, CouchDB, Firestore, and custom servers.
That combination matches the points where LowDB users tend to hit a wall.
## Where LowDB Falls Short
### One JSON File Does Not Scale
LowDB reads the entire database into memory and rewrites the full JSON file on save. A few hundred records work fine. A few hundred thousand records produce slow startup, slow writes, and high memory use. There are no indexes; every query is a linear scan over JavaScript arrays.
### No Change Observability
LowDB has no event bus and no change feed. If a CLI writes a record, a connected UI cannot find out unless it polls the file. RxDB exposes [observable queries and document streams](../../reactivity.md), so any subscriber sees the new state without polling.
### No Replication
LowDB is a single-node store. Sharing data across two devices, two processes, or a server and a browser requires building a transport on top. RxDB ships with a [replication protocol](../../replication.md) and adapters for HTTP, GraphQL, WebSocket, WebRTC, CouchDB, and Firestore.
### No Schema Validation
LowDB types come from the TypeScript generic the developer provides at construction time. There is no runtime check, so a malformed write reaches the file. RxDB validates every write against the [collection schema](../../rx-schema.md), which catches bad data before it lands on disk.
### No Multi-Process Safety
Two Node.js processes opening the same LowDB file race on read and write. The last writer wins and silently overwrites the other process's changes. RxDB storages such as the SQLite or IndexedDB adapters coordinate writes, and the [multi-instance support](../../rx-collection.md) broadcasts changes between tabs and workers.
The numbers reflect this. As of July 30, 2026, [LowDB](https://github.com/typicode/lowdb) has 22,567 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## What RxDB Adds
- **Scalable storages on Node.js.** Pick from SQLite, MongoDB, in-memory, or filesystem-backed engines. See [Node.js database options](../../nodejs-database.md).
- **Schema validation.** Enforce types, ranges, and required fields with [JSON Schema](../../rx-schema.md).
- **Reactive queries.** Subscribe with `.$` and receive new results whenever the underlying data changes. See [RxQuery](../../rx-query.md) and [reactivity](../../reactivity.md).
- **Replication.** Sync to your own backend or a peer with the [replication protocol](../../replication.md).
- **MongoDB-style query operators.** Use `$gt`, `$in`, `$regex`, and the rest of the Mango query set.
- **Offline-first behavior.** Reads and writes work without a network. See [offline-first](../../offline-first.md).
- **Observable changes.** Every collection exposes an event stream of inserts, updates, and deletes.
## Code Sample: A LowDB Lodash Chain in RxDB
A typical LowDB read against a `posts` array filters by author and sorts by date:
```ts
// LowDB
import { JSONFilePreset } from 'lowdb/node';
const db = await JSONFilePreset('db.json', { posts: [] });
const recentByAuthor = db.data.posts
.filter(p => p.author === 'alice')
.sort((a, b) => b.createdAt - a.createdAt)
.slice(0, 10);
```
The same query in RxDB uses the [RxQuery](../../rx-query.md) API:
```ts
// RxDB
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'blog',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
author: { type: 'string', maxLength: 100 },
title: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'author', 'createdAt'],
indexes: ['author', 'createdAt']
}
}
});
const recentByAuthor = await db.posts.find({
selector: { author: 'alice' },
sort: [{ createdAt: 'desc' }],
limit: 10
}).exec();
```
The schema enforces field types, the index makes the author and date filter fast, and the query result is identical in shape to the LowDB output.
## Code Sample: Subscribing to Live Query Results
LowDB has no equivalent for this. In RxDB the same query becomes a stream:
```ts
const query = db.posts.find({
selector: { author: 'alice' },
sort: [{ createdAt: 'desc' }],
limit: 10
});
query.$.subscribe(results => {
console.log('latest posts by alice:', results.map(r => r.title));
});
// Any insert, update, or delete that affects the result set
// triggers a new emission, even from another tab or process.
await db.posts.insert({
id: 'p1',
author: 'alice',
title: 'Hello',
createdAt: Date.now()
});
```
A CLI dashboard, an Electron window, or a React component can bind directly to `query.$` and stay in sync with the database without manual reload logic.
## When LowDB is Still the Right Pick
LowDB stays a sensible choice when the project meets all of these conditions:
- The data set is small, in the order of a few thousand records or less.
- One process owns the file at a time.
- No client other than the writing process needs to react to changes.
- No replication, sync, or multi-device support is on the roadmap.
- The team values a zero-configuration, single-file database for prototyping or a tiny CLI config store.
For a `git`-style config file, a personal todo CLI, or a fixture file used in tests, LowDB stays lean and pleasant to use. For anything that grows past one process or one screen, RxDB is built for the next step.
## FAQ
No. RxDB runs in a single file with the in-memory storage and adds about a minute of setup. The schema and reactive query features pay off as soon as a UI binds to the data, which usually happens early in a prototype. Teams that start with RxDB avoid rewriting the data layer once the prototype turns into a product.
RxDB does not store one big JSON document, but the [import and export helpers](../../rx-database.md) serialize a database to JSON for backup or seeding. For a `db.json` style fixture used in tests, the export output is a drop-in equivalent. For runtime storage, an indexed engine like SQLite or IndexedDB performs better than re-writing a JSON file on every change.
Yes. RxDB runs in [Node.js](../../nodejs-database.md) with adapters for SQLite, MongoDB, filesystem, and in-memory storage. A Node.js process can act as a server, a CLI, or a worker, and replicate with browser clients through HTTP, GraphQL, or WebSocket transports.
LowDB's core is a couple of kilobytes because it does very little. RxDB's core ships around 50 KB gzipped with reactive queries, schema validation, and the replication protocol included. Plugins are tree-shakeable, so an app only pays for the storages and replication adapters it imports.
## Comparison Table
| Feature | LowDB | RxDB |
| --- | --- | --- |
| Storage model | Single JSON file in memory | Pluggable: IndexedDB, OPFS, SQLite, MongoDB, in-memory, filesystem |
| Query language | Lodash chains and array methods | MongoDB-style selectors with `$gt`, `$in`, `$regex`, sort, limit |
| Indexes | None | Declared in schema, used by the query planner |
| Schema validation | TypeScript generics only | Runtime [JSON Schema](../../rx-schema.md) validation |
| Reactivity | None | [Observable queries and documents](../../reactivity.md) |
| Replication | None | HTTP, GraphQL, WebSocket, WebRTC, CouchDB, Firestore |
| Multi-process safety | Last writer wins | Coordinated through the storage layer and event bus |
| Offline-first | Yes, by default file-based | Yes, designed for [offline-first](../../offline-first.md) |
| Runtimes | Node.js, browser (with adapters) | Node.js, browser, Electron, React Native, Deno, Bun |
| Bundle size | A few KB | About 50 KB gzipped core, tree-shakeable plugins |
| Best fit | CLI configs, prototypes, tiny demos | Production apps that need reactivity, sync, or scale |
For a deeper look at the building blocks, see the [RxCollection guide](../../rx-collection.md), the [RxSchema reference](../../rx-schema.md), and the [replication overview](../../replication.md). For Node.js specific choices, the [Node.js database page](../../nodejs-database.md) covers the available storages.
---
## RxDB as a Meteor Alternative - Offline-First Without Framework Lock-In
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a Meteor Alternative
Meteor launched in 2012 and introduced many developers to the concept of full-stack JavaScript with real-time data. For a long time, it stood out as one of the few platforms that offered seamless data synchronization between client and server out of the box. Over a decade later, the JavaScript ecosystem has matured, and the requirements for offline-capable, framework-agnostic applications have grown. RxDB offers a different approach: a dedicated local-first database that works with any backend, any frontend framework, and any storage engine, without requiring a full platform.
This page compares Meteor and RxDB in detail so you can decide which tool fits your use case.
---
## What is Meteor?
Meteor is a full-stack JavaScript platform first released by Meteor Development Group in 2012. It was acquired by Tiny Capital in 2019 and remains under active development. The framework provides an integrated solution covering a server runtime (Node.js), a build system, a package manager (Atmosphere), and a data synchronization layer built around MongoDB.
Meteor's central idea is "data on the wire": instead of sending rendered HTML from the server, only data is transmitted, and the client re-renders reactively. On the server, Meteor uses MongoDB. On the client, it uses a library called Minimongo, which is an in-memory JavaScript implementation of the MongoDB query interface. A WebSocket-based protocol called DDP (Distributed Data Protocol) keeps the two sides synchronized in real time.
When a user writes to the client-side Minimongo database, the change is immediately applied locally for a fast, optimistic UI response ("latency compensation"), and a corresponding method call is sent to the server simultaneously. If the server rejects the change, the local state is rolled back.
### A Brief History
- **2012** - Meteor is introduced and quickly attracts attention due to its developer-friendly, real-time-by-default model.
- **2014-2016** - Peak popularity; Meteor raised $31.2 million in funding. The community grew rapidly.
- **2016** - Funding from the VC company runs out. The team pivots toward the Galaxy cloud hosting product.
- **2019** - Tiny Capital acquires Meteor. Development continues under new ownership.
- **2024** - Meteor 3.0 is released, removing the long-standing dependency on Fibers (a synchronous async abstraction) in favor of native `async/await`. Build tooling is modernized with support for Vite and Rspack.
- **2025-2026** - Meteor 3.x continues to receive updates. The project is maintained but occupies a much smaller share of the JavaScript ecosystem compared to its peak.
Meteor's GitHub star count reflects this history. It accumulated a large following during the 2014-2016 era, but new stars and activity have slowed significantly compared to other tools in the space. The framework is not abandoned, but it is also not seeing the level of adoption growth it once did. Most new JavaScript projects choose React or Vue with a separate API layer, rather than reaching for an integrated full-stack framework like Meteor.
---
## How Meteor Handles Data and Offline Use
Meteor's data model is designed around the assumption that the server is always the primary source of truth. Minimongo on the client is a cache and an optimistic layer, not a standalone database.
When a user goes offline in a Meteor app, the following happens:
1. The DDP WebSocket connection is lost.
2. Meteor queues method calls and attempts to reconnect automatically.
3. The in-memory Minimongo data that was already loaded remains available for display.
4. No new data can be fetched from the server.
5. If the user refreshes the browser or closes the tab, all in-memory state is lost.
Step 5 is the critical limitation. By default, Meteor applications lose their local data on a page reload because Minimongo stores everything in memory. The app must reconnect and re-fetch data from the MongoDB server before it can render again.
Community packages like `GroundDB` and `meteor-persistent-minimongo2` attempted to solve this by persisting Minimongo to IndexedDB. However, these packages are no longer maintained. The official Meteor ecosystem provides limited first-party support for persistent offline storage, and the workarounds require significant custom development.
Meteor 3.x introduced a community package for offline support (`jam:offline`), which provides IndexedDB persistence for Minimongo. It is an improvement, but it is not built into the core framework and adds complexity to the setup.
---
## How RxDB Handles Data and Offline Use
RxDB takes the opposite approach: the local database is the primary data store, and server synchronization is a background operation. Every read and write goes through the local database first.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
messages: {
schema: {
title: 'message schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
author: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'text', 'author', 'createdAt'],
indexes: ['createdAt']
}
}
});
```
When this database is created, data is stored in IndexedDB on the user's device. If the user closes the browser and reopens the app with no internet connection, the app can read all locally stored data immediately. No server round-trip is required to render the interface.
Replication with a backend server happens separately and in the background:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: db.messages,
replicationIdentifier: 'my-messages-replication',
pull: {
handler: async (checkpointOrNull, batchSize) => {
const response = await fetch(
`/api/messages/pull?checkpoint=${JSON.stringify(checkpointOrNull)}` +
`&limit=${batchSize}`
);
return response.json();
}
},
push: {
handler: async (docs) => {
const response = await fetch('/api/messages/push', {
method: 'POST',
body: JSON.stringify(docs),
headers: { 'Content-Type': 'application/json' }
});
return response.json(); // return conflicting documents if any
}
},
live: true
});
replicationState.error$.subscribe(err => console.error('Replication error:', err));
```
If the network is unavailable, the replication state waits and retries automatically. All local writes succeed immediately and are queued for the next successful sync cycle.
---
## Framework Integration
Meteor was designed as an all-in-one platform. Integrating it with non-Meteor frontend frameworks has historically been difficult. Community projects like `angular-meteor` and `vue-meteor` provided bridges, but they required keeping up with two separate ecosystems simultaneously, and many of these integrations fell behind as their respective frameworks evolved.
React became the officially supported frontend in Meteor over time, but using Meteor with Vue 3, Svelte, SolidJS, or other modern frameworks still requires significant manual integration work.
RxDB is a library, not a platform. It runs wherever JavaScript runs and has no opinion about the frontend layer. You can use it with any framework:
**React example using RxDB observables:**
```tsx
import { useRxData } from 'rxdb-hooks';
function MessageList() {
const { result: messages, isFetching } = useRxData('messages', collection =>
collection.find().sort({ createdAt: 'asc' })
);
if (isFetching) return Loading...;
return (
{messages.map(msg => (
{msg.author}: {msg.text}
))}
);
}
```
**Vanilla JavaScript with RxJS observables:**
```ts
const subscription = db.messages
.find()
.sort({ createdAt: 'asc' })
.$
.subscribe(messages => {
renderMessages(messages);
});
```
The `$` property on any RxDB query returns an RxJS Observable that emits a new result set whenever the underlying data changes. This reactive model works with React, Vue, Angular, Svelte, SolidJS, or plain JavaScript without needing framework-specific plugins.
---
## Backend Flexibility
Meteor is tightly coupled to MongoDB. While there are community projects for connecting Meteor to other databases, MongoDB is the officially supported and recommended backend. The DDP protocol was designed specifically to work with MongoDB's document model and Minimongo on the client side.
This means if your backend uses PostgreSQL, MySQL, or a custom API, you will need to either introduce MongoDB into your stack or accept significant workarounds.
RxDB is backend-agnostic. The replication protocol is based on a simple pull/push interface that you implement yourself or with one of the provided plugins:
- [CouchDB replication](../../replication-couchdb.md) for syncing with CouchDB or compatible servers
- [GraphQL replication](../../replication-graphql.md) for syncing via GraphQL endpoints
- [HTTP replication](../../replication-http.md) for custom REST APIs
- [WebSocket replication](../../replication-websocket.md) for server-push patterns
- [Firestore replication](../../replication-firestore.md) for Google Firestore backends
- [WebRTC replication](../../replication-webrtc.md) for peer-to-peer sync without a central server
You can also implement a [custom replication handler](../../replication.md) that communicates with any protocol your existing backend speaks. If you have a legacy PostgreSQL database with a REST API, RxDB can sync with it without requiring any changes to your backend schema.
---
## Storage Engine Choices
Meteor's client storage is Minimongo, which is in-memory by default. The underlying persistence story for Minimongo requires additional packages and does not support switching between storage backends without significant code changes.
RxDB has a modular storage system. You choose the storage engine that fits your deployment target:
| Environment | Recommended Storage |
|---|---|
| Browser (general) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high performance) | [OPFS](../../rx-storage-opfs.md) |
| React Native | [SQLite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [Filesystem / SQLite](../../rx-storage-sqlite.md) |
| Memory (testing) | [Memory](../../rx-storage-memory.md) |
| Multi-tab browser apps | [SharedWorker](../../rx-storage-shared-worker.md) |
Switching storage engines typically requires only changing the `storage` parameter when creating the database:
```ts
// Browser with IndexedDB
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
// React Native with SQLite
import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSQLite({ sqliteBasics })
});
```
This abstraction means you can write application code once and run it across browser, mobile, and desktop without rewriting your database interactions.
---
## Observable Queries and Reactive UI
Meteor's reactive system is built around its own reactive computation model (Tracker). Tracker-based code runs inside reactive contexts, and Meteor automatically re-runs computations when their dependencies change. While powerful for Meteor's own ecosystem, Tracker is Meteor-specific and does not integrate with the broader JavaScript reactive ecosystem.
RxDB's reactive system is built on [RxJS](https://rxjs.dev), which is one of the most widely used reactive programming libraries in JavaScript. Every query, document, and field in RxDB exposes RxJS Observables:
```ts
// Subscribe to all messages from a specific author
const authorMessages$ = db.messages
.find({ selector: { author: 'alice' } })
.sort({ createdAt: 'asc' })
.$;
authorMessages$.subscribe(messages => {
console.log('Alice has', messages.length, 'messages');
});
// Subscribe to a single document and watch one field
const doc = await db.messages.findOne('msg-001').exec();
doc.get$('text').subscribe(newText => {
console.log('Text changed to:', newText);
});
```
These observables integrate naturally with React hooks (`rxdb-hooks`), Angular's async pipe, Vue's reactivity system via `from()`, or any other tool that understands RxJS.
---
## Conflict Handling
In Meteor, the server is the authority. If a client-side Minimongo write conflicts with a server write, the server state wins and the client rolls back. Meteor does not provide mechanisms for merging concurrent changes or for handling conflicts that arise when users have been working offline for extended periods.
RxDB includes a configurable [conflict resolution system](../../transactions-conflicts-revisions.md). Every document has an associated revision, and when two versions of the same document arrive from different sources, RxDB calls your conflict handler to decide the outcome:
```ts
await db.addCollections({
messages: {
schema: messageSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Merge by taking the most recently updated version
if (newDocumentState.updatedAt > realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
You can implement any merge strategy, from simple last-write-wins to full three-way merges or even CRDT-based approaches. This is essential for applications where users might be offline for hours or days and need their changes reconciled safely when they reconnect.
RxDB also supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md) natively, which provide automatic, deterministic conflict resolution for common data structures like counters and sets without requiring custom logic.
---
## Multi-Tab and Multi-Window Support
Meteor apps run one WebSocket connection per browser tab, with each tab having its own Minimongo instance. There is no built-in coordination between multiple tabs of the same app.
RxDB includes built-in [multi-tab support](../../rx-storage-shared-worker.md). When using the SharedWorker storage, all open tabs share a single database instance running in a shared worker. Changes made in one tab are immediately visible in all other open tabs without additional configuration:
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
All tabs observe the same data stream, so a change in tab A appears immediately in the subscriptions of tab B, C, and D.
---
## Performance
Meteor's Minimongo is an in-memory database. In-memory operations are fast, but the entire dataset must fit in RAM, and the data is lost on page reload. For large datasets (tens of thousands of documents), the memory overhead and startup cost of re-fetching everything from the server become significant.
RxDB stores data persistently and indexes it for fast lookups. Queries operate on locally indexed data rather than scanning in-memory arrays. With the [OPFS storage](../../rx-storage-opfs.md) (Origin Private File System), RxDB achieves particularly high read and write throughput in modern browsers without needing WebAssembly.
RxDB also includes [event-reduce](https://github.com/pubkey/event-reduce) optimization, which reduces the number of re-queries by computing the new query result from the change event directly, without re-running the full query against the storage engine.
---
## Mobile Support
Meteor is primarily a web and Node.js framework. There is a Cordova integration for wrapping Meteor apps as mobile apps, but it is a wrapper rather than a native approach. React Native is not officially supported.
RxDB runs in React Native natively using the [SQLite storage plugin](../../rx-storage-sqlite.md) or the memory storage. The same database code runs on iOS and Android without any wrappers. This means you can share business logic, replication code, and schema definitions between your web and mobile apps.
---
## Summary: Key Differences
| Aspect | Meteor | RxDB |
|---|---|---|
| **Type** | Full-stack platform | Client-side database library |
| **Backend requirement** | MongoDB (required) | Any backend or none |
| **Offline persistence** | In-memory by default; persistence via add-on packages | IndexedDB/SQLite/OPFS natively |
| **Offline durability** | Data lost on page reload without extra packages | Data persists across reloads natively |
| **Replication protocol** | DDP (Meteor-specific) | HTTP, CouchDB, GraphQL, WebSocket, WebRTC, custom |
| **Conflict handling** | Server-wins; no merge support | Configurable conflict handlers, CRDT support |
| **Frontend frameworks** | React officially supported; others require wrappers | Any framework (React, Vue, Angular, Svelte, plain JS) |
| **Storage engines** | Minimongo (in-memory) | IndexedDB, OPFS, SQLite, Memory, SharedWorker |
| **Reactive model** | Meteor Tracker (proprietary) | RxJS Observables (ecosystem standard) |
| **Multi-tab coordination** | No built-in support | SharedWorker storage shares state across tabs |
| **Mobile (native)** | Cordova wrapper only | React Native with SQLite storage |
| **Open source** | Yes | Yes |
| **License** | MIT | Apache 2.0 |
---
## Getting Started with RxDB
Install RxDB and RxJS:
```bash
npm install rxdb rxjs
```
Create a database and a collection:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'my-local-app',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'text', 'done', 'updatedAt'],
indexes: ['updatedAt']
}
}
});
// Insert a document
await db.todos.insert({
id: 'todo-1',
text: 'Learn RxDB',
done: false,
updatedAt: Date.now()
});
// Subscribe to all todos reactively
db.todos.find().sort({ updatedAt: 'asc' }).$.subscribe(todos => {
console.log('Current todos:', todos.map(t => t.text));
});
```
From here, you can add [replication](../../replication.md) to sync with any backend, connect a frontend framework, and deploy to web, desktop, or mobile using the same codebase.
---
## FAQ
Migration is not a drop-in replacement because RxDB and Meteor have different data models and protocols. However, the general approach is:
1. Export your MongoDB collections to a format your backend REST or GraphQL API can serve.
2. Replace Minimongo usage in your frontend with RxDB collections and queries.
3. Implement a replication handler in RxDB that pulls from and pushes to your existing API.
4. Replace Meteor's Tracker reactive computations with RxJS subscriptions or the equivalent in your frontend framework.
The migration can be done incrementally if you wrap RxDB behind the same service layer that previously called Meteor methods.
No. RxDB works entirely offline with no server. You create a local database, insert and query documents, and use subscriptions to react to changes. Adding replication is optional and requires you to provide pull and push handlers that connect to a server of your choice. Many applications start with a local-only setup and add replication later.
When the user reconnects, RxDB runs a full replication cycle. It pulls all documents changed on the server since the last successful checkpoint, and pushes all local changes that were written while offline. If a document was changed on both sides, RxDB calls your [conflict handler](../../transactions-conflicts-revisions.md) to resolve the discrepancy. This cycle works correctly whether the user was offline for five minutes or five weeks.
Yes. RxDB uses indexed storage engines rather than in-memory arrays, which means query performance does not degrade linearly with collection size. The OPFS storage backend, in particular, is designed for high read and write throughput. For very large datasets, you can define compound indexes on the fields you query most frequently to keep lookups fast.
---
## RxDB as a Minimongo Alternative - Persistent, Observable, Offline-First
import {Faq, FaqItem} from '@site/src/components/faq';
import {Timeline} from '@site/src/components/timeline';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Minimongo Alternative
Minimongo started as a component inside Meteor.js in 2012 and introduced many developers to the idea of a client-side, MongoDB-compatible data layer. If you have used Minimongo outside of Meteor or are evaluating it for a new project, this page explains what Minimongo does well, where it falls short, and why RxDB is a strong alternative for building offline-first, reactive JavaScript applications.
---
## What is Minimongo?
Minimongo is a client-side, in-memory implementation of the MongoDB query interface written in JavaScript. It was created as part of the Meteor.js framework to enable a programming model called **latency compensation**: when a user performs an action, the client applies the change immediately to its local Minimongo collection, the UI updates instantly, and a corresponding write is sent to the server in the background. If the server rejects the write, the local state is rolled back.
The main strength of Minimongo is its familiar MongoDB-like API. Developers who know MongoDB can use `find`, `insert`, `update`, and `remove` on the client with the same selector syntax they use on the server.
The standalone `mWater/minimongo` package (forked from Meteor in January 2014) added support for persistent storage backends such as IndexedDB, WebSQL, and LocalStorage. This gave Minimongo a life outside the Meteor ecosystem, but this standalone package has not received active maintenance for years and should not be used in new projects.
### A Brief Timeline
- **2012** - Minimongo is introduced as a core module in Meteor.js, enabling the DDP (Distributed Data Protocol) data sync layer.
- **2014** - The `mWater/minimongo` project forks the Meteor code to make it usable as an npm package outside of Meteor. It adds geospatial query support and storage adapters.
- **2016-2020** - Minimongo within Meteor continues to receive updates as part of the larger framework. The standalone fork sees decreasing maintenance activity.
- **2024-2025** - Within Meteor 3.x, Minimongo is still shipped as the client-side cache layer. Outside of Meteor, the standalone fork is effectively unmaintained. New projects that need a client-side MongoDB-like database generally reach for more capable tools.
### Where Minimongo Is Used Today
Minimongo is still in active use, but only as part of the Meteor framework stack. As a standalone library independent of Meteor, it is rarely a recommended choice. The project's GitHub repository for the standalone fork shows no recent releases and accumulating open issues with no responses.
For developers not using Meteor, Minimongo provides a familiar query syntax but lacks the infrastructure needed for production offline-first applications: no persistent storage by default, no observable queries, no revision-based conflict handling, and no built-in replication protocol.
---
## How Minimongo Works
In the Meteor stack, Minimongo acts as a local mirror of data published from the server. The server defines **publications**, which are filtered subsets of MongoDB collections. The client **subscribes** to these publications, and Meteor's DDP protocol streams the documents to the client's Minimongo collection over a WebSocket connection.
```javascript
// Server-side: a Meteor publication
Meteor.publish('recentPosts', function () {
return Posts.find({}, { sort: { createdAt: -1 }, limit: 50 });
});
// Client-side: subscribing and querying Minimongo
Meteor.subscribe('recentPosts');
const posts = Posts.find({ category: 'news' }).fetch();
```
On the client, the `Posts` variable points to a Minimongo collection. Operations like `find()` run entirely in memory against the local cache. Writes go to Minimongo first for an optimistic result, then propagate to the server.
When you use Minimongo outside of Meteor, you lose the DDP layer. You are left with an in-memory store that you must populate and sync manually. There is no standard protocol or built-in mechanism to keep that store synchronized with any backend.
---
## Key Limitations of Minimongo
### No Persistent Storage by Default
The core Minimongo implementation stores all documents in memory. If the user closes or refreshes the browser tab, all data is gone. The application must reconnect to the server and re-fetch all data before it can function again.
Some storage adapters (IndexedDB, LocalStorage) exist in the standalone `mWater` fork, but the implementation of these adapters is not well-maintained. For production applications, relying on them introduces risk.
This is the most critical limitation for any use case that requires offline-first behavior. An offline-first application must be able to start, read data, and accept writes when the user has no internet connection at all, including after a browser refresh. Minimongo cannot guarantee this without significant additional work.
### No Observable Queries
Minimongo does not expose a native observable or reactive query interface. Within Meteor, reactivity is provided by **Tracker**, Meteor's own dependency-tracking system. A Tracker reactive computation re-runs when its reactive data sources change. This reactivity is entirely specific to the Meteor ecosystem and is not compatible with RxJS, Vue's reactivity, React, or any other standard JavaScript reactive primitive.
If you are using Minimongo standalone, you have no automatic notification when data in a collection changes. You must poll or implement a custom event system yourself.
### No Document Revisions or Conflict Handling
Minimongo stores the current state of each document indexed by its `_id`. There is no concept of a document revision, no version vector, and no mechanism for detecting write conflicts. If two clients modify the same document while one of them is offline, the Minimongo data model has no way to represent both versions or to help the application choose between them.
In Meteor's DDP model, the server always wins. When the client comes back online and the server applies a conflicting state, the local document is silently overwritten. This works for simple collaborative use cases but fails for applications where users work offline for extended periods and need their changes preserved.
### Partial MongoDB Query Support
Minimongo implements a subset of the MongoDB query language. Several operators and features available in MongoDB are missing or only partially supported in Minimongo:
- No aggregation pipeline (`$lookup`, `$group`, `$facet`, `$unwind`)
- No multi-document transactions
- Limited secondary index support
- Some query operators behave differently from their server-side equivalents
This means that query code written against Minimongo cannot always be used directly with MongoDB on the server, and vice versa.
### No Multi-Tab Support
Each browser tab running a Minimongo-based application maintains its own in-memory store. There is no coordination between tabs. A write made in tab A is not visible in tab B until both tabs re-fetch from the server. For applications where users might have multiple tabs open simultaneously, this leads to inconsistent data views.
---
The numbers reflect this. As of July 30, 2026, [Minimongo](https://github.com/mWater/minimongo) has 1,212 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `minimongo` package was downloaded 13,810 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/minimongo-vs-rxdb)). Development has slowed down as well: the newest commit on the `master` branch of the [mWater/minimongo repository](https://github.com/mWater/minimongo/commits/master/) is from October 2025, .
## How RxDB Solves These Problems
[RxDB](https://rxdb.info) is a local-first JavaScript database that treats the local store as the primary data source. Every read and write happens locally first, and replication with a backend server runs in the background. The database persists to the chosen storage engine, so data survives page refreshes and browser restarts without any network connection.
### Persistent Storage Across Environments
RxDB has a pluggable storage system. You choose the storage engine that matches your deployment:
| Environment | Storage Option |
|---|---|
| Browser | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high-throughput) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [Filesystem / SQLite](../../rx-storage-sqlite.md) |
| Tests | [Memory](../../rx-storage-memory.md) |
| Multi-tab browsers | [SharedWorker](../../rx-storage-shared-worker.md) |
Switching storage engines requires changing only the `storage` parameter when creating the database. All application code above that layer remains the same:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
category: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'title', 'category', 'createdAt'],
indexes: ['createdAt', 'category']
}
}
});
```
After this setup, all data written to `db.posts` is stored persistently in IndexedDB. If the user goes offline, refreshes the browser, or restarts the device, the data is available immediately on the next load with no server round-trip required.
### Observable Queries with RxJS
RxDB builds its reactive layer on [RxJS](https://rxjs.dev), which is one of the most widely adopted reactive programming libraries in the JavaScript ecosystem. Every query in RxDB exposes an Observable via the `$` property. The Observable emits the current result set immediately on subscription and re-emits whenever the underlying data changes.
```ts
// Subscribe to all posts in the 'news' category, ordered by date
const newsPosts$ = db.posts
.find({
selector: { category: 'news' },
sort: [{ createdAt: 'desc' }]
})
.$;
newsPosts$.subscribe(posts => {
console.log('News posts updated:', posts.length);
renderPostList(posts);
});
```
Every time a post is inserted, updated, or deleted, this subscription fires automatically. There is no polling, no manual cache invalidation, and no framework-specific wiring required. The same subscription works in React, Vue, Angular, Svelte, SolidJS, or plain JavaScript.
RxDB also exposes observables at the document and field level:
```ts
// Watch a single document's title field
const doc = await db.posts.findOne('post-001').exec();
doc.get$('title').subscribe(newTitle => {
console.log('Title changed to:', newTitle);
});
// Watch the entire collection for any change
db.posts.$.subscribe(changeEvent => {
console.log('Change event:', changeEvent.operation, changeEvent.documentId);
});
```
This granular reactivity makes it straightforward to build UIs that reflect the current data state without writing any manual refresh logic.
RxDB also optimizes query re-execution using the [event-reduce](https://github.com/pubkey/event-reduce) algorithm. When a document is inserted, updated, or deleted, RxDB checks whether the result set of existing queries can be updated from the change event alone, without re-running the full query against the storage engine. This reduces the number of storage reads significantly in write-heavy scenarios.
### Document Revisions and Conflict Handling
RxDB tracks every document's revision history. Each write operation attaches a revision identifier to the document, and the replication layer uses these revisions to detect and resolve conflicts between the local database and the server.
When two versions of the same document exist (one local, one from the server), RxDB calls a configurable **conflict handler** to decide what happens:
```ts
await db.addCollections({
posts: {
schema: postSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Strategy: keep the version with the most recent updatedAt timestamp
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
You can implement any conflict resolution strategy your application requires: last-write-wins, field-level merging, user-prompted resolution, or automatic reconciliation via CRDTs. RxDB natively supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md), which resolve conflicts automatically and deterministically without requiring custom handler logic:
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const counterSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
// The CRDT field stores the operation log for automatic merging
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDT support, RxDB can merge concurrent edits from multiple clients automatically, making it well-suited for collaborative editing use cases that Minimongo cannot handle without significant custom code.
### Replication with Any Backend
Minimongo's replication is tied to Meteor's DDP protocol, which requires a Meteor server with a MongoDB backend. Standalone Minimongo has no built-in replication at all.
RxDB's replication is backend-agnostic. The replication system uses a simple pull/push interface that you implement against any server:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: db.posts,
replicationIdentifier: 'posts-replication-v1',
pull: {
handler: async (checkpoint, batchSize) => {
const response = await fetch(
`/api/posts/pull?since=${checkpoint?.updatedAt ?? 0}` +
`&limit=${batchSize}`
);
return response.json(); // { documents: [...], checkpoint: {...} }
}
},
push: {
handler: async (rows) => {
const response = await fetch('/api/posts/push', {
method: 'POST',
body: JSON.stringify(rows),
headers: { 'Content-Type': 'application/json' }
});
return response.json(); // return conflicting documents if any
}
},
live: true,
retryTime: 5000
});
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
```
If the network goes down, the replication state retries automatically at the configured interval. Local writes always succeed immediately and are queued for the next successful sync cycle. No data is lost during offline periods.
In addition to custom HTTP replication, RxDB provides ready-made plugins for common backends:
- [CouchDB replication](../../replication-couchdb.md)
- [GraphQL replication](../../replication-graphql.md)
- [Firestore replication](../../replication-firestore.md)
- [WebSocket replication](../../replication-websocket.md)
- [WebRTC peer-to-peer replication](../../replication-webrtc.md)
### Multi-Tab Coordination
RxDB handles multi-tab browser applications with its [SharedWorker storage](../../rx-storage-shared-worker.md). When multiple tabs of the same application are open, they all connect to a single shared database instance running in a SharedWorker. All writes and subscriptions go through this shared instance, so a change made in one tab is immediately visible in all other tabs:
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
After this setup, subscriptions in any tab observe the same unified data stream. No additional code is required to keep tabs in sync.
For environments where SharedWorker is not available, RxDB falls back to using the [BroadcastChannel API](../../rx-storage-indexeddb.md) to propagate change events across tabs, so all open tabs see updates even when using the standard IndexedDB storage.
### Query Capabilities and Indexing
RxDB uses a MongoDB-compatible query syntax for its `find` operations, so developers familiar with Minimongo's query interface can use similar selectors and sort expressions. Unlike Minimongo, RxDB enforces index definitions at schema level, which means the storage engine can use efficient B-tree lookups rather than full collection scans:
```ts
// Schema with compound and single-field indexes
const postSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
category: { type: 'string', maxLength: 50 },
authorId: { type: 'string', maxLength: 100 },
createdAt: { type: 'number' }
},
required: ['id', 'category', 'authorId', 'createdAt'],
indexes: [
'createdAt',
'category',
['category', 'createdAt'] // compound index for category + time queries
]
};
// This query uses the compound index
const results = await db.posts.find({
selector: { category: 'news' },
sort: [{ createdAt: 'desc' }],
limit: 20
}).exec();
```
Minimongo always performs a linear scan over all documents in memory for each query, because it has no index infrastructure. For collections with a few hundred documents this is acceptable, but performance degrades noticeably at thousands of documents.
### Schema Validation and Type Safety
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before it is written to storage. This means data integrity is enforced at the database level, not only in application code:
```ts
// Invalid documents are rejected at insert time
try {
await db.posts.insert({
id: 'post-002',
// 'title' is missing, which is required
category: 'news',
createdAt: Date.now()
});
} catch (err) {
console.error('Validation error:', err.message);
}
```
Minimongo has no built-in schema validation. It stores whatever object you pass to `insert` or `update`. This makes it easier to write incorrect data into the collection, which can cause subtle bugs that are hard to track down.
RxDB also generates TypeScript types automatically from the schema definition, so you get full IDE autocompletion and compile-time type checking for all collection operations.
---
## Positioning: Who Should Switch?
Minimongo is a reasonable choice if you are building a standard Meteor application that does not require long-term offline capability. Within that narrow context, it does exactly what it is designed to do: provide a fast, optimistic client-side cache that mirrors a MongoDB publication.
If you are building anything outside of the Meteor ecosystem, or if your Meteor application needs:
- Data persistence across page reloads without a server connection
- Reactive queries that work with React, Vue, Angular, or plain JavaScript
- Conflict resolution for concurrent offline edits
- Replication with backends other than MongoDB
- Multi-tab state coordination
- Type-safe schemas with validation
...then Minimongo is not the right tool and RxDB covers all of these requirements out of the box.
---
## Getting Started with RxDB
Install RxDB and RxJS:
```bash
npm install rxdb rxjs
```
Create a database, add a collection, and subscribe to reactive queries:
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
// Add the dev mode plugin for schema validation errors during development
addRxPlugin(RxDBDevModePlugin);
const db = await createRxDatabase({
name: 'blogapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
category: { type: 'string' },
authorId: { type: 'string' },
createdAt: { type: 'number' },
updatedAt: { type: 'number' }
},
required: [
'id',
'title',
'category',
'authorId',
'createdAt',
'updatedAt'
],
indexes: ['createdAt', 'category']
}
}
});
// Insert a document
await db.posts.insert({
id: 'post-001',
title: 'Getting Started with Local-First Apps',
category: 'tutorial',
authorId: 'user-42',
createdAt: Date.now(),
updatedAt: Date.now()
});
// Subscribe reactively to all tutorial posts
db.posts.find({
selector: { category: 'tutorial' },
sort: [{ createdAt: 'desc' }]
}).$.subscribe(posts => {
console.log('Tutorial posts:', posts.map(p => p.title));
});
```
After the initial setup, you can add [replication](../../replication.md) to sync with your existing backend, or deploy to React Native using the [SQLite storage plugin](../../rx-storage-sqlite.md) with the same collection schema.
---
## Comparison Summary
| Aspect | Minimongo | RxDB |
|---|---|---|
| **Type** | In-memory client-side cache | Persistent local-first database |
| **Persistence** | In-memory by default; lost on page reload | IndexedDB, OPFS, SQLite, Filesystem natively |
| **Reactive queries** | Only via Meteor Tracker (Meteor-specific) | RxJS Observables (ecosystem standard) |
| **Observable changestream** | Not available standalone | Available on collections, documents, and fields |
| **Conflict handling** | None; server overwrites client | Configurable conflict handlers, CRDT support |
| **Document revisions** | None | Built-in revision tracking for every document |
| **Replication protocol** | DDP (Meteor only) or none standalone | HTTP, CouchDB, GraphQL, WebSocket, WebRTC, custom |
| **Backend requirement** | MongoDB (via Meteor DDP) | Any backend or none |
| **Multi-tab support** | None; each tab has its own in-memory store | SharedWorker for unified cross-tab state |
| **Schema validation** | None built-in | JSON Schema validation on every write |
| **TypeScript support** | Partial | Full (auto-generated types from schema) |
| **Query indexing** | Full collection scan always | Defined indexes; efficient B-tree lookups |
| **Aggregation pipeline** | Not supported | Not built-in; custom computed fields possible |
| **Mobile (React Native)** | Not supported | SQLite storage plugin for iOS and Android |
| **Active maintenance (standalone)** | No (mWater fork unmaintained) | Yes (active development, premium plugin model) |
| **Framework agnostic** | Tied to Meteor ecosystem | Works with React, Vue, Angular, Svelte, plain JS |
| **License** | MIT | Apache 2.0 |
---
## FAQ
Yes. RxDB's replication protocol can communicate with any HTTP server, including a Node.js API backed by MongoDB. You implement pull and push handlers that query your MongoDB API endpoints and return the document format RxDB expects. You do not need to replace your backend to use RxDB on the frontend.
RxDB uses a [MongoDB-compatible query syntax](../../rx-query.md) for selectors, so many queries you have written for Minimongo will work with RxDB with little or no modification. The `selector` field in RxDB queries uses the same operators (`$eq`, `$gt`, `$in`, `$or`, `$and`, etc.) that Minimongo supports. RxDB does not support the full MongoDB aggregation pipeline, but it covers the query patterns needed for client-side filtering, sorting, and pagination.
Yes. RxDB works entirely as a local database with no server. You create a database, write documents, run queries, and subscribe to reactive changes without any network connection required. Replication is optional. You can start with a local-only setup and add a replication layer later when your application requirements grow.
When the device comes back online, RxDB runs a replication cycle. It pulls all documents changed on the server since the last successful checkpoint and pushes all local writes that accumulated while offline. If the same document was changed on both sides, RxDB calls your conflict handler to determine the final state. This process works correctly whether the offline period was five minutes or several weeks.
Yes. RxDB runs on React Native using the [SQLite storage plugin](../../rx-storage-sqlite.md). The same schema definitions, query code, and replication setup you write for your web application can be shared with your React Native application. RxDB also supports [Expo](../../react-native-database.md) through dedicated storage plugins.
---
## RxDB as a MongoDB Realm Alternative After Atlas Device Sync Deprecation
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a MongoDB Realm Alternative After Atlas Device Sync Deprecation
Teams that built mobile and web applications on top of [MongoDB Realm](https://www.mongodb.com/docs/realm/) and the Atlas Device SDKs are now in a difficult position. In September 2024, MongoDB announced the deprecation of the Atlas Device SDKs and Atlas Device Sync, with end of life targeted for September 2025. Applications that still rely on Realm for client storage and bidirectional sync need a JavaScript friendly replacement that does not lock the project to a single cloud vendor and that will keep receiving updates well past 2025.
This page explains why [RxDB](https://rxdb.info/) is a strong replacement for Realm in JavaScript, TypeScript, [React Native](../../react-native-database.md), [Electron](../../electron-database.md), and browser environments. It covers the history of Realm, the technical shortcomings that existed even before the deprecation announcement, the features RxDB provides today, code samples for schema definition and replication to a MongoDB-backed HTTP endpoint, and practical migration notes.
## A short history of Realm
Realm started in 2014 as a mobile database for Android and iOS, evolving from an earlier project called TightDB. It was positioned as a replacement for SQLite but the storage model resembled an object store more than a relational database. Realm Mobile Platform later added bidirectional sync between devices and a self-hostable server. Bindings for additional languages followed, including JavaScript through `realm-js` for Node.js and React Native.
In 2019 MongoDB acquired Realm and folded it into the MongoDB Atlas product line. The local database engine and the sync layer were rebranded as the Atlas Device SDKs and Atlas Device Sync, and the focus shifted toward replication against MongoDB Atlas in the cloud. Self-hosting the sync server stopped being a supported path. In September 2024, MongoDB published the deprecation notice for the Atlas Device SDKs and Atlas Device Sync. New project sign ups were closed, and existing customers were given until September 2025 before end of life.
For JavaScript teams this timeline is short. Migrating client storage, replication, and conflict handling without losing data takes planning, which is why choosing a long term alternative now matters.
## What is RxDB?
RxDB is a [local-first](../../articles/local-first-future.md) JavaScript database that stores data on the client and replicates it to any backend. It runs in the browser, Node.js, [Electron](../../electron-database.md), [React Native](../../react-native-database.md), Capacitor, Deno, and Bun. Data is organized into [collections](../../rx-collection.md) with a JSON [schema](../../rx-schema.md), queried with a MongoDB style [query language](../../rx-query.md), and observed through [RxJS observables](../../reactivity.md). Replication is pluggable: there are adapters for HTTP, GraphQL, WebRTC, CouchDB, Firestore, Supabase, NATS, and more. The storage layer is pluggable as well, so the same codebase can use IndexedDB, OPFS, SQLite, Memory, or a custom engine.
Unlike Realm, RxDB is written in TypeScript and ships as plain JavaScript. There are no per platform native bindings to maintain, and the same query and replication code works on every supported runtime.
## Realm shortcomings before the deprecation
The deprecation is the most pressing reason to migrate, but Realm had structural limitations long before the 2024 announcement.
- **Tight coupling to Atlas**: Atlas Device Sync only synced against MongoDB Atlas. Self hosted MongoDB clusters were not a supported sync target, which forced teams onto a single managed cloud service.
- **Limited query expressiveness in JavaScript**: The Realm query language in `realm-js` is a string based filter syntax with a smaller set of operators than the MongoDB query language. Aggregations, joins across collections, and complex nested filters often required client side post processing.
- **Native bindings on every platform**: Realm uses a C++ core with platform specific bindings. Upgrading React Native versions, Electron versions, or switching to a new architecture like Hermes or the new React Native architecture frequently broke the binding and required waiting for an upstream release.
- **License and vendor lock in**: While the SDKs are open source, the sync server and conflict resolution logic live inside MongoDB Atlas. Migrating away from Atlas meant rebuilding sync from scratch.
- **Schema migrations**: Schema changes in Realm required writing imperative migration functions in every client release, with limited tooling for testing migrations against production data.
The numbers reflect this. As of July 30, 2026, [the Realm JavaScript SDK](https://github.com/realm/realm-js) has 6,000 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `realm` package was downloaded 217,533 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/realm-vs-rxdb)).
## RxDB advantages for former Realm users
RxDB addresses each of the points above.
- **Pure JavaScript**: RxDB has no native bindings of its own. Storage adapters wrap the platform's existing engine, for example IndexedDB in the browser or `op-sqlite` on React Native, and the rest of the database is plain JS that runs anywhere.
- **Bring your own backend**: RxDB does not require any specific server. The [HTTP replication](../../replication-http.md) plugin replicates to any REST style endpoint, including one that writes to MongoDB on the server side. There are also plugins for [GraphQL](../../replication-graphql.md), WebRTC, CouchDB, Firestore, and Supabase. See the [replication overview](../../replication.md) for the full list.
- **MongoDB style queries**: Queries use the familiar `$gt`, `$in`, `$regex`, `$elemMatch` operators documented under [RxQuery](../../rx-query.md). Developers coming from MongoDB or Mongoose feel at home.
- **Observable queries**: Every query and document exposes an RxJS observable. UI code subscribes once and receives updates whenever the underlying data changes, on this tab or on another tab. This replaces Realm's change listeners with a standard reactive primitive. See [reactivity](../../reactivity.md).
- **Multi tab support**: RxDB coordinates writes across browser tabs through a leader election. A query opened in tab A reflects writes performed in tab B without manual wiring.
- **Encryption**: The [encryption plugin](../../encryption.md) encrypts field values at rest using AES, with a password derived key. Realm offered file level encryption, RxDB lets you choose which fields to encrypt.
- **Conflict resolution**: Replication conflicts are resolved through a user supplied handler. The default last write wins handler is provided, and custom merge logic can be plugged in. See [transactions, conflicts, and revisions](../../transactions-conflicts-revisions.md).
- **Offline first**: Reads and writes always go to the local store first and replication runs in the background. This is the same model Realm used and the same model that makes [offline first](../../offline-first.md) apps feel fast.
## Code sample: schema and reactive query
The following snippet defines a `todos` collection and subscribes to a reactive query. Compare this to the Realm equivalent that requires a Realm class definition and a synchronous `realm.objects(...)` call wrapped in a change listener.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
const db = await createRxDatabase({
name: 'tasksdb',
storage: getRxStorageDexie()
});
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 64 },
title: { type: 'string' },
done: { type: 'boolean' },
createdAt: { type: 'number' }
},
required: ['id', 'title', 'done', 'createdAt']
}
}
});
// Reactive query, emits whenever matching documents change.
const openTodos$ = db.todos.find({
selector: { done: false },
sort: [{ createdAt: 'desc' }]
}).$;
openTodos$.subscribe(todos => {
console.log('open todos:', todos.map(t => t.title));
});
await db.todos.insert({
id: 't1',
title: 'Replace Realm with RxDB',
done: false,
createdAt: Date.now()
});
```
## Code sample: replicating to a MongoDB-backed HTTP endpoint
RxDB does not talk to MongoDB directly from the client, which is the right architectural choice because the database driver belongs on the server. Instead the [HTTP replication plugin](../../replication-http.md) calls REST endpoints that read from and write to MongoDB on the server. The endpoints follow the pull and push pattern documented in the [replication](../../replication.md) guide.
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
collection: db.todos,
replicationIdentifier: 'todos-mongo-http',
live: true,
pull: {
async handler(checkpointOrNull, batchSize) {
const updatedAt = checkpointOrNull?.updatedAt ?? 0;
const id = checkpointOrNull?.id ?? '';
const response = await fetch(
`/api/todos/pull?updatedAt=${updatedAt}&id=${id}&limit=${batchSize}`
);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
async handler(changeRows) {
const response = await fetch('/api/todos/push', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(changeRows)
});
const conflicts = await response.json();
return conflicts;
}
}
});
replicationState.error$.subscribe(err => console.error('replication error', err));
```
On the server side, `/api/todos/pull` runs a MongoDB `find({ updatedAt: { $gte: ... } })` sorted by `updatedAt` and `id`, and `/api/todos/push` performs a conditional update keyed on the document revision. This pattern preserves the Atlas Device Sync developer experience without depending on Atlas.
## Migration notes from Realm to RxDB
The dedicated [Realm to RxDB migration guide](../realm-to-rxdb-migration.md) covers schema translation, data export, query rewrites, and the replication setup in detail. In short, a migration from Realm typically follows these steps.
1. **Map Realm classes to RxDB schemas**. Each Realm object schema becomes a JSON schema under an [RxCollection](../../rx-collection.md). Relationship properties map to references by primary key, and embedded objects map to nested object types in the schema.
2. **Export existing data**. Use the Realm SDK to read every object of every type and write them as JSON. This is a one off script that runs on app start during the transition release.
3. **Bulk insert into RxDB**. Use `collection.bulkInsert(docs)` on first launch after the migration release. Mark the migration as complete in `localStorage` so it only runs once.
4. **Replace Realm queries**. Realm's filter strings translate to RxDB selectors. For example `realm.objects('Todo').filtered('done == false SORT(createdAt DESC)')` becomes `db.todos.find({ selector: { done: false }, sort: [{ createdAt: 'desc' }] })`.
5. **Replace change listeners with observables**. `collection.addListener` becomes `query.$.subscribe`. Most UI frameworks already integrate with RxJS or with hooks like `useRxQuery`.
6. **Wire up replication**. Stand up the pull and push HTTP endpoints against MongoDB or any other server. Roll out the new client and disable Atlas Device Sync once devices have synced their final state.
A staged rollout where both databases run side by side for one release is the safest path. RxDB writes the authoritative copy, Realm stays read only, and the next release removes Realm.
## FAQ
Yes. In September 2024 MongoDB announced the deprecation of the Atlas Device SDKs and Atlas Device Sync. End of life is targeted for September 2025, and new project sign ups have already been closed. Existing apps will continue to function until EOL, after which the service will be shut down.
Yes, through a server side adapter. The RxDB client uses the [HTTP replication plugin](../../replication-http.md) to call REST endpoints, and those endpoints read from and write to MongoDB on the server. Direct client to MongoDB connections are not supported, which is the correct security boundary for any production app.
Define an [RxSchema](../../rx-schema.md) for each Realm class, export every Realm object to JSON on app launch, and call `collection.bulkInsert(docs)` to load them into RxDB. Track migration completion in persistent storage so the import runs exactly once per device.
Yes. RxDB has first class support for [React Native](../../react-native-database.md) using the SQLite or memory storage adapters. The same schema and query code runs in the browser, in Node.js, in Electron, and on React Native without modification.
The RxDB core is open source under the Apache 2.0 license and free for commercial use. There is also a Premium offering with extra storage adapters, encryption modes, and performance plugins. The free core is sufficient for most applications.
## Comparison table
| Feature | MongoDB Realm / Atlas Device SDK | RxDB |
| --- | --- | --- |
| Status | Deprecated, EOL September 2025 | Active, regular releases |
| Implementation | C++ core with native bindings per platform | Pure TypeScript, no native bindings |
| Backend | MongoDB Atlas only | Any HTTP, GraphQL, WebRTC, CouchDB, Firestore, Supabase, custom |
| Query language | Realm filter strings | MongoDB style selectors with `$gt`, `$in`, `$regex`, `$elemMatch` |
| Reactive queries | Change listeners | RxJS observables |
| Multi tab | Limited in browser | Built in leader election |
| Encryption | File level | Per field with AES |
| Schema migrations | Imperative migration callbacks | Declarative migration strategies per schema version |
| Offline first | Yes | Yes |
| React Native | Yes through native binding | Yes through SQLite or memory adapter |
| Browser | Limited via WebAssembly | First class through IndexedDB, OPFS, Memory |
| License | Apache 2.0 SDK, proprietary sync | Apache 2.0 core, optional Premium add-ons |
| Self hosting | Not supported | Fully supported |
For teams currently running on Realm, the EOL date in September 2025 is firm. Starting the migration to RxDB now leaves time for a staged rollout, a tested HTTP replication layer against MongoDB, and a clean removal of the Atlas Device SDK before support ends.
---
## RxDB as a NeDB Alternative for Node.js, Electron, and the Browser
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a NeDB Alternative for Node.js, Electron, and the Browser
If you arrived here, you are likely running a project that depends on **NeDB** and you are looking for a maintained replacement. NeDB served many Node.js, Electron, and browser applications well during its active years, but the project has been frozen since 2016. Modern apps need schema validation, observable queries, multi-tab coordination, and replication, and a database that still receives security updates. **RxDB** fills that gap while keeping the embedded, document-oriented model that NeDB users are familiar with.
## A Short History of NeDB
NeDB (Node Embedded Database) was created around 2013 by Louis Chatriot. It became widely adopted because it offered a familiar **MongoDB-like API** without requiring a server process. A NeDB database was just a file, and the library appended each operation as a new line, then compacted the file in the background. That design made NeDB attractive for:
- **Node.js scripts and small servers** that wanted a local store without setting up MongoDB.
- **Electron and nw.js desktop apps** that needed to persist user data between sessions.
- **Browser-based applications** through storage adapters that wrote to IndexedDB or localStorage.
The query language mirrored MongoDB, so developers could use operators like `$gt`, `$in`, and `$regex` against documents, build indexes on fields, and project results. The last change to the library code landed in May 2016, , a "not maintained anymore" notice was added to the README in 2021, and the only commits since then are further README edits, the newest from May 2025 ([commit history](https://github.com/louischatriot/nedb/commits/master/), checked July 30, 2026). Community forks kept the code alive. The most used one, `@seald-io/nedb`, had 624,845 npm downloads in the last 30 days ([npm](https://www.npmjs.com/package/@seald-io/nedb), July 30, 2026), but it inherits the same single-file architecture and adds neither replication nor reactive queries.
## What is RxDB?
[RxDB](https://rxdb.info/) is a reactive, NoSQL, [offline-first](../../offline-first.md) database for JavaScript. It runs in the browser, in [Node.js](../../nodejs-database.md), in [Electron](../../electron-database.md), in React Native, and in any other JavaScript runtime. Documents are stored locally through a swappable storage layer, queries return observables that emit on every change, and an open [replication](../../replication.md) protocol keeps clients in sync with any backend.
RxDB has been under continuous development for nearly a decade and ships regular releases, security fixes, and new features. It treats the local database as the primary source of truth, which matches how teams build [local-first applications](../../articles/local-first-future.md) today.
## Where NeDB Falls Short
NeDB still works for trivial use cases, but production apps tend to hit hard limits:
### 1. Unmaintained for Nearly a Decade
The repository has had no updates since 2016. Reported issues sit open, dependency vulnerabilities are not patched, and the codebase predates many features of modern Node.js such as worker threads and async iterators.
### 2. Single-File Persistence Risks Corruption
NeDB writes operations as appended lines and rewrites the entire file during compaction. A crash during compaction can leave the database in a damaged state, and there is no built-in recovery beyond manual file inspection. Larger datasets also slow startup, because NeDB reloads the whole file into memory.
### 3. No Replication
NeDB has no sync layer. Sharing data between two devices, between a desktop client and a server, or between two browser tabs requires a custom solution that the developer has to build, test, and maintain.
### 4. No Observable Queries
Queries return promises or callbacks. To keep a UI in sync with the data, the application has to re-run queries manually after every write. That coupling between writes and reads quickly becomes the source of bugs in any non-trivial UI.
### 5. No Multi-Tab Coordination
A NeDB database opened in two browser tabs has no concept of shared state. Writes from one tab are invisible to the other unless the tabs communicate themselves through a `BroadcastChannel` or similar primitive.
### 6. No Schema Validation
NeDB is schemaless. Every document can have any shape, which sounds flexible at first but quickly leads to runtime errors when fields drift over time. There is no migration system either, so changing data shape has to be handled by the application.
The numbers reflect this. As of July 30, 2026, [NeDB](https://github.com/louischatriot/nedb) has 13,542 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `nedb` package was downloaded 202,005 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/nedb-vs-rxdb)). The [NeDB repository](https://github.com/louischatriot/nedb/commits/master/) has not received a code change since May 2016, : the last commits, from May 2025, only touch the README.
## How RxDB Solves These Problems
RxDB keeps the document-oriented model that NeDB users like, and adds the features missing from NeDB:
- **Active maintenance**: continuous releases with security and feature updates.
- **Schema validation**: every collection is defined by an [RxSchema](../../rx-schema.md) based on JSON Schema, with versioning and migrations.
- **MongoDB-style queries**: the [RxQuery](../../rx-query.md) API supports the same operators NeDB users are accustomed to, including `$gt`, `$in`, `$regex`, `$elemMatch`, sorting, and skip/limit.
- **Observable queries**: queries expose RxJS observables, and the UI updates automatically when results change. See [Reactivity](../../reactivity.md).
- **Multi-tab support**: writes in one tab are streamed to all other tabs through `BroadcastChannel`, with conflict-safe storage handling under the hood.
- **Replication**: the [Sync Engine](../../replication.md) connects to any HTTP, GraphQL, CouchDB, WebRTC, or custom backend.
- **Durable storages**: RxDB ships with battle-tested storage adapters. Use SQLite or the filesystem on [Node.js](../../nodejs-database.md) and [Electron](../../electron-database.md), use [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), or [SQLite-WASM](../../rx-storage-sqlite.md) in the browser.
## Code Sample: From NeDB to RxDB
A typical NeDB workflow looks like this:
```js
const Datastore = require('nedb');
const db = new Datastore({ filename: 'tasks.db', autoload: true });
db.insert({ _id: 't1', title: 'Write report', done: false }, (err, doc) => {
// ...
});
db.find({ done: false }).sort({ title: 1 }).exec((err, docs) => {
// ...
});
```
The same workflow in RxDB looks like this:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'tasks',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' }
},
required: ['id', 'title', 'done']
}
}
});
await db.tasks.insert({ id: 't1', title: 'Write report', done: false });
const openTasks = await db.tasks
.find({ selector: { done: false }, sort: [{ title: 'asc' }] })
.exec();
```
The query syntax stays close to MongoDB, so most NeDB selectors translate directly. See the full [RxQuery documentation](../../rx-query.md) for supported operators.
## Code Sample: Subscribing to a Query
NeDB has no equivalent for the snippet below. With RxDB, the result list updates automatically whenever a matching document changes:
```ts
db.tasks
.find({ selector: { done: false } })
.$.subscribe(tasks => {
renderTaskList(tasks);
});
// Inserting a new task elsewhere in the app
await db.tasks.insert({ id: 't2', title: 'Send invoice', done: false });
// The subscriber above receives the updated array immediately.
```
This pattern removes the boilerplate of re-running queries after each write and keeps your UI consistent with the database state.
## Migration Notes
Most NeDB projects can move to RxDB in a few steps:
1. **Define a schema** for every NeDB datastore. Inspect a sample of existing documents to derive the field types and required properties. The schema is required by [RxCollection](../../rx-collection.md) and unlocks validation and migrations.
2. **Pick a storage**. On Node.js or Electron, use a durable storage like SQLite (see [Node.js Database](../../nodejs-database.md) and [Electron Database](../../electron-database.md)). In the browser, [IndexedDB](../../rx-storage-indexeddb.md) or [OPFS](../../rx-storage-opfs.md) are good defaults.
3. **Import data**. Read the existing NeDB file with the legacy library, normalize each document so it matches the new schema, and call `bulkInsert` on the corresponding RxDB collection. NeDB uses `_id` as the primary key, while RxDB lets you choose any field, so a small rename is often required.
4. **Translate queries**. Most selectors port over with no changes. Replace callback APIs with async/await, and replace manual re-runs with `.$` observables where you want reactive updates.
5. **Add replication if needed**. If your old setup synced data through a custom mechanism, replace it with the official [RxDB replication](../../replication.md).
A migration script that runs once on first launch is often enough. After a successful import, the legacy NeDB file can be deleted.
## FAQ
No. The original NeDB repository has not received commits since 2016 and is archived. Issues remain open, and dependency security advisories are not addressed. Community forks exist, but none provide the long-term support that an active project like RxDB offers.
Yes. RxDB queries use the same selector format as MongoDB and NeDB, including operators like `$gt`, `$lt`, `$in`, `$nin`, `$regex`, and `$elemMatch`, plus `sort`, `skip`, and `limit`. See the [RxQuery documentation](../../rx-query.md) for the full list.
Yes. RxDB ships official guidance and storage options for Electron, including SQLite-backed storages that store data on the local filesystem. The [Electron Database](../../electron-database.md) page covers configuration in both the main and renderer processes, including multi-window setups.
Read the existing NeDB file with the legacy library, define an RxDB schema that matches the documents, and call `bulkInsert` on the new collection. Rename `_id` to your chosen primary key while you copy the data. After verifying the import, the old NeDB file can be removed.
## Comparison Table
| Feature | NeDB | RxDB |
| --- | --- | --- |
| Maintenance status | No code change since 2016, README marked unmaintained | Active, regular releases |
| Query language | MongoDB-like | MongoDB-like ([RxQuery](../../rx-query.md)) |
| Schema validation | None | JSON Schema based ([RxSchema](../../rx-schema.md)) |
| Observable queries | No | Yes, via RxJS ([Reactivity](../../reactivity.md)) |
| Multi-tab support | No | Yes |
| Replication | None | Built-in ([Sync Engine](../../replication.md)) |
| Browser storage | IndexedDB adapter | [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), [SQLite-WASM](../../rx-storage-sqlite.md) |
| Node.js storage | Single-file append log | SQLite, filesystem, memory ([Node.js Database](../../nodejs-database.md)) |
| Electron storage | Single-file append log | Durable storages ([Electron Database](../../electron-database.md)) |
| Migrations | Manual | Built-in schema migrations |
| TypeScript support | Community typings | First-class TypeScript |
| Encryption | None | Optional plugin |
| Compression | None | Optional plugin |
## Follow Up
RxDB gives NeDB users a maintained, document-oriented database with the same MongoDB-style query language, plus the features modern apps require: schemas, observable queries, multi-tab coordination, and [replication](../../replication.md). Read the [Quickstart](../../quickstart.md), pick a storage that fits your runtime, and port your collections over with a short migration script.
More resources:
- [RxDB on GitHub](/code/)
- [Local-First Future](../../articles/local-first-future.md)
- [Offline-First Guide](../../offline-first.md)
- [RxDB Sync Engine](../../replication.md)
---
## RxDB as a PouchDB Alternative - Reactive, Fast, and Backend-Agnostic
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a PouchDB Alternative
Many developers start building an offline-first web application and reach for PouchDB because of its well-known CouchDB synchronization capabilities. Over time, those same developers encounter performance bottlenecks, storage bloat, and the absence of reactive queries. [RxDB](https://rxdb.info) is a local-first JavaScript database that solves exactly these problems while keeping the offline-first architecture developers rely on.
This page explains what PouchDB is, how it works, what its architectural constraints are, and how RxDB provides a more complete solution for modern offline-first and local-first applications.
---
## What is PouchDB?
PouchDB is a JavaScript database that was created to bring the [Apache CouchDB](./couchdb-alternative.md) replication protocol to the browser. It was first released in 2012 by Dale Harvey and grew rapidly as a go-to solution for browser-based offline storage. The name is derived from "Portable CouchDB": a CouchDB-compatible database you can carry around in JavaScript environments.
PouchDB stores documents in the browser using IndexedDB (or WebSQL in older browsers) and exposes an HTTP-based API that mirrors CouchDB. This means any PouchDB database can replicate bidirectionally with a CouchDB server using the established Couch Replication Protocol, making it the natural pairing for teams already running CouchDB on the server.
### A Brief Timeline
- **2012** - First published by Dale Harvey; early adoption in the CouchDB community
- **2013** - PouchDB 1.0 released; gains IndexedDB adapter for modern browser support
- **2014** - Version 2.0; plugin ecosystem grows with adapters for SQLite, LevelDB, and memory
- **2015** - PouchDB 4.0 with significant performance improvements and the `pouchdb-find` query plugin
- **2016** - Version 5.0 brings improved conflict resolution APIs
- **2018** - PouchDB 7.0 released; WebSQL adapter deprecated
- **2022** - Ongoing maintenance with no major feature additions
- **2024** - PouchDB 9.0.0 released; the project enters incubation at the Apache Software Foundation
- **2025** - Development continues under Apache incubation; project is maintained but no longer actively gaining new features compared to newer alternatives
PouchDB played a formative role in popularizing offline-first development. RxDB itself started as a wrapper around PouchDB in 2016. As the limitations of the PouchDB architecture became clear, RxDB version 10.0.0 (released in 2021) introduced the `RxStorage` abstraction and removed the hard dependency on PouchDB. The PouchDB RxStorage was subsequently removed from RxDB because it was too slow and too difficult to maintain.
### How PouchDB Works
PouchDB stores every document as a node in a **revision tree**. Each write to a document creates a new revision (`_rev` field). When two clients modify the same document while offline, PouchDB stores both revisions as branches in the tree. On sync, the Couch Replication Protocol transmits the revision tree from one side to the other and detects which revisions are missing. The receiving side merges the trees and designates a "winning" revision using a deterministic algorithm.
This approach guarantees that no data is lost during sync and that conflicts can always be detected and resolved. The protocol is proven and robust for server-to-server replication.
The same approach creates structural problems when applied to browser storage, because the client has to store and process the full revision history of every document.
---
## Limitations of PouchDB for Modern Applications
### Revision Tree Overhead and Storage Bloat
PouchDB must store the entire revision history of every document to stay compatible with the CouchDB replication protocol. Every write to a document adds a revision node to the tree. Over time this tree accumulates, and the stored data grows well beyond the size of the actual document state.
If a document is written 100 times, PouchDB stores metadata for all 100 revisions in addition to the current state. The database gets larger with every update cycle. Compaction (`db.compact()`) can prune non-leaf revisions, but it is a manual process, and purging a revision entirely from PouchDB [has never been possible](https://github.com/pouchdb/pouchdb/issues/802). This is a fundamental constraint of the Couch Replication Protocol: removing a revision from the tree can break sync with other nodes.
For a long-running production application, this means the local IndexedDB database grows continuously with no upper bound. In browser environments where storage is limited and users can be prompted to clear site data, this is a serious operational problem.
### Slow Queries Due to Storage Layout
PouchDB's IndexedDB storage layout is organized around the revision tree, not around query performance. When a query runs, PouchDB must read documents from IndexedDB, reconstruct the current state from the winning revision in the tree, and then filter and sort the results.
This sequence is significantly slower than reading directly from a storage structure optimized for queries. RxDB's own documentation describes the problem:
> "To be compliant with CouchDB, PouchDB has to store all revision trees of documents which slows down queries."
For small datasets the overhead is acceptable, but as the dataset grows or as queries become more complex, the performance gap widens.
### No Reactive Queries
PouchDB does not have a built-in reactive query system. When a document changes, PouchDB does not automatically update query results or notify subscribers. Developers who want UI components to reflect the current database state must manually listen to PouchDB's `changes` feed, identify which queries are affected, re-run them, and update the state.
This is complex to implement correctly and is a common source of bugs, especially around race conditions between incoming changes and in-flight queries. Third-party libraries like `pouchdb-live-find` attempted to fill this gap, but they add complexity and are not officially supported.
RxDB provides reactive queries as a first-class feature using RxJS Observables. Every query can be subscribed to, and the subscription emits updated results automatically whenever matching documents change.
### Backend Lock-In to the CouchDB Protocol
PouchDB's replication is tied to the Couch Replication Protocol. If your backend is CouchDB (or a CouchDB-compatible service like Cloudant), replication works out of the box. If your backend is anything else, such as a REST API, a GraphQL endpoint, or Supabase, you have to implement synchronization yourself.
Many teams start with CouchDB for the easy PouchDB sync, then later want to move to a different backend as their requirements change. With PouchDB, this migration requires writing a custom sync layer from scratch.
### No Schema Enforcement
PouchDB does not validate document structure before writing. Documents are free-form JSON. There is no schema declaration, no type checking, and no rejection of invalid documents. The full responsibility for data integrity falls on the application layer.
Without a schema, refactoring data models or adding new required fields requires manual data migration code and careful coordination across all app versions that might have written data to local storage.
### Limited TypeScript Support
PouchDB was designed before TypeScript became the standard in JavaScript development. Its TypeScript types describe the PouchDB API but cannot provide type-safe access to document fields. Accessing a document field returns `any`, which eliminates the benefit of TypeScript's compile-time checks.
### Issues That Were Never Fixed
During RxDB's time as a PouchDB wrapper, many PouchDB bugs were encountered that could not be resolved from outside the library. For example, queries with `$gt` operators [return incorrect documents](https://github.com/pouchdb/pouchdb/pull/8471). The RxDB codebase accumulated workarounds and monkey patches to work around these issues, but some problems could not be fixed externally at all. This was one of the primary motivations for building the `RxStorage` abstraction and removing PouchDB from RxDB's core.
---
Both projects are under active development. As of July 30, 2026, [PouchDB](https://github.com/apache/pouchdb) has 17,598 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## How RxDB Solves These Problems
[RxDB](https://rxdb.info) is a local-first JavaScript database that runs on the client and handles all the challenges described above. It stores data locally, supports reactive queries through RxJS Observables, validates documents against a JSON Schema, and replicates with a wide range of backends without being tied to any single protocol.
### No Revision Tree Overhead
RxDB stores only the current state of each document. There is no revision tree, no accumulated history, and no storage bloat over time. When documents are updated, old data is overwritten, not appended. The local database size reflects the actual data, not the complete update history.
This approach is possible because RxDB has its own [conflict detection mechanism](../../replication.md) that does not depend on storing revision trees. During replication, RxDB compares document versions using a configurable conflict handler rather than inspecting revision ancestry.
### Pluggable Storage Engines
RxDB separates the query engine from the storage layer through the [RxStorage interface](../../rx-storage.md). You choose the storage engine based on your platform and performance requirements:
| Environment | Storage Option |
|---|---|
| Browser (general use) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (write-heavy workloads) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multiple browser tabs | [SharedWorker](../../rx-storage-shared-worker.md) |
| Tests | [Memory](../../rx-storage-memory.md) |
Switching storage is a one-line change in the database creation call. The rest of the application, including queries, replication configuration, and schema definitions, remains unchanged:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
// Change this import to switch storage engines
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
// import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
// import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
```
PouchDB is also available as an RxStorage backend for compatibility with existing setups, but its use in new projects is not recommended because of the performance and storage issues described above.
### Reactive Queries with RxJS Observables
RxDB builds on [RxJS](https://rxjs.dev) to provide observable queries. Every query result is a live data source. When documents that match the query change, the observable emits the updated result automatically:
```ts
// Subscribe to all active items sorted by creation time
db.items.find({
selector: { status: 'active' },
sort: [{ createdAt: 'asc' }]
}).$.subscribe(activeItems => {
console.log('Active items:', activeItems.length);
renderList(activeItems);
});
```
This subscription remains active and re-emits whenever a local write or a remote sync changes the matching set. There is no need to manually listen to change events, figure out which queries are affected, or re-run queries on every write.
RxDB uses the [event-reduce algorithm](https://github.com/pubkey/event-reduce) to determine whether a document change affects a query's result set. For most writes, the updated result can be calculated without re-querying the storage layer, making reactive queries fast even when many subscriptions are active.
You can also subscribe to individual documents or specific fields within a document:
```ts
const item = await db.items.findOne('item-001').exec();
// Fires only when the 'status' field changes
item.get$('status').subscribe(newStatus => {
console.log('Status changed to:', newStatus);
});
// Fires for every change to any document in the collection
db.items.$.subscribe(changeEvent => {
console.log(changeEvent.operation, changeEvent.documentId);
});
```
### Schema Validation and TypeScript Inference
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before it is written to storage. Invalid documents are rejected immediately:
```ts
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
items: {
schema: {
title: 'item schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
status: { type: 'string', enum: ['active', 'done', 'archived'] },
createdAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'createdAt'],
indexes: ['createdAt', 'status']
}
}
});
try {
// Missing 'status' field, schema validation rejects this
await db.items.insert({
id: 'item-001',
title: 'First item',
createdAt: Date.now()
});
} catch (err) {
console.error('Validation error:', err.message);
}
```
RxDB also infers TypeScript types from the schema automatically. IDE autocompletion and compile-time type checking work across all collection operations:
```ts
// TypeScript knows the exact shape of this document
const item = await db.items.findOne('item-001').exec();
if (item) {
// 'title' is string, 'status' is 'active' | 'done' | 'archived'
console.log(item.title);
console.log(item.status);
}
```
PouchDB documents return untyped objects. There is no connection between the stored data structure and TypeScript's type system.
### Flexible Replication with Any Backend
RxDB is not coupled to a single replication protocol. The [RxDB replication system](../../replication.md) is designed around a generic pull/push model that can work with any backend that supports a checkpoint-based sync API.
Built-in replication plugins include:
- **[CouchDB replication](../../replication-couchdb.md)** - Sync with any CouchDB-compatible endpoint without the revision-tree overhead
- **[GraphQL replication](../../replication-graphql.md)** - Sync with any GraphQL API
- **[HTTP replication](../../replication-http.md)** - Sync with a custom REST API
- **[WebSocket replication](../../replication-websocket.md)** - Real-time push-based sync over WebSocket
- **[Supabase replication](../../replication-supabase.md)** - Sync with a Supabase PostgreSQL backend
- **[WebRTC replication](../../replication-webrtc.md)** - Peer-to-peer sync between browser tabs and devices
- **[Firestore replication](../../replication-firestore.md)** - Sync with Firebase Cloud Firestore
You can also implement a custom replication handler for any backend that does not have a built-in plugin. The interface is straightforward:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
replicationIdentifier: 'my-custom-sync',
collection: db.items,
pull: {
async handler(checkpointOrNull, batchSize) {
const checkpoint = checkpointOrNull ?? { updatedAt: 0 };
const response = await fetch(
`/api/items?since=${checkpoint.updatedAt}&limit=${batchSize}`
);
const data = await response.json();
return {
documents: data.items,
checkpoint: data.checkpoint
};
}
},
push: {
async handler(rows) {
const response = await fetch('/api/items/bulk', {
method: 'POST',
body: JSON.stringify(rows)
});
const conflicts = await response.json();
return conflicts;
}
},
live: true,
retryTime: 5000
});
```
This is in contrast to PouchDB, where the only built-in replication mechanism is the Couch Replication Protocol. Teams that want to use a different backend have to build the entire sync layer from scratch.
### CouchDB Replication Without the Overhead
If you are currently using PouchDB with a CouchDB backend, you can migrate to RxDB and continue using CouchDB as the sync target. RxDB's [CouchDB replication plugin](../../replication-couchdb.md) syncs with any CouchDB-compatible endpoint using RxDB's own sync engine. This means no revision trees are stored on the client and no compaction is required.
```ts
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB({
replicationIdentifier: 'my-couchdb-replication',
collection: db.items,
url: 'https://example.com/db/items',
live: true,
pull: { batchSize: 60 },
push: { batchSize: 60 }
});
await replicationState.awaitInitialReplication();
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
```
The token can be updated mid-session when authentication is required:
```ts
import {
replicateCouchDB,
getFetchWithCouchDBAuthorization
} from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB({
replicationIdentifier: 'my-couchdb-replication',
collection: db.items,
url: 'https://example.com/db/items',
fetch: getFetchWithCouchDBAuthorization('myUsername', 'myPassword'),
live: true,
pull: { batchSize: 60 },
push: { batchSize: 60 }
});
```
### Schema Migrations
When your data model changes, RxDB handles migration automatically. You increment the schema version number and provide a migration strategy for each version step:
```ts
await db.addCollections({
items: {
schema: {
title: 'item schema',
version: 1, // incremented from 0
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
status: { type: 'string', enum: ['active', 'done', 'archived'] },
priority: { type: 'number' }, // new field
createdAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'priority', 'createdAt']
},
migrationStrategies: {
1: (oldDoc) => {
// Assign default priority to all existing documents
oldDoc.priority = 0;
return oldDoc;
}
}
}
});
```
When the database opens and detects the new schema version, it automatically runs the migration strategy on all locally stored documents before the application starts. PouchDB has no built-in migration system. Schema changes require manual update scripts that must be applied carefully to avoid data corruption.
### Conflict Resolution
RxDB resolves conflicts during replication through a configurable conflict handler:
```ts
await db.addCollections({
items: {
schema: itemSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Last-write-wins based on timestamp
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative applications where concurrent edits from multiple users should be merged, RxDB supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md):
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const itemSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDTs enabled, concurrent writes are automatically merged when clients sync. No manual conflict resolution code is required for the common case.
PouchDB stores conflicting revisions as alternate branches in the revision tree. Resolving a conflict requires fetching all conflicting revisions, comparing them, picking a winner, and posting that winner back as the new revision. This is explicit, verbose, and easy to implement incorrectly.
### Multi-Tab Support
When a user opens a web app in multiple browser tabs, each tab has its own JavaScript process. A PouchDB write in one tab does not automatically appear in reactive queries in another tab. Synchronizing state across tabs requires custom coordination code using `localStorage` events, `BroadcastChannel`, or a similar mechanism.
RxDB handles this through the [SharedWorker storage option](../../rx-storage-shared-worker.md). All tabs share a single database instance running in the SharedWorker. A write from one tab automatically propagates to reactive queries in all other tabs without any additional code:
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
### Encryption at Rest
RxDB includes a [built-in encryption plugin](../../encryption.md) for encrypting specific document fields before they are written to local storage. The raw storage (IndexedDB, SQLite, OPFS) contains ciphertext for encrypted fields:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'user-specific-passphrase'
});
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
sensitiveNote: { type: 'string' }
},
encrypted: ['sensitiveNote']
};
```
Without the passphrase, the raw IndexedDB contents are unreadable. PouchDB has no built-in encryption for document fields.
### Performance Comparison
Because RxDB does not store revision trees and its storage layer is optimized for read queries, it is significantly faster than PouchDB for most workloads. The following patterns cause the largest performance differences:
- **Bulk inserts**: RxDB writes only the current document. PouchDB initializes a revision tree for each document.
- **Document reads after many updates**: RxDB reads the current document directly. PouchDB must navigate the revision tree to determine the winning revision.
- **Range queries**: RxDB's IndexedDB storage layout uses indexes designed for range queries. PouchDB's layout is designed for replication correctness.
- **Storage size after sustained use**: RxDB database size is proportional to the number of documents. PouchDB database size grows with both the number of documents and the number of updates to each document.
When using the [OPFS storage](../../rx-storage-opfs.md), RxDB gets an additional performance advantage over PouchDB, because OPFS bypasses IndexedDB's transaction overhead entirely:
```ts
import { getRxStorageOPFS } from 'rxdb/plugins/storage-opfs';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageOPFS()
});
```
OPFS gives RxDB direct filesystem access inside the browser's origin-private file system, resulting in significantly higher write throughput than IndexedDB for bulk operations.
---
## Migrating from PouchDB to RxDB
If you have an existing application using PouchDB, migration to RxDB involves the following steps:
1. **Define a schema** for each collection. List all document fields with their types and which fields should be indexed.
2. **Replace the PouchDB initialization code** with `createRxDatabase` and `addCollections`.
3. **Replace PouchDB queries** with RxDB's MongoDB-style query selectors and `find()` / `findOne()` APIs.
4. **Replace the changes feed subscription** with RxDB's observable query subscriptions.
5. **Replace PouchDB sync** with the appropriate RxDB replication plugin.
6. **Migrate existing local data** using RxDB's migration strategies if local documents already exist in IndexedDB.
The main conceptual shift is moving from PouchDB's event-based model (listen to changes, re-run queries manually) to RxDB's reactive model (subscribe to query observables, receive automatic updates).
---
## Comparison Table
| Feature | PouchDB | RxDB |
|---|---|---|
| **Offline-first** | Yes | Yes |
| **Reactive queries** | No (manual implementation required) | Yes (RxJS Observables) |
| **Schema validation** | No | Yes (JSON Schema) |
| **TypeScript support** | Limited (untyped documents) | Full (inferred from schema) |
| **Storage engine** | Fixed (IndexedDB in browser) | Pluggable (IndexedDB, OPFS, SQLite, Memory) |
| **Revision tree overhead** | Yes (storage grows with every update) | No (current state only) |
| **CouchDB replication** | Yes (Couch Replication Protocol) | Yes (custom plugin, no revision-tree overhead) |
| **GraphQL replication** | No | Yes (built-in plugin) |
| **HTTP / REST replication** | No | Yes (built-in plugin) |
| **WebSocket replication** | No | Yes (built-in plugin) |
| **Supabase replication** | No | Yes (built-in plugin) |
| **WebRTC replication** | No | Yes (built-in plugin) |
| **Conflict resolution** | Revision-tree branches (manual) | Configurable handler or CRDT plugin |
| **Schema migrations** | Manual | Automatic via versioned strategies |
| **Multi-tab consistency** | Not built-in | SharedWorker storage |
| **Encryption at rest** | No | Per-field encryption plugin |
| **Backend flexibility** | CouchDB protocol only | Any backend via replication plugins |
| **Query language** | `pouchdb-find` Mango syntax | MongoDB-style selectors |
| **Compaction required** | Yes (storage grows without it) | No |
| **Bundle size** | Large | Modular (tree-shakeable) |
| **Active development** | Apache incubation (maintenance mode) | Active, with commercial support |
---
## FAQ
Yes. RxDB has a dedicated [CouchDB replication plugin](../../replication-couchdb.md) that syncs with any CouchDB-compatible endpoint. The key difference from PouchDB is that RxDB does not use the Couch Replication Protocol internally. It uses RxDB's own sync engine on top of the CouchDB HTTP API. This avoids the revision-tree storage overhead while still replicating correctly with CouchDB servers.
If your users have existing data in PouchDB's IndexedDB storage, you need to migrate it to RxDB's storage format. The recommended approach is to read all documents from the existing PouchDB database on first launch after the upgrade, insert them into RxDB, and then remove the old PouchDB storage. RxDB's migration strategies handle schema version changes within RxDB itself, but the initial import from PouchDB is a one-time operation that you implement in application code.
PouchDB is in incubation at the Apache Software Foundation as of 2024 and released version 9.0.0 in mid-2024. It is maintained in the sense that critical bugs are addressed, but it is not gaining significant new features. Its architecture is constrained by the requirement to remain compatible with the Couch Replication Protocol, which limits how much the performance and feature set can evolve without breaking backward compatibility.
The PouchDB RxStorage was removed from RxDB because of persistent performance issues and bugs that could not be fixed externally. If you were using RxDB with the PouchDB storage, you should migrate to a different RxStorage such as [IndexedDB](../../rx-storage-indexeddb.md) or [OPFS](../../rx-storage-opfs.md). Staying on older versions of RxDB (before version 15) is also possible but means missing out on all improvements since then.
Yes. RxDB works in browsers, React Native, Electron, and Node.js. For React Native, the recommended storage is [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md), which provides native performance on both iOS and Android. The same schema definitions, queries, and replication configuration work across all environments.
No. RxDB is a local-first database. All reads and writes go to local storage. Replication with a backend is optional and can be enabled or disabled at any time. An application using RxDB works fully offline without any backend connection. Replication runs in the background and syncs when connectivity is available.
---
## RxDB as a PowerSync Alternative for JavaScript Local-First Apps
import {Faq, FaqItem} from '@site/src/components/faq';
# RxDB as a PowerSync Alternative for JavaScript Local-First Apps
PowerSync looks attractive when you already run Postgres or MongoDB on the server and want a managed sync engine on top. For JavaScript teams the practical picture is more mixed. The browser client runs on top of WASM SQLite which adds read and write latency, the FSL source-available license restricts shipping competing products, and the server-authoritative model leaves little room to plug in a custom backend or non-SQL query layer.
RxDB takes a different route. It is a [local-first](../../offline-first.md) JavaScript database that stays storage-agnostic, runs MongoDB-style queries against [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), or [SQLite](../../rx-storage-sqlite.md), and lets you bring your own backend over [HTTP](../../replication-http.md), [GraphQL](../../replication-graphql.md), or any other transport. RxDB is licensed under Apache 2.0, so commercial use is unrestricted.
## A Short History of PowerSync
PowerSync is built by JourneyApps, a South African company that ran an internal sync platform for years before publishing PowerSync as a standalone product around 2023. The first public version targeted Postgres, using logical replication and a change data capture (CDC) pipeline to stream row updates into a sync service. Clients embed a SQLite database, downloaded as a WASM build in the browser or as native SQLite on Flutter, Kotlin, and Swift.
Since the initial release the team has added a MongoDB backend connector, expanded the TypeScript-first SDKs for web and React Native, and tightened the integration with Supabase. The whole stack is published under the Functional Source License (FSL), a source-available license that converts to Apache 2.0 after two years. Until that conversion happens, you may not use PowerSync to build a product that competes with PowerSync itself or with the JourneyApps Platform.
## What is RxDB?
RxDB is a [reactive](../../reactivity.md), [local-first](../../articles/local-first-future.md) JavaScript database. Each [collection](../../rx-collection.md) is defined by a JSON [schema](../../rx-schema.md), stored on a pluggable storage layer, and queried with a Mango (MongoDB-style) query API. Every [query](../../rx-query.md) is observable, so UI components re-render automatically when underlying data changes, locally or through a sync. The [replication protocol](../../replication.md) is transport-agnostic, which means you can sync with Postgres, Mongo, Couch, Firestore, or any custom REST or GraphQL service, without changing how the application code reads or writes data.
## PowerSync Limitations for JavaScript Teams
### WASM SQLite Latency in the Browser
The PowerSync web client is a WASM build of SQLite that persists into IndexedDB or OPFS. Each query crosses the JavaScript and WASM boundary, decodes a SQLite result set, and returns rows back to the main thread. Benchmarks across local-first databases show that this layered approach adds milliseconds per query compared to a JavaScript-native engine that reads typed objects directly from IndexedDB or OPFS. For interactive UIs that observe many small queries the overhead compounds. RxDB sidesteps the WASM hop by running the query planner in JavaScript and going straight to the storage layer. See [Slow IndexedDB](../../slow-indexeddb.md) for the underlying constraints and the strategies RxDB uses to stay fast.
### Server-Authoritative Model
PowerSync places business logic, conflict resolution, and access rules on the central Postgres or Mongo server through Sync Rules. This works well when the server is the unambiguous source of truth and you are happy to express policy in SQL. It is more restrictive when you want client-side conflict handlers, offline writes that merge with custom logic, or a backend that is not a single managed Postgres or Mongo cluster. RxDB lets you define a [conflict handler](../../transactions-conflicts-revisions.md) per collection, run logic on the client, and choose any backend shape.
### Source-Available License
PowerSync ships under FSL, which forbids using the software to build a product that competes with PowerSync or the JourneyApps Platform for two years after each release. For agencies, platform vendors, and SaaS products that might overlap with sync tooling, this clause matters. RxDB is Apache 2.0, with no field-of-use restriction.
### SQL-Only Query DSL
PowerSync queries are SQL strings. That suits backend teams that already think in SQL, but it means schema migrations, type generation, and reactive bindings all flow through string parsing. RxDB queries are plain objects, type-checked against the schema, and composable in JavaScript:
```ts
const query = db.tasks.find({
selector: {
done: false,
priority: { $gte: 2 }
},
sort: [{ updatedAt: 'desc' }]
});
```
Both projects are under active development. As of July 30, 2026, [the PowerSync JavaScript SDK](https://github.com/powersync-ja/powersync-js) has 698 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## Why RxDB Works Well as a PowerSync Alternative
### Storage-Agnostic Client
RxDB separates the database API from the storage engine. You can pick the storage that fits the runtime:
- [IndexedDB storage](../../rx-storage-indexeddb.md) for broad browser support.
- [OPFS storage](../../rx-storage-opfs.md) for low-latency file system access in modern browsers.
- [Dexie storage](../../rx-storage-dexie.md) when you want a familiar IndexedDB wrapper.
- [SQLite storage](../../rx-storage-sqlite.md) for Node.js, Electron, Capacitor, or React Native.
Switching storages is a configuration change, not a rewrite.
### MongoDB-Style Queries with Reactivity
[RxQuery](../../rx-query.md) accepts Mango selectors and returns observable results. Subscribing to a query gives you a stream of updates that fires whenever a matching document is inserted, updated, or deleted, locally or via [replication](../../replication.md). This pairs naturally with React, Vue, Svelte, Angular, or any other framework that consumes observables.
### Bring Your Own Backend
RxDB does not require a managed sync server. The [HTTP replication](../../replication-http.md) plugin handles the protocol details, and you provide pull and push handlers that talk to whatever endpoint you have, including a Postgres-backed REST API, a Mongo Atlas function, or a [GraphQL](../../replication-graphql.md) gateway. You can also combine multiple replications, for example a server sync plus a peer-to-peer WebRTC sync, on the same collection.
### Apache 2.0 Licensing
The core RxDB engine is Apache 2.0. Premium plugins are available, but the base library imposes no field-of-use limits.
### First-Class Multi-Tab Support
RxDB coordinates writes and query state across multiple browser tabs through a leader election and broadcast channel system, so opening the app in two tabs does not duplicate sync work or produce inconsistent UI state.
## Code Sample: Schema and Mango Query
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'appdb',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 40 },
title: { type: 'string' },
done: { type: 'boolean' },
priority: { type: 'integer', minimum: 0, maximum: 5 },
updatedAt: { type: 'string', format: 'date-time' }
},
required: ['id', 'title', 'done', 'priority', 'updatedAt'],
indexes: ['priority', 'updatedAt']
}
}
});
// Observable Mango query
db.tasks.find({
selector: { done: false, priority: { $gte: 2 } },
sort: [{ updatedAt: 'desc' }]
}).$.subscribe(results => {
console.log('open high-priority tasks:', results.length);
});
```
## Code Sample: HTTP Replication With a Postgres-Backed REST API
The handler shape below maps cleanly onto a Postgres backend. The pull handler returns documents whose `updated_at` is greater than the checkpoint, and the push handler upserts incoming rows and reports any server-side conflicts.
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replication = replicateRxCollection({
collection: db.tasks,
replicationIdentifier: 'tasks-postgres-rest',
live: true,
pull: {
async handler(checkpoint, batchSize) {
const since = checkpoint ? checkpoint.updatedAt : '1970-01-01T00:00:00Z';
const res = await fetch(
`/api/tasks/pull?since=${encodeURIComponent(since)}&limit=${batchSize}`
);
const body = await res.json();
return {
documents: body.documents,
checkpoint: body.checkpoint
};
}
},
push: {
async handler(changeRows) {
const res = await fetch('/api/tasks/push', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(changeRows)
});
// Server returns the documents that lost a conflict
return await res.json();
}
}
});
replication.error$.subscribe(err => console.error('sync error', err));
```
The REST endpoints can wrap any Postgres query you like, including row-level security, computed columns, and stored procedures. The same pattern applies to MongoDB, Supabase, or a service mesh in front of multiple databases.
## When PowerSync Makes Sense
PowerSync is a strong fit when:
- The team is SQL-first and wants to keep writing SQL on both client and server.
- The backend is already Postgres or Mongo and you want managed CDC without writing pull and push handlers.
- The product targets Flutter, Kotlin, or Swift in addition to web, and you want a single vendor SDK across all of them.
- The license terms are acceptable for the product you are building.
If those constraints match, PowerSync gives you a coherent path. If you need browser-first performance, a NoSQL query model, custom backends, or unrestricted licensing, RxDB is the better tool.
## FAQ
RxDB queries run in JavaScript directly against [IndexedDB](../../rx-storage-indexeddb.md) or [OPFS](../../rx-storage-opfs.md), so each read returns typed objects without crossing a WASM boundary or decoding a SQLite result set. PowerSync executes queries inside a WASM SQLite build that persists to the same browser primitives, which adds extra serialization on every read and write. See [Slow IndexedDB](../../slow-indexeddb.md) for the underlying constraints both engines have to work around.
Yes. RxDB does not ship a built-in Postgres connector, but the [HTTP replication](../../replication-http.md) plugin lets you put any REST service in front of Postgres and stream changes both ways. You can also use [GraphQL replication](../../replication-graphql.md) with PostGraphile, Hasura, or a custom resolver layer.
PowerSync is source-available under the Functional Source License (FSL), which permits non-competing use and converts to Apache 2.0 two years after each release. There is also a hosted cloud tier with usage-based pricing. RxDB core is Apache 2.0 with no field-of-use restriction.
No. RxDB targets JavaScript and TypeScript runtimes, including the browser, Node.js, Electron, Capacitor, and [React Native](../../react-native-database.md). PowerSync ships native SDKs for Flutter, Kotlin, and Swift, so if those platforms are required without a JavaScript bridge, PowerSync covers more ground.
RxDB schemas are versioned. When you bump the version of a [collection schema](../../rx-schema.md), you provide a migration strategy that maps documents from the previous version to the new one. The migration runs on the client when the database opens. PowerSync handles migrations through SQL DDL on the server plus client schema definitions that mirror the SQL tables.
## Comparison Table
| Topic | RxDB | PowerSync |
| --- | --- | --- |
| License | Apache 2.0 | FSL, source-available, two-year delayed Apache 2.0 |
| Client storage | Pluggable: [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), [Dexie](../../rx-storage-dexie.md), [SQLite](../../rx-storage-sqlite.md), memory | WASM SQLite in browser, native SQLite elsewhere |
| Query language | Mango (MongoDB-style) JSON queries | SQL strings |
| Reactivity | Observable queries via RxJS | Watched SQL queries |
| Backend | Bring your own via [HTTP](../../replication-http.md), [GraphQL](../../replication-graphql.md), WebRTC, Firestore, CouchDB | Managed sync service connected to Postgres or MongoDB |
| Conflict resolution | Per-collection custom handler on the client | Server-authoritative through Sync Rules |
| Client SDKs | JavaScript and TypeScript across browser, Node.js, Electron, [React Native](../../react-native-database.md) | JavaScript, Flutter, Kotlin, Swift |
| Multi-tab | Built-in leader election and broadcast | Limited, depends on storage configuration |
| Self-hosting | Full client and replication code is open source | Self-hosted service available, gated by FSL terms |
## Follow Up
If your stack is JavaScript-first, runs in the browser, or needs a backend shape that PowerSync does not cover out of the box, RxDB is worth a closer look. Start with the [Replication guide](../../replication.md), explore the [HTTP replication](../../replication-http.md) plugin, and read [The Local-First Future](../../articles/local-first-future.md) for the broader context. For real-time UI patterns, see [Realtime Database](../../articles/realtime-database.md).
More resources:
- [RxDB Sync Engine](../../replication.md)
- [HTTP Replication](../../replication-http.md)
- [GraphQL Replication](../../replication-graphql.md)
- [RxQuery API](../../rx-query.md)
- [RxDB GitHub Repository](/code/)
---
## RxDB as a Replicache Alternative for Local-First Web Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Replicache Alternative for Local-First Web Apps
Replicache built strong mindshare in the local-first community with its mutator-based sync model and tight focus on collaborative web apps. Teams pick it because it advertises support for many backend stacks and ships a polished developer experience. Once a project grows, the mutator architecture, the source-available license, and the opinionated query API push some teams to look for a Replicache alternative. RxDB is an open source [local-first](../../articles/local-first-future.md), NoSQL database for JavaScript that stores data on the client, runs MongoDB-style queries against a local store, and replicates with any backend you control.
## A Short History of Replicache
Replicache was built by [Rocicorp](https://rocicorp.dev/), founded by Aaron Boodman and Erik Arvidsson, and first appeared around 2020. It was distributed under a source-available license rather than an OSI-approved open source license. In 2024 Rocicorp announced that Replicache would be free to use and that the next-generation product, Zero (sometimes referred to as Zerosync), would succeed both Replicache and Reflect.
The defining trait of Replicache is its mutator-driven model. Instead of writing to a local database and pushing the changes through a generic replication protocol, you define mutator functions that describe how a piece of input data changes the state. Each mutator runs first on the client for instant feedback and then again on the server to produce the authoritative state. The frontend reads data through `useSubscribe` and similar hooks that fire when the local cache changes. This gives strong [optimistic UI](../../articles/optimistic-ui.md) behavior, but it forces you to mirror logic across both sides of the stack and to design your APIs around mutator names rather than collections, queries, or REST resources.
## What Is RxDB?
RxDB (Reactive Database) is a JavaScript database that stores documents in a local [RxCollection](../../rx-collection.md) and exposes [reactive queries](../../reactivity.md) on top of that store. It runs in the browser, in Node.js, in Electron, and in [React Native](../../react-native-database.md). The local store is the source of truth for the UI, and the [Sync Engine](../../replication.md) keeps it aligned with a remote endpoint. RxDB is licensed under Apache 2.0 with optional premium plugins, so the core code is fully open source and auditable.
Key properties:
- Document model with JSON schemas and indexes.
- MongoDB-style query language through [RxQuery](../../rx-query.md).
- Reactive results based on RxJS Observables.
- Pluggable storage layer (IndexedDB, OPFS, SQLite, in-memory, and more).
- Replication with [HTTP](../../replication-http.md), [GraphQL](../../replication-graphql.md), CouchDB, Firestore, WebRTC, and custom transports.
- Custom [conflict handlers](../../transactions-conflicts-revisions.md) per collection.
## Where Replicache Falls Short
Replicache is a focused product, and that focus shows up as friction once requirements expand.
### 1. Mutator Architecture Forces Shared Logic
Every write goes through a mutator. The same function definition has to exist on the client and on the server, and both must produce the same delta for the same input. Teams that already own a REST or GraphQL backend end up wrapping their existing endpoints in mutators or rewriting business logic on the client. RxDB instead treats the local collection as a regular database. Writes happen locally and the [replication protocol](../../replication.md) ships changes to whatever endpoint you already run.
### 2. Source-Available, Not Open Source Until 2024
For most of Replicache's history the source code shipped under a source-available license. Free use was capped at non-commercial projects, companies under $200k ARR, and companies with less than $500k in funding. The 2024 announcement made Replicache free, but the long-term product investment has shifted to Zero. RxDB has been Apache 2.0 from the start, the source lives on [GitHub](/code/), and the project does not gate features behind revenue thresholds.
### 3. Opinionated Query API
Replicache reads data through key/value scans and `useSubscribe`. There is no built-in support for MongoDB-style operators, secondary indexes defined in a schema, or aggregation. Anything that resembles a query is something you assemble in JavaScript on top of the scan API. RxDB ships a full [RxQuery](../../rx-query.md) engine with `$gt`, `$in`, `$regex`, sorting, limits, and indexed lookups, plus observable results.
### 4. Server-Side State Is Your Problem
Replicache hands you a sync protocol but expects you to maintain the canonical state on the server, including version tracking, client groups, and patch generation. RxDB's pull and push handlers are simple async functions that return documents and a checkpoint. The server side can be a thin wrapper around an existing database, a stored procedure, or a CouchDB instance.
### 5. No First-Party Peer-to-Peer
Replicache is a client-server protocol. RxDB ships a [WebRTC replication plugin](../../replication.md) so peers can sync directly without a central server.
The numbers reflect this. As of July 30, 2026, [Replicache](https://github.com/rocicorp/replicache) has 1,172 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `replicache` package was downloaded 50,744 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/replicache-vs-rxdb)). The last commit to the [Replicache repository](https://github.com/rocicorp/replicache) was in April 2022, , and the repository is archived and read-only today. Rocicorp's development happens in the Zero monorepo instead.
## Why Teams Pick RxDB Instead
- **Apache 2.0 license** with no revenue gates on the core.
- **Document database** with JSON schema validation and typed queries.
- **Observable queries** that update the UI when underlying data changes.
- **Replication with arbitrary endpoints**, including [HTTP](../../replication-http.md), [GraphQL](../../replication-graphql.md), CouchDB, Firestore, and WebRTC.
- **Multi-storage** so the same code runs on IndexedDB in the browser, SQLite in React Native, and in-memory in tests.
- **No required mutator definitions**. Writes are normal `insert`, `patch`, and `remove` calls on the collection.
- **Conflict handlers** that you control per collection.
- **Real-time** behavior through the [reactive query engine](../../articles/realtime-database.md).
## Code Sample: HTTP Replication Without Shared Mutators
The following example creates a collection and replicates it against a plain REST endpoint. There is no mutator definition shared with the server. The server only needs to accept a batch of documents on push and return new documents plus a checkpoint on pull.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { replicateRxCollection } from 'rxdb/plugins/replication';
const db = await createRxDatabase({
name: 'app',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'text', 'done', 'updatedAt']
}
}
});
replicateRxCollection({
collection: db.todos,
replicationIdentifier: 'todos-http',
live: true,
pull: {
async handler(checkpoint, batchSize) {
const url = `https://api.example.com/todos/pull?cp=${
encodeURIComponent(JSON.stringify(checkpoint || {}))
}&limit=${batchSize}`;
const res = await fetch(url);
const body = await res.json();
return {
documents: body.documents,
checkpoint: body.checkpoint
};
}
},
push: {
async handler(changeRows) {
const res = await fetch('https://api.example.com/todos/push', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(changeRows)
});
return await res.json();
}
}
});
// Standard write. No mutator needed.
await db.todos.insert({
id: 't1',
text: 'Try RxDB',
done: false,
updatedAt: Date.now()
});
```
The same collection can be wired to a [GraphQL endpoint](../../replication-graphql.md), a CouchDB server, or a custom WebSocket stream by swapping the replication plugin. The collection API does not change.
## Code Sample: Subscribing to a Query in React
Replicache exposes `useSubscribe` over its scan API. RxDB exposes the full query language and returns an Observable that React can consume directly.
```tsx
import { useEffect, useState } from 'react';
import { RxDocument } from 'rxdb';
type Todo = {
id: string;
text: string;
done: boolean;
updatedAt: number;
};
export function OpenTodos({ db }) {
const [todos, setTodos] = useState[]>([]);
useEffect(() => {
const sub = db.todos
.find({
selector: { done: false },
sort: [{ updatedAt: 'desc' }]
})
.$.subscribe(results => setTodos(results));
return () => sub.unsubscribe();
}, [db]);
return (
{todos.map(t => (
t.patch({ done: true })}>
{t.text}
))}
);
}
```
The query is declarative. The patch call is a normal write on the document. Optimistic UI behavior comes from the local store and is described in the [Optimistic UI guide](../../articles/optimistic-ui.md).
## Mutators vs Documents
The core difference between Replicache and RxDB is the mental model.
Replicache treats every change as a named operation. A mutator like `addTodo({ id, text })` is the only way to mutate state. The client runs the mutator against the local cache for an instant result, the server runs the same mutator to produce the canonical state, and the protocol reconciles the two. Application logic lives inside mutators. Reads are scans over a key/value store.
RxDB treats every change as a write to a document in a collection. Code calls `collection.insert`, `doc.patch`, or `collection.bulkUpsert`. The local store records the change, the [Sync Engine](../../replication.md) ships it to the server through a generic push handler, and the server stores it in whatever database it already uses. Reads are MongoDB-style queries with reactive results.
This has practical consequences:
- **Backend reuse**: RxDB plugs into existing REST, GraphQL, or SQL backends without renaming endpoints to match mutator semantics.
- **Schema-driven storage**: RxDB validates documents against a JSON schema. Replicache stores arbitrary JSON values keyed by strings.
- **Query expressiveness**: RxDB supports operators, sorting, and indexes. Replicache requires you to scan and filter manually.
- **Conflict handling**: RxDB lets you write a [custom conflict handler](../../transactions-conflicts-revisions.md) per collection. Replicache merges through mutator replay.
Neither model is universally better. Mutators are convenient for tightly coupled collaborative editing. Documents are convenient for general application data, offline-first apps, and existing backends.
## FAQ
Yes. RxDB core is licensed under Apache 2.0 and the source is on GitHub. There are optional premium plugins for advanced storages and enterprise features, but the database, the query engine, and the replication protocol are open source with no revenue gating.
No. RxDB uses regular collection methods such as `insert`, `patch`, `bulkUpsert`, and `remove`. The replication protocol forwards the resulting changes to your backend through pull and push handlers. You can still centralize write logic in helper functions if you want to, but the database does not require it.
Yes. The replication protocol is checkpoint-based, so each client only fetches changes since its last sync. The server can be any system that exposes pull and push endpoints, which means horizontal scaling is the same problem as scaling your existing API. RxDB also ships [WebRTC replication](../../replication.md) for peer-to-peer scenarios.
Zero is Rocicorp's successor to Replicache and Reflect. It is a different product with its own protocol and trade-offs. RxDB is not a drop-in port of Zero, but it covers the same use cases of local-first apps with reactive queries and sync. RxDB has been stable for years, ships under Apache 2.0, and works with any backend you already run.
Yes. RxDB runs anywhere JavaScript runs. In NextJS and Remix you instantiate the database in the browser and use the React bindings to subscribe to queries. In [React Native](../../react-native-database.md) you pick a native storage such as SQLite. The same collection definitions and queries work across all environments.
## Comparison Table
| Feature | Replicache | RxDB |
| ----------------------------- | ----------------------------------- | ---------------------------------------- |
| License | Source-available, free since 2024 | Apache 2.0 (open source) |
| Data model | Key/value store | Document collections with JSON schema |
| Writes | Mutator functions on client+server | Direct `insert`, `patch`, `remove` calls |
| Query API | `useSubscribe` over scans | MongoDB-style [RxQuery](../../rx-query.md) |
| Reactive results | Yes | Yes, via RxJS Observables |
| Server requirements | Implement mutators and patch API | Implement pull and push handlers |
| Storage options | IndexedDB | IndexedDB, OPFS, SQLite, memory, more |
| Conflict resolution | Mutator replay | Custom per-collection handlers |
| Peer-to-peer sync | No | Yes, [WebRTC](../../replication.md) |
| Transports | Replicache protocol | HTTP, GraphQL, CouchDB, Firestore, WebRTC|
| Runtimes | Browser, React Native | Browser, Node.js, Electron, React Native |
## Follow Up
If the mutator architecture or the historical license terms of Replicache are blocking your project, RxDB is a direct alternative. It keeps the local-first developer experience, adds a real document database with reactive queries, and replicates with whatever backend you already run.
More resources:
- [RxDB Sync Engine](../../replication.md)
- [HTTP Replication](../../replication-http.md)
- [GraphQL Replication](../../replication-graphql.md)
- [RxQuery](../../rx-query.md)
- [Reactivity](../../reactivity.md)
- [Conflict Resolution](../../transactions-conflicts-revisions.md)
- [Local-First Future](../../articles/local-first-future.md)
- [Realtime Database](../../articles/realtime-database.md)
- [RxDB GitHub Repository](/code/)
---
## RxDB as a RethinkDB Alternative - Offline-First, Client-Side Reactive Database
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
import {CenteredImage} from '@site/src/components/centered-image';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a RethinkDB Alternative
RethinkDB introduced a compelling idea: instead of polling a database for changes, let the database push updates to your application the moment data changes. That capability, called changefeeds, attracted developers building realtime dashboards, chat applications, and collaborative tools. But the company behind RethinkDB shut down in 2016, and while the project continues as a community effort, the fundamental architecture of RethinkDB is a poor fit for modern offline-first applications. RethinkDB lives on the server and streams data to connected clients, which means your application stops working the moment a user loses network access.
This page explains what RethinkDB is, where it falls short for client-side and offline-first use cases, and why [RxDB](https://rxdb.info) covers the same reactive programming model while also working entirely on the client.
---
## What is RethinkDB?
RethinkDB is a distributed, document-oriented database built for realtime applications. It was founded in 2009 and launched publicly in 2012. Its defining feature is the **changefeed**: a persistent connection from a client to the database server that delivers change events (inserts, updates, deletes) as they happen, without the client needing to poll.
RethinkDB used its own query language called **ReQL**, which is chainable and embedded directly in the host language (JavaScript, Python, Ruby). A basic query with a changefeed looks like this:
```javascript
// RethinkDB: subscribe to all changes in the 'messages' table
r.table('messages')
.changes()
.run(connection, (err, cursor) => {
cursor.each((err, change) => {
console.log('Old value:', change.old_val);
console.log('New value:', change.new_val);
});
});
```
The changefeed approach was genuinely novel. Before RethinkDB, building a realtime app typically required polling, WebSocket infrastructure built on top of a regular database, or a specialized pub/sub system layered on top of storage. RethinkDB baked this directly into the query layer.
### RethinkDB's Timeline
- **2009** - RethinkDB Inc. is founded. The project begins as a storage engine optimized for SSDs.
- **2012** - Public launch of RethinkDB 1.0 as a realtime document database with ReQL and changefeeds.
- **2013-2015** - Active development, growing community, and strong interest from teams building realtime applications.
- **October 2016** - RethinkDB Inc. shuts down. The founders publish a post-mortem explaining they failed to build a sustainable business model competing against MongoDB and cloud-managed databases.
- **February 2017** - The Linux Foundation (via the Cloud Native Computing Foundation) acquires RethinkDB and relicenses it under the Apache License 2.0. Community maintenance continues.
- **2018-present** - RethinkDB receives occasional bug fixes and maintenance releases from community contributors, but no significant new feature development. The project is functionally stable but not actively evolving.
The post-mortem published by the founders is candid: the market for databases rewarded operational simplicity and managed services, and RethinkDB was difficult to operate compared to hosted alternatives like Firebase or later Supabase. Teams already running MongoDB had little reason to migrate just for changefeeds, especially as MongoDB added its own change streams feature.
### What RethinkDB Does Well
For server-to-client data streaming in a connected environment, RethinkDB's architecture works cleanly. Its ReQL query language is expressive, changefeeds are deeply integrated into the query model, and its distributed architecture handles sharding and replication across nodes. For a dashboard that monitors sensor data or a chat application where all users are assumed to be online, RethinkDB solved a real problem elegantly.
---
## Where RethinkDB Falls Short
### No Offline Support
RethinkDB is a server-side database. Your application queries data by sending a network request to the RethinkDB cluster. If a user loses network connectivity, every read and write fails immediately.
This is not a configuration problem or a missing plugin. The architecture does not include client-side storage. There is no local cache that can serve queries when offline. All changefeeds disconnect when the network drops, and the driver raises an error. Any changes occurring while the client is offline are simply lost to that client; RethinkDB does not buffer per-client missed events.
The companion client library Horizon, which provided authentication and subscription helpers for RethinkDB, explicitly never implemented offline support. An [open GitHub issue from 2016](https://github.com/rethinkdb/horizon/issues/58) requested offline support, and the thread was closed without resolution when the company shut down.
For modern web and mobile applications, offline support is not an edge case. Users open applications on trains, in buildings with poor signal, and in situations where the network is intermittent. An application that throws errors when the network drops creates a poor experience.
### Changefeeds Do Not Survive Disconnection
When a changefeed client disconnects and reconnects, it does not automatically receive the changes that occurred while it was offline. The application must re-establish the connection, re-run the query, and perform its own reconciliation between the last known state and the current server state.
The server buffers changes in memory up to `changefeed_queue_size` (default: 100,000 events). If the client is offline long enough that the buffer fills, the server drops events and notifies the client with an error. At that point, the application has an incomplete picture of what changed and must perform a full re-read.
This architecture shifts significant complexity onto application code. Every feature that uses a changefeed needs reconnection logic, backfill logic, and buffer overflow handling.
### Server-Side Architecture Requires Infrastructure Management
Running RethinkDB in production means managing a cluster. Sharding, replication factor, and server topology are configured manually. Compared to managed cloud databases like Firebase or Supabase, RethinkDB places operational responsibility on the team running it.
This was one of the reasons cited in the founders' post-mortem for why RethinkDB lost to MongoDB in the market: developers preferred managed services where operational concerns are abstracted away. Since the company closed, there is no official support contract or managed hosting service for RethinkDB.
### Community Maintenance, Not Active Development
RethinkDB is maintained by volunteers. It receives bug fixes but not new features. Driver support for newer JavaScript runtimes (Deno, Bun) and modern ecosystem tooling is limited compared to actively developed databases.
For a new project starting in 2025 or 2026, building on a database with no commercial backer, no managed hosting, and no active feature roadmap carries risk. If a security vulnerability is discovered or an incompatibility with a new Node.js version appears, the fix depends on community volunteers with no obligation to respond.
### ReQL Is Not Portable
ReQL is specific to RethinkDB. Knowledge of ReQL does not transfer to other databases, and ReQL queries cannot be reused if you switch storage backends. Compared to MongoDB-style query syntax (which RxDB, among others, implements), ReQL has a much smaller community knowledge base.
---
The numbers reflect this. As of July 30, 2026, the `rethinkdb` package was downloaded 65,703 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/rethinkdb-vs-rxdb)).
## How RxDB Approaches the Same Problems
[RxDB](https://rxdb.info) is a local-first JavaScript database built for client-side environments: browsers, React Native, Electron, and Node.js. It shares the reactive programming goal of RethinkDB (data changes should automatically propagate to the UI) but implements it on the client side rather than relying on a persistent server connection.
### Reactive Queries Without a Server Connection
In RxDB, every query is observable. When you subscribe to a query result, you receive the current result set immediately, and the observable re-emits whenever the underlying data changes, whether that change came from a local write or from a replication event.
```typescript
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
messages: {
schema: {
title: 'message schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
roomId: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'text', 'roomId', 'createdAt'],
indexes: ['roomId', 'createdAt']
}
}
});
// Subscribe to all messages in a specific room, sorted by creation time
db.messages.find({
selector: { roomId: 'room-42' },
sort: [{ createdAt: 'asc' }]
}).$.subscribe(messages => {
renderChatUI(messages); // called immediately and on every change
});
```
This works entirely offline. The query runs against IndexedDB (or any other configured storage), not against a remote server. There is no connection to establish and no disconnection to handle.
RxDB uses the [event-reduce](https://github.com/pubkey/event-reduce) algorithm to make reactive updates efficient. When a document write occurs, RxDB checks whether the existing query result can be updated by applying the change directly without re-executing the full query. This means reactive UI updates remain fast even in write-heavy workloads.
### True Offline-First Operation
When a user opens an RxDB application without network access, every feature works normally. Writes go to local storage. Queries return from local storage. The UI renders without any loading spinner or error state.
When network connectivity becomes available, RxDB's replication plugins synchronize local changes with the remote backend in the background. When the user goes offline again, the local database continues working. This is the [offline-first architecture](../../offline-first.md) pattern.
RethinkDB's architecture cannot deliver this. Data lives on the server, so offline means no data. There is no path to genuine offline-first operation without adding a separate local storage layer and writing the reconciliation logic yourself, at which point RethinkDB is just a backend, not a realtime client database.
### Flexible Storage Backends
RxDB has a pluggable storage layer. The same application code works with different storage engines depending on the environment:
| Environment | Storage Option |
|---|---|
| Browser (standard) | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high-throughput) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multi-tab browsers | [SharedWorker](../../rx-storage-shared-worker.md) |
| Testing / CI | [Memory](../../rx-storage-memory.md) |
Switching storage requires changing one parameter when creating the database:
```typescript
import { getRxStorageOpfs } from 'rxdb/plugins/storage-opfs';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageOpfs() // use OPFS for better browser performance
});
```
### Flexible Replication to Any Backend
RethinkDB is both the storage layer and the realtime transport. RxDB separates these concerns. RxDB stores data locally, and replication to a backend is a separate, configurable plugin.
The [HTTP replication plugin](../../replication-http.md) works with any REST or HTTP endpoint. The [GraphQL replication plugin](../../replication-graphql.md) connects to GraphQL APIs including AWS AppSync. The [WebSocket replication plugin](../../replication-websocket.md) provides low-latency push from a server. The [CouchDB replication plugin](../../replication-couchdb.md) uses CouchDB's multi-master protocol. You can also implement a [custom replication handler](../../replication.md) for any proprietary API.
```typescript
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: db.messages,
replicationIdentifier: 'messages-http-v1',
pull: {
handler: async (checkpoint, batchSize) => {
const url =
`/api/messages/changes?since=${checkpoint?.updatedAt ?? 0}` +
`&limit=${batchSize}`;
const response = await fetch(url);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
handler: async (rows) => {
const response = await fetch('/api/messages/push', {
method: 'POST',
body: JSON.stringify(rows),
headers: { 'Content-Type': 'application/json' }
});
return response.json(); // returns conflicting docs or []
}
},
live: true,
retryTime: 5000
});
// Observable replication state
replicationState.active$.subscribe(active => console.log('Syncing:', active));
replicationState.error$.subscribe(err => console.error('Sync error:', err));
```
The replication state is fully observable. You know exactly when replication is active, when it errors, and what documents were sent or received. Nothing is hidden.
### Multi-Tab Support in the Browser
RethinkDB is a server process; it does not have a concept of browser tabs. On the client side, running multiple browser tabs with independent in-memory state is a common source of consistency problems.
RxDB solves this with the [SharedWorker storage](../../rx-storage-shared-worker.md). All tabs share a single database instance running in a SharedWorker, so a write from any tab is immediately reflected in reactive queries in all other tabs:
```typescript
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
For tab coordination in scenarios that require exactly one tab to do background work (like running replication), RxDB includes a [leader election plugin](../../leader-election.md). One tab is elected leader and performs background tasks, while others wait. If the leader tab closes, another takes over automatically.
```typescript
import { RxDBLeaderElectionPlugin } from 'rxdb/plugins/leader-election';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBLeaderElectionPlugin);
// Wait until this tab is the leader before starting replication
await db.waitForLeadership();
startReplication(db);
```
### Observable Change Events
RxDB exposes a [changestream](../../rx-database.md) on both the database and collection level. You can subscribe to all document changes, similar to RethinkDB's table changefeeds, but the events come from the local database rather than a server:
```typescript
// Subscribe to all changes in the messages collection
db.messages.$.subscribe(changeEvent => {
console.log('Operation:', changeEvent.operation); // INSERT, UPDATE, DELETE
console.log('Document ID:', changeEvent.documentId);
console.log('Document data:', changeEvent.documentData);
});
// Subscribe to changes on a specific document
const doc = await db.messages.findOne('message-001').exec();
doc.$.subscribe(updatedDoc => {
console.log('Document updated:', updatedDoc?.text);
});
```
This is the client-side equivalent of a RethinkDB point changefeed. The difference is that these events originate locally, so they fire even when the user is offline.
### Conflict Resolution
In a realtime multi-user system, two users can edit the same document concurrently. RethinkDB's conflict model relied on the server having a single authoritative view, which worked because every write went through the server immediately.
RxDB operates on a local-first model: users can edit locally while offline, and those edits sync when connectivity returns. If two clients edited the same document while disconnected, both versions must be reconciled when they sync.
RxDB handles this with a configurable [conflict handler](../../replication.md):
```typescript
await db.addCollections({
messages: {
schema: messageSchema,
conflictHandler: async ({ newDocumentState, realMasterState }) => {
// Keep whichever version was updated more recently
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative editing scenarios where merge semantics matter (text that two users edited in different places), RxDB supports [CRDT-based conflict resolution](../../crdt.md). CRDTs merge concurrent edits deterministically without requiring a central authority:
```typescript
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const messageSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
roomId: { type: 'string' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
### Schema Validation and TypeScript Support
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before writing it to storage. Documents that do not match the schema are rejected at the database level, preventing corrupted data from entering the local store.
```typescript
try {
await db.messages.insert({
id: 'msg-001',
// 'text' field is required but missing
roomId: 'room-42',
createdAt: Date.now()
});
} catch (err) {
console.error(err); // Schema validation error: missing 'text'
}
```
RxDB generates TypeScript types automatically from the schema, giving you IDE autocompletion and compile-time type safety for all collection operations.
### Schema Migration
As an application evolves, data models change. RxDB has a built-in [schema migration system](../../migration-schema.md). When the local database opens with a higher schema version than the stored data, RxDB runs the migration automatically:
```typescript
await db.addCollections({
messages: {
schema: messageSchemaV2, // version: 1
migrationStrategies: {
1: (oldDoc) => {
// Migrate from version 0: add a 'roomId' field with a default
return {
...oldDoc,
roomId: oldDoc.roomId ?? 'general'
};
}
}
}
});
```
Migrations run locally on each client's data independently. They do not require a coordinated backend deployment.
### Encryption at Rest
RxDB includes a built-in [encryption plugin](../../encryption.md) that encrypts individual document fields before writing them to the local storage. This is useful for applications that store sensitive user data locally:
```typescript
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'your-encryption-passphrase'
});
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
private: { type: 'string' }
},
encrypted: ['private'] // stored as ciphertext in IndexedDB
};
```
---
## Getting Started with RxDB
Install RxDB and RxJS:
```bash
npm install rxdb rxjs
```
Create a database, insert some documents, and subscribe to reactive queries:
```typescript
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
addRxPlugin(RxDBDevModePlugin);
const db = await createRxDatabase({
name: 'chatapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
messages: {
schema: {
title: 'message schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' },
roomId: { type: 'string' },
createdAt: { type: 'number' }
},
required: ['id', 'text', 'roomId', 'createdAt'],
indexes: ['roomId', 'createdAt']
}
}
});
// Write a message
await db.messages.insert({
id: 'msg-001',
text: 'Hello from RxDB!',
roomId: 'room-42',
createdAt: Date.now()
});
// Reactive query: always reflects the current state
db.messages.find({
selector: { roomId: 'room-42' },
sort: [{ createdAt: 'asc' }]
}).$.subscribe(messages => {
console.log('Current messages:', messages.map(m => m.text));
});
```
All of this works offline. Connect a replication plugin when you need server sync.
---
## Comparison Summary
| Aspect | RethinkDB | RxDB |
|---|---|---|
| **Where it runs** | Server-side cluster | Client-side (browser, mobile, desktop) |
| **Offline support** | None (network required) | Full offline-first operation |
| **Reactive queries** | Server pushes changefeed events | Client-side observable queries via RxJS |
| **Data location** | Remote server only | Local storage (IndexedDB, OPFS, SQLite) |
| **Disconnection handling** | Changefeed drops; missed events are lost | Local database continues working |
| **Query language** | ReQL (RethinkDB-specific) | Mango (MongoDB-compatible JSON) |
| **Backend dependency** | Must run a RethinkDB cluster | Any backend or no backend |
| **Conflict resolution** | Server-authoritative (last write wins) | Configurable client-side handler or CRDTs |
| **Multi-tab support** | N/A (server concept) | SharedWorker (shared state across tabs) |
| **Schema validation** | None | JSON Schema enforced on every write |
| **Schema migration** | Manual | Built-in versioned migration strategies |
| **Encryption at rest** | None built-in | Built-in field-level encryption plugin |
| **TypeScript** | Community-maintained typings | Auto-generated from schema |
| **Current status** | Community-maintained since 2017 | Actively maintained since 2016 |
| **Commercial support** | None (company closed 2016) | Premium plugins and active development |
| **License** | Apache 2.0 | Apache 2.0 |
---
## FAQ
RxDB does not have a native RethinkDB replication plugin. If you run RethinkDB on the server, you can build a custom HTTP or WebSocket API in front of it and use RxDB's [custom replication](../../replication.md) or [WebSocket replication](../../replication-websocket.md) plugin to sync. RxDB's replication protocol only requires that the backend can serve document changes since a given checkpoint and accept pushed documents. Any server-side language with a RethinkDB driver can expose this interface.
RethinkDB changefeeds push individual change events (old value and new value) from the server to the client. The client receives raw events and must maintain its own state from them.
RxDB reactive queries emit the complete, current result set after every relevant change. When a query matches ten documents and one is updated, the subscriber receives all ten current documents. This maps directly to UI rendering: you always have the full state, not a stream of deltas to apply. The [event-reduce](https://github.com/pubkey/event-reduce) algorithm makes this efficient by computing result set updates from change events without re-running the full query against storage.
Yes. RxDB is used in production for collaborative applications. The local database ensures the UI is always responsive. Replication keeps all clients synchronized. For concurrent edits by multiple users on the same document, RxDB supports both custom [conflict handlers](../../replication.md) and [CRDT-based merging](../../crdt.md). The SharedWorker storage mode handles the case of multiple browser tabs in the same session sharing state without duplication.
RxDB's replication plugins run continuously with automatic retry. When the network is unavailable, the pull and push handlers fail, and RxDB waits for `retryTime` milliseconds before trying again. When the network returns, replication resumes automatically from the last successful checkpoint. No changes are lost: writes made while offline are stored locally and pushed to the server as soon as the connection is re-established.
No. The local database works without any authentication. Only the replication handlers need credentials, and those are plain async functions where you include whatever headers or tokens your backend requires. If authentication expires while the app is running, replication pauses, and you can re-supply credentials and resume without restarting the database.
---
## RxDB as a SignalDB Alternative for Local-First JavaScript Apps
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a SignalDB Alternative for Local-First JavaScript Apps
Teams that adopt [SignalDB](https://signaldb.js.org/) usually want a tiny, reactive store that plays well with framework signals in Vue, Solid, or React. The MongoDB-style API feels familiar, the in-memory engine is fast, and reactive queries plug straight into component re-renders. The trade-offs show up later: persistence is opt-in, replication is left to the developer, and the storage layer has fewer adapters than long-running local-first databases.
This page compares SignalDB with **RxDB** and shows when each tool fits. It also covers a hybrid setup where SignalDB handles the reactive UI layer while RxDB takes over persistence, multi-tab coordination, and backend sync.
## A Short History of SignalDB
SignalDB appeared in 2023 and matured through 2024 with contributions from Maximilian Stoiber and a small open-source community. The project targets developers who already think in **signals**: fine-grained reactive primitives popularized by Solid, Vue's `ref`, Preact signals, and Angular signals. SignalDB exposes a MongoDB-style query API (`find`, `findOne`, `insert`, `updateOne`) and returns reactive cursors that re-evaluate when signal dependencies change.
By default SignalDB stores data **in memory**. Persistence is added through adapters (localStorage, OPFS, custom). Sync is also pluggable: SignalDB ships a sync manager interface and expects the application to bring its own transport, conflict policy, and server. This minimalism keeps the bundle small and the API approachable, but it pushes a lot of work onto the integrator once an app needs offline guarantees, multi-device sync, or large datasets.
## What Is RxDB?
[RxDB](https://rxdb.info/) is a [local-first](../../offline-first.md), NoSQL JavaScript database that has been developed since 2016. It runs in browsers, Node.js, Electron, React Native, Deno, Bun, and Capacitor. Data is stored through a swappable [RxStorage](../../rx-storage-dexie.md) layer, queried with a Mongo-style selector engine, and observed through RxJS. The [Sync Engine](../../replication.md) provides a battle-tested protocol for [HTTP](../../replication-http.md), WebSocket, GraphQL, CouchDB, Firestore, NATS, and custom backends.
RxDB treats storage, queries, and replication as first-class primitives rather than optional add-ons. That maturity is the main reason teams move to it after outgrowing a smaller library.
## Where SignalDB Hits Its Limits
SignalDB is well designed for what it covers, but several gaps appear in production workloads:
- **In-memory by default.** Data lives in RAM. A page reload wipes the collection unless you wire up a persistence adapter. Large datasets compete with the rest of the JS heap.
- **Bring-your-own sync.** The sync manager is an interface, not a protocol. You implement pull, push, checkpoints, retry, and conflict handling. Real-world sync is harder than it looks once partial offline writes and reconnects enter the picture.
- **Fewer storage adapters.** The list of supported backends is short compared to RxDB's storage matrix that covers IndexedDB, OPFS, Dexie, SQLite, Memory, MongoDB, DenoKV, FoundationDB, and more.
- **Limited multi-client guarantees.** SignalDB does not include built-in [multi-tab coordination](../../rx-storage-indexeddb.md) or leader election. Two open tabs can drift unless you build that yourself.
- **Smaller ecosystem.** Community plugins, examples, and long-term issue history are thinner. For business apps that ship for years, ecosystem depth matters.
- **No schema-driven migrations.** SignalDB collections are loosely typed at runtime. RxDB enforces JSON Schema and runs versioned migrations on schema changes.
The numbers reflect this. As of July 30, 2026, [SignalDB](https://github.com/maxnowack/signaldb) has 673 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `@signaldb/core` package was downloaded 6,573 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/@signaldb/core-vs-rxdb)).
## Why Teams Pick RxDB Instead
### Durable storage with a swappable engine
RxDB writes through an [RxStorage](../../rx-storage-dexie.md) interface. Pick the storage that fits the runtime:
- [IndexedDB](../../rx-storage-indexeddb.md) for broad browser support.
- [OPFS](../../rx-storage-opfs.md) for high-throughput browser writes via the Origin Private File System.
- [Dexie](../../rx-storage-dexie.md) for a lightweight IndexedDB wrapper.
- [Memory](../../rx-storage-memory.md) for tests and ephemeral state.
- SQLite, MongoDB, DenoKV, and FoundationDB on the server side.
Switching engines is a one-line change. The query, replication, and reactivity layers stay identical.
### A real replication protocol
The [RxDB Sync Engine](../../replication.md) defines pull, push, checkpoint, and conflict semantics so applications do not reinvent them. Plugins exist for [HTTP](../../replication-http.md), WebSocket, GraphQL, CouchDB, Firestore, Supabase, NATS, and P2P. Conflict handlers are explicit functions you control per collection.
### MongoDB-style queries with reactivity
Both libraries expose a Mongo-like API. RxDB extends it with [observable queries](../../rx-query.md) that emit through RxJS, plus [framework hooks](../../reactivity.md) for React, Vue, Svelte, Solid, and Angular signals.
### Multi-tab and conflict resolution out of the box
Open the same app in three tabs. RxDB elects a leader, broadcasts changes, and keeps queries in sync across tabs without extra code. Custom conflict handlers run on every replication round and decide how concurrent edits merge.
### Mature ecosystem since 2016
RxDB has a decade of releases, paid support options, and production deployments at scale. The [local-first](../../articles/local-first-future.md) movement has grown around projects like RxDB precisely because long-running data layers need this kind of stability.
## Code Sample: Collection and Reactive Query in RxDB
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
const db = await createRxDatabase({
name: 'tasksdb',
storage: getRxStorageDexie()
});
await db.addCollections({
tasks: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 40 },
title: { type: 'string' },
done: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'done', 'updatedAt']
}
}
});
// Reactive query: emits whenever the result set changes,
// across tabs and after replication updates.
const openTasks$ = db.tasks
.find({ selector: { done: false } })
.sort({ updatedAt: 'desc' })
.$;
openTasks$.subscribe(tasks => {
console.log('Open tasks:', tasks.length);
});
await db.tasks.insert({
id: 't1',
title: 'Write SignalDB comparison',
done: false,
updatedAt: Date.now()
});
```
See [RxCollection](../../rx-collection.md) and [RxQuery](../../rx-query.md) for the full surface.
## Code Sample: HTTP Replication
The [HTTP replication plugin](../../replication-http.md) syncs an RxDB collection with any REST endpoint that exposes pull and push routes.
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
collection: db.tasks,
replicationIdentifier: 'tasks-http-replication',
pull: {
async handler(checkpoint, batchSize) {
const url =
`/api/tasks/pull?since=${checkpoint?.updatedAt ?? 0}` +
`&limit=${batchSize}`;
const response = await fetch(url);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
async handler(changeRows) {
const response = await fetch('/api/tasks/push', {
method: 'POST',
body: JSON.stringify(changeRows)
});
return await response.json(); // conflicts, if any
}
},
live: true,
retryTime: 5000
});
replicationState.error$.subscribe(err => console.error(err));
```
The same protocol scales to multi-device sync, partial replication by user or tenant, and resumable transfers after the device goes offline.
## Running SignalDB on Top of RxDB Persistence
A pattern that has emerged in the local-first community is to use SignalDB for **front-end reactivity** while delegating storage and sync to RxDB. SignalDB exposes a persistence adapter interface, so an RxDB-backed adapter can:
1. Read from an [RxCollection](../../rx-collection.md) on startup and feed the documents into the SignalDB collection.
2. Forward SignalDB writes to RxDB so they hit durable storage.
3. Subscribe to RxDB's change stream and push remote updates back into SignalDB so signals re-emit.
The result keeps the signal-friendly API that Vue, Solid, and React components consume, while RxDB handles IndexedDB or OPFS persistence, multi-tab coordination, schema migrations, and backend replication. This hybrid is a pragmatic upgrade path for teams already invested in SignalDB but hitting its persistence or sync limits.
## FAQ
Yes. RxDB ships a [reactivity adapter API](../../reactivity.md) that maps observable queries to Vue refs, Angular signals, Solid signals, Svelte stores, and Preact signals. Component code reads collections through the framework's native primitive while RxDB drives updates underneath.
Yes. SignalDB's persistence interface accepts a custom adapter. An RxDB-backed adapter stores documents in an [RxCollection](../../rx-collection.md), which gives SignalDB durable storage on IndexedDB, OPFS, SQLite, or any other RxStorage, plus the full [RxDB sync engine](../../replication.md) for backend replication.
RxDB has been developed since 2016, ships regular releases, and runs in production across browsers, Node.js, Electron, React Native, Deno, and Bun. SignalDB started in 2023 and is still expanding its adapter and sync surface. For long-lived applications, RxDB's release history and ecosystem are the safer bet.
Both libraries use a MongoDB-style selector. RxDB queries return [RxQuery](../../rx-query.md) objects with `.exec()` for one-shot reads and `.$` for an observable that emits on every change, including changes from other tabs and replication. SignalDB returns reactive cursors tied to its signal runtime. Migrating selectors between the two is mostly mechanical.
## Comparison Table
| Feature | SignalDB | RxDB |
| --- | --- | --- |
| First release | 2023 | 2016 |
| Default storage | In memory | Durable via RxStorage |
| Storage adapters | localStorage, OPFS, custom | IndexedDB, OPFS, Dexie, Memory, SQLite, MongoDB, DenoKV, FoundationDB, more |
| Query API | MongoDB-style, reactive cursors | MongoDB-style, [observable queries](../../rx-query.md) |
| Reactivity | Framework signals | RxJS plus [framework adapters](../../reactivity.md) for React, Vue, Svelte, Solid, Angular |
| Schema and migrations | Loose typing | JSON Schema with versioned migrations |
| Replication | Bring-your-own sync interface | Built-in [Sync Engine](../../replication.md) with [HTTP](../../replication-http.md), WebSocket, GraphQL, CouchDB, Firestore, NATS, P2P |
| Conflict resolution | Application-defined | Per-collection conflict handlers |
| Multi-tab support | Manual | Built-in leader election and broadcast |
| Runtimes | Browser, Node.js | Browser, Node.js, Electron, React Native, Deno, Bun, Capacitor |
| Ecosystem age | New | Decade of releases and plugins |
For more on the broader shift toward client-side data ownership, see [The Future of Local-First Apps](../../articles/local-first-future.md) and the [offline-first guide](../../offline-first.md).
---
## RxDB as a sql.js Alternative for Browser Persistence
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a sql.js Alternative for Browser Persistence
Developers often pick **sql.js** when they want to run SQL inside the browser without a server. It is a faithful port of SQLite compiled to WebAssembly, so any valid SQLite query runs in a JavaScript runtime. The catch shows up the moment a real application needs to keep data around: sql.js holds the entire database as an in-memory buffer. Closing the tab, reloading the page, or losing a process wipes the state. There is no built-in [persistence](../../offline-first.md), no [reactive query](../../reactivity.md) layer, no [replication](../../replication.md), and no awareness of other browser tabs.
This page explains why teams that started with sql.js often migrate to **RxDB** once their prototype turns into a product, and how RxDB fills the gaps while still letting you keep SQLite as the underlying storage if you want.
## A Short History of sql.js
sql.js started around 2014, created by Alon Zakai (kripken), the author of Emscripten. The original release compiled the SQLite C source to asm.js so a full SQL engine could run inside any JavaScript runtime. As browser support for WebAssembly matured, sql.js switched to a WASM build that delivered better startup time and smaller payloads. A pure JavaScript fallback remained for older browsers.
The library became the default choice for in-browser SQL demos, teaching tools, and offline document viewers that ship a prebuilt SQLite file. Because sql.js mirrors the SQLite feature set, queries written for the desktop or server work without changes inside the browser. What sql.js never aimed to solve was persistence, multi-tab coordination, or sync. Those concerns sit one layer above the engine, and most teams build them by hand or move to a database that already includes them.
## What is RxDB?
[RxDB](https://rxdb.info/) is a NoSQL, [local-first](../../articles/local-first-future.md) database for JavaScript applications. It runs in the browser, [Node.js](../../nodejs-database.md), [Electron](../../electron-database.md), [React Native](../../react-native-database.md), and any other runtime that can execute JavaScript. RxDB stores data in a swappable storage layer, validates documents against [JSON schemas](../../rx-schema.md), exposes [reactive queries](../../reactivity.md) through RxJS observables, and ships a [replication protocol](../../replication.md) that keeps clients in sync with any backend.
Where sql.js is one engine, RxDB is a full database product. The storage engine is just one configuration choice, and SQLite is one of several supported options.
## Where sql.js Falls Short
The list below covers the recurring problems that push teams away from sql.js once a project leaves the demo stage.
### 1. In-memory only
sql.js loads the database into a `Uint8Array`. To save state you serialize the buffer with `db.export()` and write it somewhere yourself, often [IndexedDB](../../rx-storage-indexeddb.md) or a server endpoint. To restore, you fetch the bytes and pass them to `new SQL.Database(bytes)`. Every change forces a manual export, which means either writing the full file on every mutation (slow for large datasets) or losing the most recent edits on a crash.
### 2. No observability
sql.js answers a query with a single result set. There is no way to subscribe to a query and receive updates when underlying rows change. Building a UI that reacts to data requires custom diff tracking or a full re-query after every write.
### 3. No schema validation per document
SQLite enforces table schemas, but the data model is row-and-column. Document-shaped data with nested objects, arrays, or optional fields needs hand-rolled JSON columns and manual validation.
### 4. No replication protocol
There is no built-in way to sync sql.js with a remote backend or another client. You write the protocol, the conflict logic, and the change tracking yourself.
### 5. No multi-tab coordination
Two browser tabs running sql.js each hold their own copy of the in-memory database. Writes in one tab do not appear in the other unless you re-export and re-import the buffer through some channel you implement.
### 6. Manual indexing strategy
You get SQLite indexes, but you also get the responsibility of designing them around access patterns that change as the app grows.
The numbers reflect this. As of July 30, 2026, [sql.js](https://github.com/sql-js/sql.js) has 13,652 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## Where RxDB Helps
RxDB addresses each of those gaps without giving up on the option to keep SQLite as the engine.
### Durable storage options
RxDB ships several storage backends that persist data without manual export steps.
- [SQLite Storage](../../rx-storage-sqlite.md) runs SQLite via WASM (or native bindings in Node.js, Electron, React Native) and writes through to a durable file or OPFS handle.
- [IndexedDB Storage](../../rx-storage-indexeddb.md) uses the standard browser database for broad compatibility.
- [OPFS Storage](../../rx-storage-opfs.md) writes to the Origin Private File System for the fastest pure-browser persistence available today.
You change the storage in one line of configuration and the rest of the application stays the same.
### MongoDB-style queries
RxDB exposes a [Mango query language](../../rx-query.md) that targets nested document fields, array contents, and compound conditions. The same query string runs against any storage backend.
### Reactive queries
Every [RxQuery](../../rx-query.md) returns an RxJS observable. UI components subscribe once and receive a fresh result set whenever a relevant document changes, including updates from other tabs.
### Full replication
The [Replication Protocol](../../replication.md) supports custom HTTP backends, [GraphQL](../../replication-graphql.md), [CouchDB](../../replication-couchdb.md), [Firestore](../../replication-firestore.md), [WebRTC](../../replication-webrtc.md), and more. Conflict handlers are pluggable.
### Schema validation
[RxCollections](../../rx-collection.md) require a JSON schema at creation time. Documents are validated on insert and update, indexes are derived from the schema, and TypeScript types can be generated from it.
### Multi-tab support
RxDB uses a leader election mechanism so writes from any tab propagate to all open tabs of the same origin without extra code.
## Code Sample: Schema-Driven Collection
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'appdb',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
invoices: {
schema: {
title: 'invoice schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
customer: { type: 'string' },
amount: { type: 'number' },
paid: { type: 'boolean' },
createdAt: { type: 'number' }
},
required: ['id', 'customer', 'amount', 'createdAt'],
indexes: ['createdAt']
}
}
});
// Insert a document
await db.invoices.insert({
id: 'inv-1001',
customer: 'acme',
amount: 420,
paid: false,
createdAt: Date.now()
});
// Reactive query: re-emits whenever a matching invoice changes
db.invoices
.find({ selector: { paid: false }, sort: [{ createdAt: 'desc' }] })
.$
.subscribe(unpaid => {
console.log('unpaid invoices:', unpaid.length);
});
```
The query returns an observable. There is no polling, no manual export, and no diff logic in the UI layer.
## Code Sample: SQLite Storage in Browser, Node, and Electron
If you want SQLite as the engine but still need durability, reactivity, and replication, swap the storage to the [RxDB SQLite Storage](../../rx-storage-sqlite.md). The application code does not change.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsWasm
} from 'rxdb/plugins/storage-sqlite';
import sqliteWasm from '@vlcn.io/wa-sqlite';
// Browser: SQLite compiled to WASM, persisted via OPFS
const storage = getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsWasm(sqliteWasm)
});
const db = await createRxDatabase({
name: 'sqlite-app',
storage
});
await db.addCollections({
notes: {
schema: {
title: 'note schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' }
},
required: ['id', 'title']
}
}
});
await db.notes.insert({ id: 'n1', title: 'first', body: 'hello sqlite' });
```
The same configuration works in Node.js with `better-sqlite3` and in Electron with the native SQLite binding. You write your application against RxCollections once, and the storage adapter handles the runtime details.
## Need Raw SQL?
Some teams reach for sql.js because they have an existing body of SQL queries or a SQLite file they want to read in the browser. RxDB's [SQLite Storage](../../rx-storage-sqlite.md) keeps SQLite as the engine, so the same WASM build that powers sql.js sits underneath your collections. You get [Mango queries](../../rx-query.md) for the application layer, and you can still drop down to SQL when you need it. For most CRUD work, the RxDB query API is shorter than equivalent SQL and avoids string concatenation around dynamic filters.
If your goal is to ship a static, read-only SQLite file for full-text search or reference data, sql.js remains a fine fit. If your goal is an application that writes data, syncs across devices, and reacts to changes, RxDB on top of SQLite covers the same engine plus everything sql.js leaves to you.
## FAQ
RxDB uses a pluggable storage layer. SQLite is one supported backend through the [RxDB SQLite Storage](../../rx-storage-sqlite.md), which can run on a WASM build of SQLite in the browser, on `better-sqlite3` in Node.js, on the native binding in Electron, or on the React Native SQLite module. Other storages such as [IndexedDB](../../rx-storage-indexeddb.md) and [OPFS](../../rx-storage-opfs.md) use no SQLite at all.
The primary RxDB query API is [Mango-style](../../rx-query.md), which is JSON based and works the same across every storage backend. When you choose the SQLite storage you can still execute raw SQL through the underlying SQLite handle for reporting or migrations, while keeping the application code on top of [RxCollections](../../rx-collection.md).
Each storage backend writes to a durable target. [OPFS](../../rx-storage-opfs.md) and [IndexedDB](../../rx-storage-indexeddb.md) persist inside the browser's storage area for the origin. The [SQLite Storage](../../rx-storage-sqlite.md) writes a SQLite file in Node.js and Electron, and uses OPFS files in the browser. Inserts and updates are flushed by the storage layer; you do not call an export step the way sql.js requires.
For typical application workloads with many small reads and writes, RxDB is faster because changes do not require re-serializing a full database buffer. sql.js stays competitive for one-shot analytical queries over a preloaded dataset, since the whole database already sits in memory. For write-heavy apps that must persist after every mutation, RxDB's storages avoid the export and reimport cycle that dominates sql.js write costs.
## Comparison Table
| Feature | sql.js | RxDB |
| --- | --- | --- |
| Persistence | In-memory only, manual export | Durable through every storage backend |
| Storage options | Single in-memory buffer | [SQLite](../../rx-storage-sqlite.md), [IndexedDB](../../rx-storage-indexeddb.md), [OPFS](../../rx-storage-opfs.md), and more |
| Query API | SQL | Mango queries with [reactive results](../../rx-query.md) |
| Reactive queries | Not supported | Built in via [RxJS observables](../../reactivity.md) |
| Schema validation | Row and column types only | JSON schema per [collection](../../rx-collection.md) |
| Replication | Not supported | [Full sync engine](../../replication.md) for HTTP, GraphQL, CouchDB, Firestore, WebRTC |
| Multi-tab coordination | Each tab is isolated | Shared state with leader election |
| TypeScript types | Manual | Generated from schema |
| Runtime support | Browser and Node.js | Browser, Node.js, Electron, React Native, Deno, Bun |
| Conflict handling | Application responsibility | Pluggable conflict handlers |
## Follow Up
If you started with sql.js for the SQL feature set and ran into the persistence and reactivity gaps, RxDB lets you keep SQLite as the engine while adding the rest of what an application database needs. Read the [Quickstart](../../quickstart.md), explore the [SQLite Storage docs](../../rx-storage-sqlite.md), and join the RxDB community on Discord and GitHub.
More resources:
- [RxDB Replication](../../replication.md)
- [RxDB SQLite Storage](../../rx-storage-sqlite.md)
- [RxDB OPFS Storage](../../rx-storage-opfs.md)
- [Local-First Future](../../articles/local-first-future.md)
- [RxDB on GitHub](/code/)
---
## RxDB as a Supabase Alternative - Offline-First, Local Storage, Reactive Queries
import {Faq, FaqItem} from '@site/src/components/faq';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a Supabase Alternative
Supabase is a popular backend platform built on PostgreSQL. It provides authentication, storage, auto-generated REST APIs (PostgREST), and a realtime WebSocket layer. What Supabase does not provide is a client-side database. When the network is unavailable, standard Supabase queries fail. When a user opens your app in multiple tabs, each tab reads directly from the server. There is no local data layer, no offline queue, and no reactive query system built into the Supabase client SDK.
This page explains what Supabase is, where it falls short for local-first applications, and how [RxDB](https://rxdb.info) fills the gap as a client-side database that can sync with Supabase in the background.
---
## What is Supabase?
Supabase was founded in 2020 by Paul Copplestone and Ant Wilson. The company describes its product as "an open source Firebase alternative." It is built around PostgreSQL and wraps it with several services:
- **PostgREST**: auto-generates a REST API from your database schema
- **GoTrue**: a JWT-based authentication service
- **Supabase Storage**: object storage built on S3-compatible APIs
- **Supabase Realtime**: an Elixir-based WebSocket server that reads PostgreSQL's logical replication stream (WAL) and broadcasts changes to subscribed clients
- **Edge Functions**: Deno-based serverless functions
Supabase grew rapidly. By 2024 it had reached roughly $30 million in annual recurring revenue and managed over one million hosted databases. In April 2025 it raised a Series D at a $2 billion valuation. It has become a default backend choice for many AI-assisted development tools and Y Combinator-backed projects.
The platform is genuinely open source. Its components (PostgREST, GoTrue, Realtime, Kong) can be self-hosted using Docker Compose. This sets it apart from Firebase, which is entirely proprietary.
### A Brief Timeline
- **2020** - Supabase is founded; initial public beta launches
- **2021** - Generally available; raises Series A of $30 million
- **2022** - Adds edge functions, database branching, and self-hosting documentation
- **2023** - Reaches hundreds of thousands of projects; launches Vector support (pgvector) positioning as an AI backend
- **2024** - Crosses one million hosted databases; Series C at $900 million valuation; becomes a default backend in AI coding tools (Bolt.new, Lovable, Cursor)
- **2025** - Series D at $2 billion valuation; adds Supabase AI assistant; changes default public schema security to protect new projects
- **2026** - Continues active development; estimated ARR reaches $70 million
### How Supabase Realtime Works
Supabase Realtime reads PostgreSQL's Write-Ahead Log (WAL) through logical replication. When a row changes, the Realtime server parses the WAL event and broadcasts it over WebSocket to subscribed clients. On the client, you subscribe like this:
```ts
import { createClient } from '@supabase/supabase-js';
const supabase = createClient(SUPABASE_URL, SUPABASE_ANON_KEY);
const channel = supabase
.channel('posts-changes')
.on(
'postgres_changes',
{ event: '*', schema: 'public', table: 'posts' },
(payload) => {
console.log('Change received:', payload);
}
)
.subscribe();
```
This works well while the user is online and the WebSocket connection is open. However, there are several significant limitations for application development.
---
## Key Limitations of Supabase for Local-First Applications
### No Client-Side Database
Supabase does not include a local data store. Every query goes to the server:
```ts
// This call FAILS when the user is offline
const { data, error } = await supabase
.from('posts')
.select('*')
.eq('published', true);
```
If the network is unavailable, `data` is `null` and `error` contains a fetch failure. The application has no fallback. Users who open the app while offline see a broken state.
Adding a meaningful offline experience requires you to choose a client-side database yourself, implement a synchronization protocol, handle conflict resolution, and manage the replication lifecycle. Supabase provides none of this.
### WebSocket Connections Are Not a Sync Engine
Supabase Realtime delivers changes to connected clients over WebSockets. This is not the same as synchronization:
- When a client disconnects and reconnects, it does not receive the changes that occurred during the gap. It must re-fetch data to determine the current state.
- WebSocket connections drop silently in several common situations: when a browser tab is moved to the background, when a mobile app goes to sleep, or when a network changes from Wi-Fi to cellular.
- There is no "catch-up" mechanism. Realtime is a streaming protocol, not a sync log.
For a true offline-first application, you need a sync engine that tracks a checkpoint, fetches all changes since the last checkpoint, and applies them to local storage in order. Supabase Realtime is not that.
### Vendor Dependency for Auth and Data Access
Supabase combines authentication and data access through Row Level Security (RLS). Your PostgreSQL RLS policies reference `auth.uid()` from the Supabase JWT. This is a tight coupling: the authorization model is baked into the database schema itself, and it only works if clients authenticate through Supabase Auth. Migrating to a different auth provider or a different backend later requires changes to every RLS policy in your database.
### No Observable Queries
The Supabase client SDK does not have a reactive query system. If you want your UI to update when data changes, you must combine the Realtime channel subscription with a manual re-fetch or state update:
```ts
// Without RxDB, you have to wire this yourself:
channel.on('postgres_changes', { event: 'INSERT', table: 'posts' }, async () => {
// Re-fetch the entire list every time something changes
const { data } = await supabase.from('posts').select('*');
setPosts(data);
});
```
This approach re-fetches all matching rows on every change event. It does not know which specific documents changed, does not support sorted or filtered re-queries efficiently, and requires custom state management to avoid flickering or race conditions.
### Relational Model Does Not Map to UI Directly
Supabase is built on PostgreSQL. The data model is relational: tables, rows, foreign keys, joins. Modern web UIs work with JSON documents. When your schema involves multiple related tables, fetching data for a single UI component often requires joins that PostgREST must construct from query parameters. Deeply nested or polymorphic data shapes are awkward to express.
---
## How RxDB Addresses These Problems
[RxDB](https://rxdb.info) is a local-first JavaScript database. All reads and writes go to local storage first. Replication with a backend runs in the background. The application works offline by design, and data is synced when connectivity is available.
RxDB includes a dedicated [Supabase Replication Plugin](../../replication-supabase.md) that connects your RxDB collections to Supabase tables using PostgREST for pull and push, and Supabase Realtime for live streaming. This gives you the best of both: a locally cached, reactive database on the client, and a PostgreSQL backend in the cloud.
### Local-First Data Storage
When you use RxDB, every read and write goes to local storage (IndexedDB in browsers, SQLite on mobile). The application works offline immediately:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
authorId: { type: 'string', maxLength: 100 },
published: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'body', 'authorId', 'published', 'updatedAt'],
indexes: ['updatedAt', 'authorId']
}
}
});
// This works offline. No network required.
const posts = await db.posts.find({
selector: { published: true },
sort: [{ updatedAt: 'desc' }]
}).exec();
```
Writes made while offline are stored locally and automatically pushed to Supabase when the connection is restored.
### The RxDB Supabase Replication Plugin
RxDB provides a dedicated plugin for syncing with Supabase:
```bash
npm install rxdb @supabase/supabase-js
```
First, create your Supabase table with the required fields:
```sql
create extension if not exists moddatetime schema extensions;
create table "public"."posts" (
"id" text primary key,
"title" text not null,
"body" text not null,
"authorId" text not null,
"published" boolean DEFAULT false NOT NULL,
"_deleted" boolean DEFAULT false NOT NULL,
"_modified" timestamp with time zone DEFAULT now() NOT NULL
);
-- Auto-update the _modified timestamp on every write
CREATE TRIGGER update_modified_datetime BEFORE UPDATE ON public.posts FOR EACH ROW
EXECUTE FUNCTION extensions.moddatetime('_modified');
-- Enable realtime streaming for this table
alter publication supabase_realtime add table "public"."posts";
```
Then start the replication in your application:
```ts
import { createClient } from '@supabase/supabase-js';
import { replicateSupabase } from 'rxdb/plugins/replication-supabase';
const supabase = createClient(
import.meta.env.VITE_SUPABASE_URL,
import.meta.env.VITE_SUPABASE_ANON_KEY
);
const replication = replicateSupabase({
tableName: 'posts',
client: supabase,
collection: db.posts,
replicationIdentifier: 'posts-supabase-v1',
live: true,
pull: {
batchSize: 50
},
push: {
batchSize: 50
}
});
// Wait for the initial sync to complete before showing data
await replication.awaitInitialReplication();
// Monitor sync errors
replication.error$.subscribe(err => {
console.error('Replication error:', err);
});
```
The plugin uses PostgREST for incremental pull and push operations, and Supabase Realtime to trigger live updates. When a row changes in Supabase, the Realtime channel fires, the plugin pulls the latest changes from PostgREST, and the local RxDB collection updates automatically. Your UI reacts to the local change without any additional wiring.
### Reactive Queries Without Polling
RxDB queries return RxJS Observables. Every query re-emits whenever the matching documents change in the local database, whether the change came from a local write or from a sync event with Supabase:
```ts
// Subscribe to published posts, sorted by most recent
const publishedPosts$ = db.posts.find({
selector: { published: true },
sort: [{ updatedAt: 'desc' }]
}).$;
publishedPosts$.subscribe(posts => {
console.log('Published posts updated:', posts.length);
renderPostList(posts);
});
```
When a remote user publishes a post and that change reaches this client through Supabase Realtime and the RxDB replication plugin, the observable emits the updated list immediately. There is no polling, no manual re-fetch, and no separate state management layer needed.
RxDB uses the [event-reduce](https://github.com/pubkey/event-reduce) algorithm to update query results efficiently. When a single document changes, RxDB checks whether the change affects the current query result and updates only what is necessary, rather than re-running the full query against storage.
You can subscribe to individual documents or specific fields:
```ts
// Subscribe to a single document
const doc = await db.posts.findOne('post-001').exec();
doc.get$('title').subscribe(newTitle => {
console.log('Title changed to:', newTitle);
});
// Watch the entire change stream of a collection
db.posts.$.subscribe(changeEvent => {
console.log(changeEvent.operation, changeEvent.documentId);
});
```
### Conflict Resolution
When the same document is modified on different clients while one is offline, a conflict occurs when they reconnect. The Supabase client SDK has no mechanism for handling this. RxDB includes a configurable conflict handler:
```ts
await db.addCollections({
posts: {
schema: postSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Last-write-wins by updatedAt timestamp
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative applications where concurrent edits from different users should be merged rather than one discarding the other, RxDB supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md):
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const postSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
published: { type: 'boolean' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDTs, concurrent writes to the same document are merged deterministically when clients sync. No custom conflict handler logic is needed.
### Multiple Storage Backends
RxDB's storage layer is pluggable. You choose the storage engine based on the platform and performance requirements. The rest of your application code remains unchanged:
| Environment | Storage Option |
|---|---|
| Browser | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high-throughput writes) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via expo-sqlite or op-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multiple browser tabs | [SharedWorker](../../rx-storage-shared-worker.md) |
| Tests | [Memory](../../rx-storage-memory.md) |
Switching storage is a one-line change:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
// For React Native:
// import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
// React Native: storage: getRxStorageSQLite({ sqliteBasics })
});
```
The [OPFS storage](../../rx-storage-opfs.md) option is worth noting specifically for applications that Supabase users might build. OPFS gives browsers access to a private file system with low-level read and write operations. This is significantly faster than IndexedDB for write-heavy workloads, because IndexedDB transactions carry significant overhead per operation.
### Multi-Tab Support in the Browser
When a user opens a web application in multiple browser tabs, each tab typically has its own JavaScript runtime. Without coordination, each tab would have its own copy of the local database, and writes from one tab would not appear in others.
RxDB solves this with the [SharedWorker storage](../../rx-storage-shared-worker.md):
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
All tabs share one database instance running in the SharedWorker. A write from tab A appears in tab B's reactive queries immediately, without any additional IPC or state management code.
### Schema Validation and TypeScript Support
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before writing it to storage. Invalid documents are rejected at the database level:
```ts
try {
await db.posts.insert({
id: 'post-002',
// 'title' is required but missing
authorId: 'user-1',
published: true,
updatedAt: Date.now()
});
} catch (err) {
// Rejected: document does not match schema
console.error(err.message);
}
```
RxDB also infers TypeScript types from the schema automatically. You get compile-time type checking and IDE autocompletion for all collection operations:
```ts
// TypeScript knows the shape of this document
const post = await db.posts.findOne('post-001').exec();
if (post) {
console.log(post.title); // string
console.log(post.published); // boolean
}
```
### Schema Migrations
When your data model changes, RxDB's [migration system](../../migration-schema.md) handles the transition automatically. You increment the schema version number and provide a migration strategy:
```ts
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 1, // incremented from 0
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
authorId: { type: 'string', maxLength: 100 },
published: { type: 'boolean' },
slug: { type: 'string' }, // new field
updatedAt: { type: 'number' }
},
required: [
'id',
'title',
'body',
'authorId',
'published',
'slug',
'updatedAt'
]
},
migrationStrategies: {
1: (oldDoc) => {
// Generate a slug from the title
oldDoc.slug = oldDoc.title.toLowerCase().replace(/\s+/g, '-');
return oldDoc;
}
}
}
});
```
When the database is opened with the new schema version, RxDB migrates the existing local documents automatically before the application starts.
### Encryption at Rest
RxDB includes a [built-in encryption plugin](../../encryption.md) for encrypting document fields before writing them to local storage. This is important for mobile applications that store sensitive user data locally:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'user-specific-passphrase'
});
// Fields marked 'encrypted' in the schema are stored as ciphertext
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
sensitiveData: { type: 'string' }
},
encrypted: ['sensitiveData']
};
```
Supabase has no client-side encryption. All data written to the browser's local storage (if you implement local caching yourself) would be stored in plaintext unless you add a separate encryption layer.
---
## RxDB + Supabase as a Stack
RxDB and Supabase are not necessarily alternatives. They work together well:
- **Supabase** serves as the PostgreSQL backend: hosted, accessible, with auth and storage
- **RxDB** serves as the client-side database: local-first, reactive, offline-capable
This combination gives you a complete local-first application stack. The RxDB Supabase replication plugin handles the sync protocol between the two.
```
[User Device]
RxDB (IndexedDB / SQLite)
|
| Supabase Replication Plugin
| (PostgREST pull/push + Realtime WebSocket)
|
[Supabase Cloud]
PostgreSQL
Row Level Security
Auth (GoTrue)
```
You can also add custom backends or migrate away from Supabase later. RxDB supports [HTTP replication](../../replication-http.md), [GraphQL replication](../../replication-graphql.md), [CouchDB replication](../../replication-couchdb.md), [WebSocket replication](../../replication-websocket.md), and [WebRTC replication](../../replication-webrtc.md) without changing any of the application logic that works against the local database.
---
## When to Use Supabase Without RxDB
Supabase alone is appropriate when:
- The application is fully online-only and users will never need data when disconnected
- The data changes infrequently and does not need reactive UI updates
- You need PostgreSQL's relational model and SQL queries on the server without a client-side abstraction
- You are building a backend-heavy application where most logic runs in edge functions or server-side code
For applications that require any of the following, you need a client-side layer like RxDB in addition to Supabase:
- Offline support (the user can read and write data without a network connection)
- Reactive queries that update the UI automatically when data changes
- Multi-tab consistency without full page reloads
- Fast local reads without round-trips to the server for every query
---
## Getting Started
Install RxDB, RxJS, and the Supabase client:
```bash
npm install rxdb rxjs @supabase/supabase-js
```
Create a database and start the replication:
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
import { createClient } from '@supabase/supabase-js';
import { replicateSupabase } from 'rxdb/plugins/replication-supabase';
addRxPlugin(RxDBDevModePlugin);
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
body: { type: 'string' },
published: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'body', 'published', 'updatedAt'],
indexes: ['updatedAt']
}
}
});
const supabase = createClient(SUPABASE_URL, SUPABASE_ANON_KEY);
const replication = replicateSupabase({
tableName: 'posts',
client: supabase,
collection: db.posts,
replicationIdentifier: 'posts-sync-v1',
live: true,
pull: { batchSize: 50 },
push: { batchSize: 50 }
});
await replication.awaitInitialReplication();
// All local queries work offline automatically
db.posts.find({
selector: { published: true },
sort: [{ updatedAt: 'desc' }]
}).$.subscribe(posts => {
console.log('Posts ready:', posts.length);
});
```
From this point, the application reads and writes to IndexedDB, and the replication plugin keeps it in sync with Supabase in the background. Going offline does not break the app.
---
## Comparison Summary
| Aspect | Supabase (alone) | RxDB + Supabase |
|---|---|---|
| **Data location** | Server (PostgreSQL) | Client (IndexedDB/SQLite) + Server |
| **Offline support** | None | Full offline-first |
| **Reactive queries** | Manual re-fetch on WebSocket event | RxJS Observables, auto-updating |
| **Multi-tab consistency** | None (separate fetch per tab) | SharedWorker with unified local DB |
| **Conflict handling** | None built-in | Configurable handler, CRDT support |
| **Query performance** | Network latency on every query | Local storage, sub-millisecond reads |
| **Data model** | Relational (SQL) | Document-based (JSON) |
| **Schema validation** | Database constraints | JSON Schema enforced on every write |
| **TypeScript** | Generated types from schema | Inferred types from JSON Schema |
| **Encryption** | Server-side only | Client-side field encryption |
| **Schema migrations** | SQL ALTER TABLE | Automatic via versioned migration strategies |
| **Backend flexibility** | Supabase only | Supabase, CouchDB, GraphQL, HTTP, WebRTC |
| **Vendor lock-in** | Auth + DB tightly coupled | Swap backend without changing app code |
---
## FAQ
No. RxDB is a client-side database and does not replace a backend. It stores data locally in the browser or on the device. Supabase provides the PostgreSQL backend, authentication, and storage. The two are designed to work together: RxDB handles local data and sync logic, Supabase handles server-side persistence and auth. If you want to sync RxDB with Supabase, use the [Supabase Replication Plugin](../../replication-supabase.md).
Yes. The Supabase replication plugin uses the official `@supabase/supabase-js` client, which works with both hosted and self-hosted Supabase. Point the client at your self-hosted instance URL and the replication plugin will work without any changes.
When both clients reconnect, RxDB detects that the local version and the server version differ. It calls the conflict handler you defined when creating the collection. You decide the resolution strategy: last-write-wins by timestamp, field-level merge, or server-always-wins. For complex collaborative scenarios, RxDB's [CRDT plugin](../../crdt.md) can merge changes from multiple clients automatically without a custom handler.
Yes. The Supabase replication plugin uses the official Supabase JS client, which sends the user's JWT with every request. Your RLS policies apply normally. The plugin does not bypass or override RLS. Each user's RxDB instance only pulls and pushes the rows that their RLS policies permit.
The replication plugin subscribes to the Supabase Realtime channel for the table. When a row changes in PostgreSQL, Realtime broadcasts the event over WebSocket. The plugin receives the event and triggers a pull from PostgREST to fetch the latest changes since the last checkpoint. This approach is robust: even if the WebSocket event is missed, the next scheduled pull will catch the change. No data is lost during temporary disconnections.
Yes. Your application code reads and writes against the local RxDB collection. The replication plugin is configured separately and can be swapped. If you replace Supabase with a different backend (a custom REST API, CouchDB, or a GraphQL server), you change only the replication configuration. The schema, queries, and UI code remain unchanged.
---
## RxDB as a WatermelonDB Alternative - Cross-Platform, Observable, Offline-First
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
import {Timeline} from '@site/src/components/timeline';
# RxDB as a WatermelonDB Alternative
WatermelonDB set out to solve a real problem: React Native apps slowed down under the weight of large datasets loaded entirely into the JavaScript thread. If you are evaluating WatermelonDB for a new project or looking for alternatives to it, this page explains what WatermelonDB does well, where it falls short, and why RxDB covers more ground for teams building offline-first applications across web and mobile.
---
## What is WatermelonDB?
WatermelonDB is a reactive, asynchronous database library created by [Nozbe](https://nozbe.com), a productivity software company. The library was built to solve a specific performance problem inside Nozbe's own React Native apps: as the number of tasks, projects, and comments grew, loading all that data into JavaScript memory at startup made the app feel sluggish. WatermelonDB's answer was **lazy loading**: data is never loaded unless explicitly requested, and all database queries run on a native thread separate from the JavaScript UI thread.
The project was open-sourced around 2018. On React Native, WatermelonDB wraps SQLite with a native bridge (and later a JSI adapter) so that queries execute natively. On the web, it uses LokiJS as an in-memory adapter backed by IndexedDB. WatermelonDB's data model is relational: you define models and associations, and the library generates an Objective-C/Java model layer on native platforms.
### A Brief Timeline
- **2018** - WatermelonDB is open-sourced by Nozbe, targeting React Native performance.
- **2019** - Adoption grows in the React Native community. Web support via LokiJS adapter is added.
- **2020-2021** - A JSI-based SQLite adapter is introduced to remove the async bridge bottleneck on native platforms.
- **2022-2023** - The React Native ecosystem shifts toward the New Architecture (Fabric, TurboModules). WatermelonDB's integration with this new architecture becomes an ongoing compatibility challenge.
- **2024-2025** - Community reports accumulate about stagnant maintenance, build failures on recent React Native versions (0.76+), and unresolved issues with the New Architecture. Developers begin migrating to alternatives.
### How WatermelonDB Works
WatermelonDB uses a **record-based**, relational data model. You define models with typed fields and associations:
```js
import { Model } from '@nozbe/watermelondb';
import { field, date, readonly, relation } from '@nozbe/watermelondb/decorators';
class Post extends Model {
static table = 'posts';
static associations = {
comments: { type: 'has_many', foreignKey: 'post_id' }
};
@field('title') title;
@field('body') body;
@readonly @date('created_at') createdAt;
}
```
Queries return observables that re-emit when the underlying records change:
```js
const posts = database.collections
.get('posts')
.query(Q.where('published', true))
.observe();
posts.subscribe(results => {
console.log('Published posts:', results.length);
});
```
The synchronization layer provides a pull/push protocol that your backend must implement:
```js
import { synchronize } from '@nozbe/watermelondb/sync';
await synchronize({
database,
pullChanges: async ({ lastPulledAt, schemaVersion, migration }) => {
const response = await fetch(`/api/sync/pull?lastPulledAt=${lastPulledAt}`);
const { changes, timestamp } = await response.json();
return { changes, timestamp };
},
pushChanges: async ({ changes, lastPulledAt }) => {
await fetch('/api/sync/push', {
method: 'POST',
body: JSON.stringify({ changes }),
});
},
});
```
---
## Key Limitations of WatermelonDB
### Sync Is Your Problem to Solve
WatermelonDB's documentation is explicit: implementing synchronization is one of the hardest parts of building with the library. The built-in `synchronize()` helper defines a protocol (pull changes since `lastPulledAt`, push local changes), but you must build and maintain every part of the server side yourself.
The protocol has several documented edge cases:
- If a record is modified on the server between the client's pull and push steps, the push may fail or produce incorrect results.
- The protocol pushes entire updated records rather than just changed fields, which is wasteful for large objects.
- There is no built-in mechanism to prevent pulled records from being pushed back to the server on the next cycle, requiring additional logic in your backend.
- The default conflict resolution is "client-wins" for modified columns, which is not suitable for all applications.
For teams without a dedicated backend engineer comfortable with these trade-offs, the sync implementation is a significant time investment that WatermelonDB does not reduce.
### React Native New Architecture Compatibility
WatermelonDB was designed before the React Native New Architecture (Fabric renderer, TurboModules, Bridgeless mode). As React Native 0.76 and later made the New Architecture the default, WatermelonDB users began reporting build failures, Gradle configuration errors, and runtime instability. The library's JSI adapter helps performance but does not resolve the architectural mismatch.
Community discussions through 2024 and into 2025 show that many teams working on React Native 0.76+ encountered issues they could not resolve without reverting to older React Native versions, applying unofficial patches, or switching to a different database library entirely.
### Browser/Web Support Is a Secondary Concern
In the browser, WatermelonDB falls back to the LokiJS adapter, an in-memory JavaScript database backed by IndexedDB. LokiJS is no longer actively maintained. This means the web version of WatermelonDB is built on an unmaintained dependency, and it does not take advantage of modern browser storage APIs like the [Origin Private File System (OPFS)](../../rx-storage-opfs.md), which offers significantly faster persistent storage than IndexedDB for write-heavy workloads.
For teams building apps that run on both web and mobile, WatermelonDB's web support feels like an afterthought compared to the optimized native experience.
### Relational Model Requires Native Code Generation
WatermelonDB generates native model code (Objective-C for iOS, Java/Kotlin for Android) from your schema. This means adding a new table or field to your schema requires a native rebuild of your application. For teams using managed Expo workflows, this is a barrier because native code generation is not compatible with Expo Go.
The tight coupling to native code generation also makes schema migrations more complex. WatermelonDB has a migration system, but it must be coordinated with native builds, which slows down iteration.
### React-Centric API
WatermelonDB's observable and reactive helpers are primarily designed for React hooks. While the core observables are framework-agnostic, the ergonomic layer (such as `withObservables` and the newer `useQuery` hooks) targets React and React Native specifically. Teams building on Vue, Angular, Svelte, or plain JavaScript do not have the same quality of integration helpers.
---
The numbers reflect this. As of July 30, 2026, [WatermelonDB](https://github.com/Nozbe/WatermelonDB) has 11,754 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296, and the `@nozbe/watermelondb` package was downloaded 247,698 times on npm in the last 30 days compared to 270,494 downloads of `rxdb` ([npm trends](https://npmtrends.com/@nozbe/watermelondb-vs-rxdb)). The last commit to the [WatermelonDB repository](https://github.com/Nozbe/WatermelonDB) was in August 2025, .
## How RxDB Addresses These Problems
[RxDB](https://rxdb.info) is a local-first JavaScript database built around the principle that all reads and writes happen against the local storage first, and replication with a server runs in the background. It has been in active development since 2016 and runs on browsers, React Native, Electron, and Node.js with the same API.
### Built-In Replication Protocols
RxDB includes a replication system with ready-made plugins for common backends. You do not need to design a sync protocol from scratch:
| Plugin | Backend |
|---|---|
| [HTTP replication](../../replication-http.md) | Any REST API |
| [CouchDB replication](../../replication-couchdb.md) | CouchDB or compatible (e.g., PouchDB sync server) |
| [GraphQL replication](../../replication-graphql.md) | Any GraphQL API |
| [Firestore replication](../../replication-firestore.md) | Google Firebase Firestore |
| [WebSocket replication](../../replication-websocket.md) | WebSocket-based backends |
| [WebRTC replication](../../replication-webrtc.md) | Peer-to-peer, no server required |
| [Supabase replication](../../replication-http.md) | Supabase Postgres backend |
For a custom backend, the replication interface requires only a pull handler and an optional push handler:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: db.posts,
replicationIdentifier: 'posts-sync-v1',
pull: {
handler: async (checkpoint, batchSize) => {
const since = checkpoint?.updatedAt ?? 0;
const response = await fetch(
`/api/posts/changes?since=${since}&limit=${batchSize}`
);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
handler: async (rows) => {
const response = await fetch('/api/posts/push', {
method: 'POST',
body: JSON.stringify(rows),
headers: { 'Content-Type': 'application/json' }
});
// Return conflicting documents (server wins) or empty array
return response.json();
}
},
live: true,
retryTime: 5000
});
// Monitor replication status
replicationState.active$.subscribe(active => {
console.log('Replication active:', active);
});
replicationState.error$.subscribe(error => {
console.error('Replication error:', error);
});
```
The replication layer handles offline queuing automatically. Writes made while offline are persisted locally and pushed once the network is available again. No data is lost during offline periods.
### Conflict Handling You Control
WatermelonDB's built-in sync uses a simple "client-wins" strategy for field-level conflicts. RxDB gives you a configurable conflict handler that runs whenever a document exists in different states on the client and server simultaneously:
```ts
await db.addCollections({
posts: {
schema: postSchema,
conflictHandler: async (input) => {
const { newDocumentState, realMasterState } = input;
// Merge strategy: keep the most recently updated version
if (newDocumentState.updatedAt >= realMasterState.updatedAt) {
return { documentData: newDocumentState };
}
return { documentData: realMasterState };
}
}
});
```
For collaborative editing scenarios, RxDB also supports [CRDTs (Conflict-free Replicated Data Types)](../../crdt.md), which resolve conflicts automatically and deterministically without custom handler logic:
```ts
import { getCRDTSchemaPart, RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBcrdtPlugin);
const taskSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
completed: { type: 'boolean' },
crdts: getCRDTSchemaPart()
},
crdt: { field: 'crdts' }
};
```
With CRDTs, concurrent writes to the same document from different clients are merged automatically when they sync. This is useful for applications where users work offline for extended periods and their changes should be preserved rather than overwritten.
### Cross-Platform Storage That Matches the Environment
RxDB's storage system is pluggable. You choose the storage engine based on your deployment target, and the rest of your application code stays the same:
| Environment | Storage Option |
|---|---|
| Browser | [IndexedDB](../../rx-storage-indexeddb.md) |
| Browser (high-throughput) | [OPFS (Origin Private File System)](../../rx-storage-opfs.md) |
| React Native / Expo | [SQLite via op-sqlite or expo-sqlite](../../rx-storage-sqlite.md) |
| Node.js / Electron | [SQLite (better-sqlite3)](../../rx-storage-sqlite.md) |
| Multi-tab browsers | [SharedWorker](../../rx-storage-shared-worker.md) |
| Tests / CI | [Memory](../../rx-storage-memory.md) |
Switching storage is a one-line change to the `storage` parameter when creating the database:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
// Or for React Native:
// import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageIndexedDB()
// storage: getRxStorageSQLite({ sqliteBasics: sqliteBasics })
});
await db.addCollections({
posts: {
schema: {
title: 'post schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
authorId: { type: 'string', maxLength: 100 },
published: { type: 'boolean' },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'authorId', 'published', 'updatedAt'],
indexes: ['updatedAt', 'authorId']
}
}
});
```
The OPFS storage option is particularly relevant for teams moving away from WatermelonDB's web adapter. OPFS gives browsers access to a private file system with low-level read and write operations that are [significantly faster](../../rx-storage-opfs.md) than standard IndexedDB for bulk reads and sequential writes. Unlike the LokiJS adapter in WatermelonDB, OPFS is a modern, browser-native API maintained by the W3C.
### Framework-Agnostic Reactive Queries
RxDB builds its reactive layer on [RxJS](https://rxjs.dev), the industry-standard library for reactive programming in JavaScript. Every query exposes an Observable via the `$` property. The Observable emits the current result set on subscription and re-emits whenever the underlying data changes, without polling:
```ts
// Subscribe to published posts sorted by most recent
const publishedPosts$ = db.posts
.find({
selector: { published: true },
sort: [{ updatedAt: 'desc' }]
})
.$;
publishedPosts$.subscribe(posts => {
renderPostList(posts);
});
```
Because this is a standard RxJS Observable, you can use it in React, Vue, Angular, Svelte, SolidJS, or plain JavaScript without any framework-specific adapter. In React:
```tsx
import { useRxQuery } from 'rxdb-hooks';
function PostList({ db }) {
const { result: posts, isFetching } = useRxQuery(
db.posts.find({
selector: { published: true },
sort: [{ updatedAt: 'desc' }]
})
);
if (isFetching) return Loading...;
return {posts.map(p => {p.title})};
}
```
RxDB also lets you subscribe at a more granular level: individual documents, specific fields on a document, or the collection's change stream:
```ts
// Watch a single field on a single document
const doc = await db.posts.findOne('post-001').exec();
doc.get$('title').subscribe(newTitle => {
console.log('Title changed to:', newTitle);
});
// Watch the entire collection change stream
db.posts.$.subscribe(changeEvent => {
console.log(changeEvent.operation, changeEvent.documentId);
});
```
RxDB uses the [event-reduce](https://github.com/pubkey/event-reduce) algorithm to update observable query results efficiently. When a document is written, RxDB checks whether the change can update the existing query result directly, without re-running the full query against storage. This keeps UI updates fast even in write-heavy scenarios.
### React Native Support Without Native Code Generation
RxDB on React Native uses the [SQLite storage plugin](../../rx-storage-sqlite.md), which supports multiple underlying SQLite drivers including `expo-sqlite` (for Expo managed workflows) and `op-sqlite` (a high-performance JSI-based driver). No custom Objective-C or Java code generation is required. You define your schema in TypeScript and the library handles everything else:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageSQLite } from 'rxdb/plugins/storage-sqlite';
import { sqliteBasics } from 'rxdb/plugins/storage-sqlite/expo-sqlite';
import * as ExpoSQLite from 'expo-sqlite';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSQLite({
sqliteBasics: sqliteBasics(ExpoSQLite)
})
});
```
This works in Expo managed workflows without ejecting or running a prebuild. The same schema, query code, and replication configuration that runs in your web application can be reused in the React Native app, because the storage layer is swapped independently of the application logic.
RxDB also runs correctly on the React Native New Architecture (Fabric, TurboModules, Bridgeless mode) without the compatibility issues that WatermelonDB users have encountered on recent React Native versions.
### Schema Validation and TypeScript
RxDB validates every document against a [JSON Schema](../../rx-schema.md) before it is written. Invalid documents are rejected at the database level, not by application-layer checks:
```ts
try {
await db.posts.insert({
id: 'post-002',
// 'title' is a required field but is missing
authorId: 'user-1',
published: true,
updatedAt: Date.now()
});
} catch (err) {
// Caught: document does not match schema
console.error(err.message);
}
```
RxDB also generates TypeScript types automatically from the schema, so you get IDE autocompletion and compile-time type checking for all collection operations. WatermelonDB provides TypeScript support through decorators, but the type safety is not as tight because the field types come from JavaScript property decorators rather than a formal schema definition.
### Multi-Tab Support in the Browser
When a user opens your web application in multiple browser tabs, each tab needs access to the same data. RxDB handles this with its [SharedWorker storage](../../rx-storage-shared-worker.md): all tabs connect to a single database instance running in a SharedWorker, so writes from any tab are immediately visible in all others:
```ts
import { getRxStorageSharedWorker } from 'rxdb/plugins/storage-shared-worker';
const db = await createRxDatabase({
name: 'myapp',
storage: getRxStorageSharedWorker({
workerInput: new SharedWorker(
new URL('rxdb/plugins/storage-shared-worker/worker.js', import.meta.url),
{ type: 'module' }
)
})
});
```
WatermelonDB's web adapter does not have a multi-tab coordination mechanism because LokiJS is an in-memory database: each tab has its own independent in-memory store. Keeping those stores in sync requires additional application-level logic.
### Encryption
RxDB has a [built-in encryption plugin](../../encryption.md) that encrypts document fields at rest. This is particularly relevant for mobile applications that store sensitive data locally. You can mark individual schema fields as encrypted:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
const db = await createRxDatabase({
name: 'myapp',
storage: wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageIndexedDB()
}),
password: 'your-encryption-passphrase'
});
// Fields marked as encrypted in the schema are stored as ciphertext
const schema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
token: { type: 'string' }
},
encrypted: ['token']
};
```
---
## Positioning: When to Choose RxDB
WatermelonDB is most valuable for React Native applications with very large datasets (tens of thousands of records) where the primary requirement is fast query execution on the native thread and you have the engineering capacity to build and maintain a custom sync backend.
RxDB is a better fit when:
- You need a sync layer that works out of the box against a common backend (CouchDB, GraphQL, Firestore, or a custom REST API).
- Your application must run on both web and mobile with shared code.
- You are using the React Native New Architecture and need stable support.
- You are using Expo managed workflows and cannot add native modules.
- You want a single database library that works in browsers (including with OPFS for performance), React Native, Electron, and Node.js.
- Your conflict resolution requirements go beyond "client-wins" and you want control over how concurrent edits are merged.
- Your team builds with Vue, Angular, Svelte, or plain JavaScript and needs framework-agnostic reactive queries.
---
## Getting Started with RxDB
Install RxDB and RxJS:
```bash
npm install rxdb rxjs
```
Create a database, define a collection, and subscribe to reactive queries:
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { getRxStorageIndexedDB } from 'rxdb/plugins/storage-indexeddb';
addRxPlugin(RxDBDevModePlugin);
const db = await createRxDatabase({
name: 'taskapp',
storage: getRxStorageIndexedDB()
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
completed: { type: 'boolean' },
projectId: { type: 'string', maxLength: 100 },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'completed', 'projectId', 'updatedAt'],
indexes: ['projectId', 'updatedAt']
}
}
});
// Insert a task
await db.tasks.insert({
id: 'task-001',
title: 'Write documentation',
completed: false,
projectId: 'proj-rxdb',
updatedAt: Date.now()
});
// Subscribe to all incomplete tasks in a project
db.tasks.find({
selector: { projectId: 'proj-rxdb', completed: false },
sort: [{ updatedAt: 'desc' }]
}).$.subscribe(tasks => {
console.log('Pending tasks:', tasks.map(t => t.title));
});
```
To add replication with your backend, import `replicateRxCollection` and point it at your API endpoints. The local database continues to work offline regardless of the replication state.
---
## Comparison Summary
| Aspect | WatermelonDB | RxDB |
|---|---|---|
| **Primary focus** | React Native performance with large datasets | Offline-first across web, mobile, Node.js |
| **Data model** | Relational (SQLite-based, decorators) | Document-based (JSON, JSON Schema) |
| **Reactive queries** | Observables (React-focused helpers) | RxJS Observables (framework-agnostic) |
| **Built-in replication** | Pull/push protocol scaffold only | HTTP, CouchDB, GraphQL, WebSocket, WebRTC |
| **Conflict handling** | Client-wins by default | Configurable handler, CRDT support |
| **Browser storage** | LokiJS in-memory (unmaintained dependency) | IndexedDB, OPFS (modern, fast) |
| **React Native storage** | SQLite via JSI adapter | SQLite (expo-sqlite, op-sqlite) |
| **Expo managed workflow** | Requires prebuild / native modules | Supported without prebuild |
| **React Native New Arch** | Compatibility issues (2024-2025) | Works with New Architecture |
| **Multi-tab browser support** | None (in-memory per tab) | SharedWorker for unified state |
| **Schema validation** | None built-in | JSON Schema enforced on every write |
| **TypeScript** | Decorators-based (partial) | Auto-generated from schema (full) |
| **Encryption** | Not built-in | Built-in encryption plugin |
| **Cross-framework** | Primarily React / React Native | React, Vue, Angular, Svelte, plain JS |
| **Maintenance status** | Reduced activity, New Arch issues | Active development since 2016 |
| **Backend requirement** | You build it | Optional; many ready-made plugins |
| **License** | MIT | Apache 2.0 |
---
## FAQ
RxDB on React Native uses SQLite through drivers like `op-sqlite`, which is a JSI-based SQLite driver. This puts RxDB in the same performance class as WatermelonDB's native SQLite adapter for most workloads. RxDB also runs queries outside the UI thread when using the [Worker storage plugin](../../rx-storage-worker.md), which prevents database work from blocking React Native's JavaScript thread.
Yes. RxDB supports `expo-sqlite` as a storage backend through the SQLite plugin. This works in Expo managed workflows without ejecting or running `expo prebuild`. You can also use the [Memory storage](../../rx-storage-memory.md) for tests in a Node.js environment without any native dependencies.
RxDB has a built-in [migration system](../../migration-schema.md). When you increment the schema `version` number, you provide migration strategies that transform documents from the old shape to the new shape. RxDB runs these migrations automatically when the database is opened with a newer schema version. No native rebuild is required, and migrations run against the stored documents in the local database.
Yes, as long as the API can express the two operations RxDB needs: a way to fetch documents changed since a given checkpoint, and a way to submit local changes. The exact endpoint shape and authentication are entirely up to you. The `pull.handler` and `push.handler` functions in the replication config are plain async JavaScript functions that can call any API using `fetch`, Axios, or any other HTTP client.
Yes. On the web, RxDB stores all data in IndexedDB or OPFS, both of which are persistent browser storage mechanisms. If the user goes offline, the application continues to read and write data normally. When the connection returns, replication resumes automatically from the last checkpoint. The experience is identical regardless of whether the storage backend is IndexedDB in a browser or SQLite on a mobile device.
---
## RxDB as a Yjs Alternative for Local-First Apps with Queries and Persistence
import {Faq, FaqItem} from '@site/src/components/faq';
import {ComparisonTable} from '@site/src/components/comparison-table';
# RxDB as a Yjs Alternative for Local-First Apps with Queries and Persistence
[Yjs](https://github.com/yjs/yjs) is a CRDT runtime that solves one problem well: merging concurrent edits to shared data structures without a central authority. It is the engine behind many collaborative editors built on TipTap, ProseMirror, Slate, and Monaco. When your application is mostly a shared text document, Yjs is an excellent fit.
The trouble starts when the application also has lists, settings, user profiles, attachments, search, and reporting. Those features need indexes, schemas, queries, durable storage, and a sync model that goes beyond merging characters into a rope. Yjs does not provide any of that on its own. You assemble it from third party providers, and you write the indexing and query layer yourself.
This page explains where Yjs ends, where [RxDB](https://rxdb.info/) begins, and how to combine the two when you need both rich-text collaboration and a structured local database.
## A Short History of Yjs
Yjs was started around 2014 by Kevin Jahns as a research project on operation-based CRDTs. Over the following years it grew into the de facto CRDT library for the JavaScript ecosystem. The shared types `Y.Doc`, `Y.Text`, `Y.Array`, and `Y.Map` became the foundation for collaborative editor bindings such as `y-prosemirror`, `y-tiptap`, `y-monaco`, and `y-codemirror`.
Around the core library, a set of providers handles transport and persistence:
- `y-websocket` for server-relayed sync.
- `y-webrtc` for peer-to-peer sync over WebRTC.
- `y-indexeddb` for browser persistence.
- `y-leveldb` for server-side persistence.
Each provider plugs into a `Y.Doc` and synchronizes its update stream. The model is simple and effective for documents, and it is the reason Yjs dominates the collaborative editor space.
## What RxDB Brings to the Table
[RxDB](https://rxdb.info/) is a [local-first](../../articles/local-first-future.md), reactive, NoSQL database for JavaScript. It stores JSON documents in a pluggable storage layer, exposes MongoDB-style queries, and supports several replication protocols out of the box. Queries and documents are observable, so any UI bound to them updates automatically when the underlying data changes. See the [reactivity guide](../../reactivity.md) for details.
RxDB also ships an optional [CRDT plugin](../../crdt.md) that adds operation-based merging on top of regular RxDB documents. If you want CRDT semantics for parts of your data without giving up schemas, queries, and indexes, you do not need a separate CRDT runtime.
## Where Yjs Falls Short as a General Database
Yjs was designed as a CRDT runtime, not a database. The following gaps appear once you try to model an entire application on top of it:
### No Query Engine
Yjs ships shared types like `Y.Map` and `Y.Array`, but there is no query language. Filtering, sorting, joining, or paginating data means iterating over shared types in JavaScript and building the result set by hand. There is no equivalent of `collection.find({ status: 'open' }).sort({ updatedAt: -1 })`.
### No Schema or Validation
A `Y.Map` accepts any key and any value. There is no schema, no required fields, no type checking, and no migration story. Application code is responsible for guarding every read and write. Schema drift between clients running different app versions is a recurring source of bugs.
### No Indexes
Lookups in Yjs are linear scans of the shared types. There is no secondary index, no compound index, and no way to ask the storage layer to keep one. For a thousand documents this is fine. For a hundred thousand, every list view becomes a full traversal.
### No Aggregation
Yjs has no `count`, no `group by`, no `sum`. If you need a dashboard or any derived view, you compute it in application code on every change.
### Persistence Is Provider by Provider
Persistence is delegated to providers like `y-indexeddb` or `y-leveldb`. Each provider has its own format, its own quirks, and its own lifecycle. Switching environments, for example moving from browser to React Native or Node.js, means swapping the provider and accepting whatever it offers.
### Sync Is Document Shaped
Yjs sync moves the entire update stream of a `Y.Doc`. That works for one shared document. For an app with thousands of independent records (orders, messages, contacts), you either pack everything into one giant `Y.Doc` and pay for it on every load, or you manage many small docs and write your own catalog, fan-out, and authorization layer.
Both projects are under active development. As of July 30, 2026, [Yjs](https://github.com/yjs/yjs) has 22,263 GitHub stars while [RxDB](https://github.com/pubkey/rxdb) has 23,296.
## Where RxDB Fits
RxDB approaches the same problem from the database side:
- **Document database**: collections of JSON documents with a [JSON Schema](../../rx-collection.md) per collection.
- **MongoDB-style queries**: rich [`RxQuery`](../../rx-query.md) API with selectors, sort, skip, limit, and indexes.
- **Observable everything**: queries, documents, and fields emit on every change. See [reactivity](../../reactivity.md).
- **Pluggable storage**: IndexedDB, OPFS, SQLite, in-memory, and more. The same code runs in browser, Electron, React Native, and Node.js.
- **Replication primitives**: a generic [replication protocol](../../replication.md) plus ready-made adapters for HTTP, GraphQL, WebSocket, CouchDB, and [WebRTC](../../replication-webrtc.md).
- **Conflict handling**: per-collection [conflict handlers](../../transactions-conflicts-revisions.md) with revisions, so you can merge, prefer remote, prefer local, or fold in CRDT logic.
- **Optional CRDT plugin**: the [CRDT plugin](../../crdt.md) adds operation-based merging where you need it.
## Code Sample: Defining a Collection and Subscribing to a Query
A typical RxDB collection has a schema, supports queries, and emits updates over time:
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
const db = await createRxDatabase({
name: 'app',
storage: getRxStorageDexie()
});
await db.addCollections({
tasks: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 40 },
title: { type: 'string' },
status: { type: 'string', enum: ['open', 'done'] },
updatedAt: { type: 'number' }
},
required: ['id', 'title', 'status', 'updatedAt'],
indexes: ['status', 'updatedAt']
}
}
});
await db.tasks.insert({
id: 't1',
title: 'Write Yjs comparison',
status: 'open',
updatedAt: Date.now()
});
const openTasks$ = db.tasks
.find({ selector: { status: 'open' } })
.sort({ updatedAt: 'desc' })
.$;
openTasks$.subscribe(tasks => {
console.log('open tasks:', tasks.map(t => t.title));
});
```
Schemas, indexes, queries, and live results come from the database itself. Compare this to building the same view on top of a `Y.Array` of `Y.Map`.
## Code Sample: RxDB CRDT Plugin for Collaborative Fields
When a field needs CRDT semantics, the [CRDT plugin](../../crdt.md) lets you express updates as operations against a regular RxDB document:
```ts
import { addRxPlugin } from 'rxdb';
import { RxDBcrdtPlugin } from 'rxdb/plugins/crdt';
addRxPlugin(RxDBcrdtPlugin);
await db.addCollections({
shoppingLists: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
crdt: { field: 'crdts' },
properties: {
id: { type: 'string', maxLength: 40 },
items: { type: 'array', items: { type: 'string' } },
crdts: { type: 'object' }
},
required: ['id']
}
}
});
const list = await db.shoppingLists.insert({ id: 'home', items: [] });
await list.updateCRDT({
ifMatch: { $set: { items: ['milk'] } }
});
await list.updateCRDT({
ifMatch: { $push: { items: 'bread' } }
});
```
Two clients running these operations in any order converge to the same result, with the same query, schema, and replication infrastructure as every other collection.
## Use Both: Yjs for Rich Text, RxDB for Everything Else
Yjs and RxDB are not mutually exclusive. A common pattern in production apps:
- **Yjs** handles the rich-text body of documents through a `Y.Doc` per document, edited via TipTap or ProseMirror, synced through `y-websocket` or `y-webrtc`.
- **RxDB** handles document metadata, the document list, comments, permissions, search indexes, attachments, user settings, and offline queues.
The serialized Yjs update (a `Uint8Array`) can be stored as a base64 string inside an RxDB document field. RxDB takes care of persistence, replication, and querying the metadata, while Yjs takes care of merging concurrent character edits.
For real-time fan-out between peers, RxDB's [WebRTC replication](../../replication-webrtc.md) and the standard [server replication](../../replication.md) cover the structured side, and the existing Yjs providers cover the document side. See also the [realtime database article](../../articles/realtime-database.md) for the broader pattern.
## FAQ
No. Yjs is a CRDT library. It defines shared data types and a merge algorithm, and it leaves persistence, networking, indexing, and queries to providers and to the application. A database stores, indexes, and queries data. Yjs does the merging part of that picture and nothing else.
Yes. The optional [CRDT plugin](../../crdt.md) adds operation-based merging on top of regular RxDB documents. You keep schemas, queries, indexes, and replication, and you opt in to CRDT semantics for the fields that need them. For most app data, the default [conflict handler](../../transactions-conflicts-revisions.md) is enough.
Yes. A `Y.Doc` can be encoded with `Y.encodeStateAsUpdate` and stored as a binary or base64 field inside an RxDB document. RxDB then handles persistence and replication of the encoded blob, while Yjs handles merging in memory when the document is opened.
Yjs CRDTs guarantee deterministic convergence for the built-in shared types, with no application code involved. RxDB conflict resolvers are per-collection functions that take the local and remote versions and return the merged result. They are more general, since they can express last-write-wins, field-level merges, business rules, or full CRDT logic via the CRDT plugin. They require you to define the merge policy explicitly.
Yjs, paired with TipTap, ProseMirror, Slate, or Monaco. The bindings are mature and the merge semantics for text are exactly what editors need. RxDB is the better choice for the surrounding application: the document list, metadata, comments, permissions, offline queue, and search.
## Comparison Table
| Capability | Yjs | RxDB |
| ---------------------------------- | ------------------------------------ | -------------------------------------------------------- |
| Primary purpose | CRDT runtime | Local-first document database |
| Data model | Shared types (`Y.Map`, `Y.Array`, `Y.Text`) | JSON documents in typed collections |
| Schema and validation | None | JSON Schema per collection |
| Query API | None, manual iteration | MongoDB-style [`RxQuery`](../../rx-query.md) |
| Indexes | None | Single and compound indexes |
| Reactive results | Per shared type observers | Observable [queries and documents](../../reactivity.md) |
| Persistence | Provider based (`y-indexeddb`, etc.) | Pluggable storages (IndexedDB, OPFS, SQLite, memory) |
| Replication | Provider based, per `Y.Doc` | Generic [replication protocol](../../replication.md), HTTP, GraphQL, WebSocket, CouchDB, [WebRTC](../../replication-webrtc.md) |
| Conflict resolution | Built-in CRDT | Pluggable [conflict handlers](../../transactions-conflicts-revisions.md) plus optional [CRDT plugin](../../crdt.md) |
| Best fit | Collaborative rich-text editors | Offline-first apps with structured data and queries |
If your product is a collaborative editor, start with Yjs. If your product is an app that happens to need collaboration on some fields, start with RxDB and add the [CRDT plugin](../../crdt.md) or embed Yjs documents where they pay off.
---
## RxDB as a Database in an Angular Application
import {VideoBox} from '@site/src/components/video-box';
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB as a Database in an Angular Application
In modern web development, Angular has emerged as a popular framework for building robust and scalable applications. As Angular applications often require persistent [storage](./browser-storage.md) and efficient data handling, choosing the right database solution is crucial. One such solution is [RxDB](https://rxdb.info/), a reactive JavaScript database for the [browser](./browser-database.md), [node.js](../nodejs-database.md), and [mobile devices](./mobile-database.md). In this article, we will explore the integration of RxDB into an Angular application and examine its various features and techniques.
## Angular Web Applications
Angular is a powerful JavaScript framework developed and maintained by Google. It enables developers to build single-page applications (SPAs) with a modular and component-based approach. Angular provides a comprehensive set of tools and features for creating dynamic and responsive web applications.
## Importance of Databases in Angular Applications
Databases play a vital role in Angular applications by providing a structured and efficient way to store, retrieve, and manage data. Whether it's handling user authentication, caching data, or persisting application state, a robust database solution is essential for ensuring optimal performance and user experience.
## Introducing RxDB as a Database Solution
RxDB stands for Reactive Database and is built on the principles of reactive programming. It combines the best features of [NoSQL databases](./in-memory-nosql-database.md) with the power of reactive programming to provide a scalable and efficient database solution. RxDB offers seamless integration with Angular applications and brings several unique features that make it an attractive choice for developers.
## Getting Started with RxDB
To begin our journey with RxDB, let's understand its key concepts and features.
### What is RxDB?
[RxDB](https://rxdb.info/) is a client-side database that follows the principles of reactive programming. It is built on top of IndexedDB, the [native browser database](./browser-database.md), and leverages the RxJS library for reactive data handling. RxDB provides a simple and intuitive API for managing data and offers features like data [replication](../replication.md), multi-tab support, and efficient query handling.
### Reactive Data Handling
At the core of RxDB is the concept of reactive data handling. RxDB leverages observables and reactive streams to enable real-time updates and data synchronization. With RxDB, you can easily subscribe to data changes and react to them in a reactive and efficient manner.
### Offline-First Approach
One of the standout features of RxDB is its offline-first approach. It allows you to build applications that can work seamlessly in offline scenarios. RxDB stores data locally and automatically synchronizes changes with the server when the network becomes available. This capability is particularly useful for applications that need to function in low-connectivity or unreliable network environments.
### Data Replication
RxDB provides built-in support for data replication between clients and servers. This means you can synchronize data across multiple devices or instances of your application effortlessly. RxDB handles [conflict resolution](../transactions-conflicts-revisions.md) and ensures that data remains consistent across all connected clients.
### Observable Queries
RxDB offers a powerful querying mechanism with support for [observable queries](../rx-query.md). This allows you to create dynamic queries that automatically update when the underlying data changes. By leveraging RxDB's observable queries, you can build reactive UI components that respond to data changes in real-time.
### Multi-Tab Support
RxDB provides out-of-the-box support for multi-tab scenarios. This means that if your Angular application is running in multiple browser tabs, RxDB automatically keeps the data in sync across all tabs. It ensures that changes made in one tab are immediately reflected in others, providing a seamless user experience.
### RxDB vs. Other Angular Database Options
While there are other database options available for Angular applications, RxDB stands out with its reactive programming model, offline-first approach, and built-in synchronization capabilities. Unlike traditional SQL databases, RxDB's NoSQL-like structure and observables-based API make it well-suited for real-time applications and complex data scenarios.
## Using RxDB in an Angular Application
Now that we have a good understanding of RxDB and its features, let's explore how to integrate it into an Angular application.
### Installing RxDB in an Angular App
To use RxDB in an Angular application, we first need to install the necessary dependencies. You can install RxDB using npm or yarn by running the following command:
```bash
npm install rxdb --save
```
Once installed, you can import RxDB into your Angular application and start using its API to create and manage databases.
### Patch Change Detection with zone.js
Angular uses change detection to detect and update UI elements when data changes. However, RxDB's data handling is based on observables, which can sometimes bypass Angular's change detection mechanism. To ensure that changes made in RxDB are detected by Angular, we need to patch the change detection mechanism using zone.js. Zone.js is a library that intercepts and tracks asynchronous operations, including observables. By patching zone.js, we can make sure that Angular is aware of changes happening in RxDB.
:::warning
RxDB creates rxjs observables outside of angulars zone
So you have to import the rxjs patch to ensure the [angular change detection](https://angular.io/guide/change-detection) works correctly.
[link](https://www.bennadel.com/blog/3448-binding-rxjs-observable-sources-outside-of-the-ngzone-in-angular-6-0-2.htm)
```ts
//> app.component.ts
import 'zone.js/plugins/zone-patch-rxjs';
```
:::
### Use the Angular async pipe to observe an RxDB Query
Angular provides the async pipe, which is a convenient way to subscribe to observables and handle the subscription lifecycle automatically. When working with RxDB, you can use the async pipe to observe an RxDB query and bind the results directly to your Angular template. This ensures that the UI stays in sync with the data changes emitted by the RxDB query.
```ts
constructor(
private dbService: DatabaseService,
private dialog: MatDialog
) {
this.heroes$ = this.dbService
.db.hero // collection
.find({ // query
selector: {},
sort: [{ name: 'asc' }]
})
.$;
}
```
```html
{{hero.name}}
```
### Different RxStorage layers for RxDB
RxDB supports multiple storage layers for persisting data. Some of the available storage options include:
- [LocalStorage RxStorage](../rx-storage-localstorage.md): Uses the [LocalStorage API](./localstorage.md) without any third party plugins.
- [IndexedDB RxStorage](../rx-storage-indexeddb.md): RxDB directly supports IndexedDB as a storage layer. IndexedDB is a low-level browser database that offers good performance and reliability.
- [OPFS RxStorage](../rx-storage-opfs.md): The OPFS [RxStorage](../rx-storage.md) for RxDB is built on top of the [File System Access API](https://webkit.org/blog/12257/the-file-system-access-api-with-origin-private-file-system/) which is available in [all modern browsers](https://caniuse.com/native-filesystem-api). It provides an API to access a sandboxed private file system to persistently store and retrieve data.
Compared to other persistent storage options in the browser (like [IndexedDB](../rx-storage-indexeddb.md)), the OPFS API has a **way better performance**.
- [Memory RxStorage](../rx-storage-memory.md): In addition to persistent storage options, RxDB also provides a memory-based storage layer. This is useful for testing or scenarios where you don't need long-term data persistence.
You can choose the storage layer that best suits your application's requirements and configure RxDB accordingly.
## Synchronizing Data with RxDB between Clients and Servers
Data replication between an Angular application and a server is a common requirement. RxDB simplifies this process and provides built-in support for data synchronization. Let's explore how to replicate data between an Angular application and a server using RxDB.
### Offline-First Approach
One of the key strengths of RxDB is its [offline-first approach](../offline-first.md). It allows Angular applications to function seamlessly even in offline scenarios. RxDB stores data locally and automatically synchronizes changes with the server when the network becomes available. This capability is particularly useful for applications that need to operate in low-connectivity or unreliable network environments.
### Conflict Resolution
In a distributed system, conflicts can arise when multiple clients modify the same data simultaneously. RxDB offers conflict resolution mechanisms to handle such scenarios. You can define conflict resolution strategies based on your application's requirements. RxDB provides hooks and events to detect conflicts and resolve them in a consistent manner.
### Bidirectional Synchronization
RxDB supports bidirectional data synchronization, allowing updates from both the client and server to be replicated seamlessly. This ensures that data remains consistent across all connected clients and the server. RxDB handles conflicts and resolves them based on the defined conflict resolution strategies.
### Real-Time Updates
RxDB provides real-time updates by leveraging reactive programming principles. Changes made to the data are automatically propagated to all connected clients in real-time. Angular applications can subscribe to these updates and update the user interface accordingly. This real-time capability enables collaborative features and enhances the overall user experience.
## Advanced RxDB Features and Techniques
RxDB offers several advanced features and techniques that can further enhance your Angular application.
### Indexing and Performance Optimization
To improve query performance, RxDB allows you to define indexes on specific fields of your documents. Indexing enables faster data retrieval and query execution, especially when working with large datasets. By strategically creating indexes, you can optimize the performance of your Angular application.
### Encryption of Local Data
RxDB provides built-in support for [encrypting](../encryption.md) local data using the Web Crypto API. With encryption, you can protect sensitive data stored in the client-side database. RxDB transparently encrypts the data, ensuring that it remains secure even if the underlying storage is compromised.
### Change Streams and Event Handling
RxDB exposes change streams, which allow you to listen for data changes at a database or collection level. By subscribing to change streams, you can react to data modifications and perform specific actions, such as updating the UI or triggering notifications. Change streams enable real-time event handling in your Angular application.
### JSON Key Compression
To reduce the storage footprint and improve performance, RxDB supports [JSON key compression](../key-compression.md). With key compression, RxDB replaces long keys with shorter aliases, reducing the overall storage size. This optimization is particularly useful when working with large datasets or frequently updating data.
## Best Practices for Using RxDB in Angular Applications
To make the most of RxDB in your Angular application, consider the following best practices:
### Use Async Pipe for Subscriptions so you do not have to unsubscribe
Angular's `async` pipe is a powerful tool for handling observables in templates. By using the async pipe, you can avoid the need to manually subscribe and unsubscribe from RxDB observables. Angular takes care of the subscription lifecycle, ensuring that resources are released when they are no longer needed. Instead of manually subscribing to Observables, you should always prefer the `async` pipe.
```ts
// WRONG:
let amount;
this.dbService
.db.hero
.find({
selector: {},
sort: [{ name: 'asc' }]
})
.$.subscribe(docs => {
amount = 0;
docs.forEach(d => amount = d.points);
});
// RIGHT:
this.amount$ = this.dbService
.db.hero
.find({
selector: {},
sort: [{ name: 'asc' }]
})
.$.pipe(
map(docs => {
let amount = 0;
docs.forEach(d => amount = d.points);
return amount;
})
);
```
### Use custom reactivity to have signals instead of rxjs observables
RxDB supports adding custom reactivity factories that allow you to get angular signals out of the database instead of rxjs observables. [read more](../reactivity.md).
### Use Angular Services for Database creation
To ensure proper separation of concerns and maintain a clean codebase, it is recommended to create an Angular service responsible for managing the RxDB database instance. This service can handle database creation, initialization, and provide methods for interacting with the database throughout your application.
### Efficient Data Handling
RxDB provides various mechanisms for efficient data handling, such as batching updates, debouncing, and throttling. Leveraging these techniques can help optimize performance and reduce unnecessary UI updates. Consider the specific data handling requirements of your application and choose the appropriate strategies provided by RxDB.
### Data Synchronization Strategies
When working with data synchronization between clients and servers, it's important to consider strategies for conflict resolution and handling network failures. RxDB provides plugins and hooks that allow you to customize the replication behavior and implement specific synchronization strategies tailored to your application's needs.
## FAQ
You fetch, save, and manage data robustly in Angular by coupling your UI directly to a reactive local-first database like RxDB. Because RxDB queries return native RxJS `Observable` objects, you can skip writing manual HTTP request management services and instead pipe RxDB observables straight into your Angular components using the `AsyncPipe`. All CRUD operations hit the local IndexedDB instantly, and RxDB synchronizes the data with your remote server automatically in the background.
## Conclusion
RxDB is a powerful database solution for Angular applications, offering reactive data handling, offline-first capabilities, and seamless data synchronization. By integrating RxDB into your Angular application, you can build responsive and scalable web applications that provide a rich user experience. Whether you're building real-time collaborative apps, progressive web applications, or offline-capable applications, RxDB's features and techniques make it a valuable addition to your Angular development toolkit.
## Follow Up
To explore more about RxDB and leverage its capabilities for browser database development, check out the following resources:
- [RxDB GitHub Repository](https://github.com/pubkey/rxdb): Visit the official GitHub repository of RxDB to access the source code, documentation, and community support.
- [RxDB Quickstart](../quickstart.md): Get started quickly with RxDB by following the provided quickstart guide, which provides step-by-step instructions for setting up and using RxDB in your projects.
- [RxDB Angular Example at GitHub](https://github.com/pubkey/rxdb/tree/master/examples/angular)
---
## Build Smarter Offline-First Angular Apps - How RxDB Beats IndexedDB Alone
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# Build Smarter Offline-First Angular Apps: How RxDB Beats IndexedDB Alone
In modern web applications, offline capabilities and fast interactions are crucial. IndexedDB, the [browser](./browser-database.md)'s built-in database, allows you to store data locally, making your Angular application more robust and responsive. However, IndexedDB can be cumbersome to work with directly. That's where RxDB (Reactive Database) shines. In this article, we'll walk you through how to utilize IndexedDB in your Angular project using [RxDB](https://rxdb.info/) as a convenient abstraction layer.
## What Is IndexedDB?
[IndexedDB](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API) is a low-level JavaScript API for client-side storage of large amounts of structured data. It allows you to create key-value or object store-based data storage right in the user's browser. IndexedDB supports transactions and indexing but lacks a robust query API and can be complex to use due to its callback-based nature.
## Why Use IndexedDB in Angular
- [Offline-First](../offline-first.md)/[Local-First](./local-first-future.md): If your app needs to function with limited or no internet connectivity, IndexedDB provides a reliable local storage layer. Users can continue using the application offline, and data can sync when the connection is restored.
- **Performance**: Local data access comes with [near-zero latency](./zero-latency-local-first.md), removing the need for constant server requests and eliminating most loading spinners.
- **Easier to Implement**: By replicating all necessary data to the client once, you avoid implementing numerous backend endpoints for each user interaction.
- **Scalability**: Local data queries remove processing load from your servers and reduce bandwidth usage by handling queries on the client side.
## Why Using Plain IndexedDB is a Problem
Despite the advantages, directly working with IndexedDB has several drawbacks:
- **Callback-Based**: IndexedDB was originally designed around a callback-based API, which can be unwieldy compared to modern Promise or RxJS-based flows.
- **Difficult to Implement**: IndexedDB is often described as a "low-level" API. It's more suitable for library authors rather than application developers who simply need a robust local store.
- **Rudimentary Query API**: Complex or dynamic queries are cumbersome with IndexedDB's basic get/put approach and limited indexes.
- **TypeScript Support**: Maintaining strong TypeScript types for all document structures is not straightforward with IndexedDB's untyped object stores.
- **No Observable API**: IndexedDB cannot directly emit live data changes. With RxDB, you can subscribe to changes on a collection or even a single document field.
- **Cross-Tab Synchronization**: Handling concurrent data changes across multiple browser tabs is difficult in IndexedDB. RxDB has built-in multi-tab support that keeps all tabs in sync.
- **Advanced Features Missing**: IndexedDB lacks built-in support for [encryption](../encryption.md), compression, or other advanced data management features.
- **Browser-Only**: IndexedDB works in the browser but not in environments like [React Native](../react-native-database.md) or [Electron](../electron-database.md). RxDB offers storage adapters to seamlessly reuse the same code on different platforms.
## Set Up RxDB in Angular
### Installing RxDB
You can [install RxDB](../install.md) into your Angular application via npm:
```bash
npm install rxdb --save
```
### Patch Change Detection with zone.js
RxDB creates RxJS observables outside of Angular's zone, meaning Angular won't automatically trigger change detection when new data arrives. You must patch RxJS with zone.js:
```ts
//> app.component.ts
/**
* IMPORTANT: RxDB creates rxjs observables outside of Angular's zone
* So you have to import the rxjs patch to ensure change detection works correctly.
* @link https://www.bennadel.com/blog/
* 3448-binding-rxjs-observable-sources-
* outside-of-the-ngzone-in-angular-6-0-2.htm
*/
import 'zone.js/plugins/zone-patch-rxjs';
```
### Create a Database and Collections
RxDB supports multiple storage options. The free and simple approach is using the [localstorage-based](../rx-storage-localstorage.md) storage. For higher performance, there's a premium plain [IndexedDB storage](../rx-storage-indexeddb.md).
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
// Define your schema
const heroSchema = {
title: 'hero schema',
version: 0,
description: 'Describes a hero in your app',
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
power: {
type: 'string'
}
},
required: ['id', 'name']
};
```
### Localstorage
```ts
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
export async function initDB() {
// Create a database
const db = await createRxDatabase({
name: 'heroesdb', // the name of the database
storage: getRxStorageLocalstorage()
});
// Add collections
await db.addCollections({
heroes: {
schema: heroSchema
}
});
return db;
}
```
### IndexedDB
```ts
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
export async function initDB() {
// Create a database
const db = await createRxDatabase({
name: 'heroesdb', // the name of the database
storage: getRxStorageIndexedDB()
});
// Add collections
await db.addCollections({
heroes: {
schema: heroSchema
}
});
return db;
}
```
It's recommended to encapsulate database creation logic in an Angular service, such as in a DatabaseService. A full example is available in [RxDB's Angular example](https://github.com/pubkey/rxdb/blob/master/examples/angular/src/app/services/database.service.ts).
### CRUD Operations
Once your database is initialized, you can perform all CRUD operations:
```ts
// insert
await db.heroes.insert({ name: 'Iron Man', power: 'Genius-level intellect' });
// bulk insert
await db.heroes.bulkInsert([
{ name: 'Thor', power: 'God of Thunder' },
{ name: 'Hulk', power: 'Superhuman Strength' }
]);
// find and findOne
const heroes = await db.heroes.find().exec();
const ironMan = await db.heroes.findOne({ selector: { name: 'Iron Man' } }).exec();
// update
const doc = await db.heroes.findOne({ selector: { name: 'Hulk' } }).exec();
await doc.update({ $set: { power: 'Unlimited Strength' } });
// delete
const doc = await db.heroes.findOne({ selector: { name: 'Thor' } }).exec();
await doc.remove();
```
## Reactive Queries and Live Updates
A key benefit of RxDB is reactivity. You can subscribe to changes and have your UI automatically reflect updates in [real time](./realtime-database.md) even across browser tabs.
### With RxJS Observables and Async Pipes
In Angular, you can display this data with the `AsyncPipe`:
```ts
constructor(private dbService: DatabaseService) {
this.heroes$ = this.dbService.db.heroes.find({
selector: {},
sort: [{ name: 'asc' }]
}).$;
}
```
```html
{{ hero.name }}
```
### With Angular Signals
Angular Signals are a newer approach for reactivity. RxDB supports them via a [custom reactivity](../reactivity.md) factory. You can convert RxJS Observables to Signals using Angular's `toSignal`:
```ts
import { RxReactivityFactory } from 'rxdb/plugins/core';
import { Signal, untracked, Injector } from '@angular/core';
import { toSignal } from '@angular/core/rxjs-interop';
export function createReactivityFactory(
injector: Injector
): RxReactivityFactory> {
return {
fromObservable(observable$, initialValue) {
return untracked(() =>
toSignal(observable$, {
initialValue,
injector,
rejectErrors: true
})
);
}
};
}
```
Pass this factory when creating your [RxDatabase](../rx-database.md):
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { inject, Injector } from '@angular/core';
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: createReactivityFactory(inject(Injector))
});
```
Use the double-dollar sign (`$$`) to get a `Signal` instead of an `Observable`:
```ts
const heroesSignal = database.heroes.find().$$;
```
```html
{{ hero.name }}
```
## Angular IndexedDB Example with RxDB
A comprehensive example of RxDB in an Angular application is available in the [RxDB GitHub repository](https://github.com/pubkey/rxdb/tree/master/examples/angular). It demonstrates [database](./angular-database.md) creation, queries, and Angular integration using best practices.
## Advanced RxDB Features
Beyond simple CRUD and local data storage, RxDB supports:
- **[Replication](../replication.md)**: Sync your local data with a remote database. Learn more at [RxDB Replication](https://rxdb.info/replication.html).
- **Data Migration on Schema Changes**: RxDB supports automatic or manual schema migrations to manage backward-compatibility and evolve your data structure. See [RxDB Migration](https://rxdb.info/migration-schema.html).
- **Encryption**: Easily encrypt sensitive data at rest. See [RxDB Encryption](https://rxdb.info/encryption.html).
- **Compression**: Reduce storage and bandwidth usage using [key compression](../key-compression.md). Learn more at [RxDB Key Compression](https://rxdb.info/key-compression.html).
## Limitations of IndexedDB
While IndexedDB works well for many use cases, it does have a few constraints:
- **Potentially Slow**: While adequate for most use cases, IndexedDB performance can degrade for very large datasets. More details at RxDB [Slow IndexedDB](../slow-indexeddb.md).
- **Storage Limits**: Browsers may cap the amount of data you can store in IndexedDB. For more info, see [Local Storage Limits of IndexedDB](./indexeddb-max-storage-limit.md).
## Alternatives to IndexedDB
Depending on your needs, you might explore:
- **Origin Private File System (OPFS)**: A newer browser storage mechanism that can offer better performance. RxDB supports [OPFS storage](../rx-storage-opfs.md).
- **SQLite**: When building a mobile or hybrid app (e.g., with [Capacitor](../capacitor-database.md) or [Ionic](./ionic-database.md)), you can use SQLite locally. See [RxDB with SQLite](../rx-storage-sqlite.md).
## Performance comparison with other browser storages
Here is a [performance overview](../rx-storage-performance.md) of the various browser based storage implementation of RxDB:
## FAQ
You should avoid interacting with the raw IndexedDB callback API inside Angular components. Instead, you wrap IndexedDB in a reactive abstraction like **[RxDB](https://rxdb.info)**. RxDB seamlessly translates IndexedDB data changes into standard RxJS Observables. By configuring a custom reactivity factory with `toSignal` from `@angular/core/rxjs-interop`, you can extract pure Angular Signals straight from local IndexedDB queries, guaranteeing extremely fast and fully reactive UI renders.
## Follow Up
Continue your deep dive into RxDB with official quickstart guides and star the repository on GitHub to stay updated.
- **RxDB Quickstart**: Get started quickly with the [RxDB Quickstart](../quickstart.md).
- **RxDB GitHub**: Explore the source, open issues, and star β the project at [RxDB GitHub Repo](https://github.com/pubkey/rxdb).
By combining IndexedDB's local storage with RxDB's powerful features, you can build performant, robust, and offline-capable Angular applications. RxDB takes care of the lower-level complexities, letting you focus on delivering a great user experience-online or off.
---
## Benefits of RxDB & Browser Databases
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB: The benefits of Browser Databases
In the world of web development, efficient data management is a cornerstone of building successful and performant applications. The ability to store data directly in the browser brings numerous advantages, such as caching, offline accessibility, simplified replication of database state, and real-time application development. In this article, we will explore [RxDB](https://rxdb.info/), a powerful browser JavaScript database, and understand why it is an excellent choice for implementing a browser database solution.
## Why you might want to store data in the browser
There are compelling reasons to consider storing data in the browser:
### Use the database for caching
By leveraging a browser database, you can harness the power of caching. Storing frequently accessed data locally enables you to reduce server requests and greatly improve application performance. Caching provides a faster and smoother user experience, enhancing overall user satisfaction.
### Data is offline accessible
Storing data in the browser allows for offline accessibility. Regardless of an active internet connection, users can access and interact with the application, ensuring uninterrupted productivity and user engagement.
### Easier implementation of replicating database state
Browser databases simplify the replication of database state across multiple devices or instances of the application. Compared to complex REST routes, replicating data becomes easier and more streamlined. This capability enables the development of real-time and collaborative applications, where changes are seamlessly synchronized among users.
### Building real-time applications is easier with local data
With a local browser database, building real-time applications becomes more straightforward. The availability of local data allows for reactive data flows and dynamic user interfaces that instantly reflect changes in the underlying data. Real-time features can be seamlessly implemented, providing a rich and interactive user experience.
### Browser databases can scale better
Browser databases distribute the query workload to users' devices, allowing queries to run locally instead of relying solely on server resources. This decentralized approach improves scalability by reducing the burden on the server, resulting in a more efficient and responsive application.
### Running queries locally has low latency
Browser databases offer the advantage of running queries locally, resulting in low latency. Eliminating the need for server round-trips significantly improves query performance, ensuring faster data retrieval and a more responsive application.
### Faster initial application start time
Storing data in the browser reduces the initial application start time. Instead of waiting for data to be fetched from the server, the application can leverage the [local database](./local-database.md), resulting in faster initialization and improved user satisfaction right from the start.
### Easier integration with JavaScript frameworks
Browser databases, including [RxDB](https://rxdb.info/), seamlessly integrate with popular JavaScript frameworks such as [Angular](./angular-database.md), [React.js](./react-database.md), [Vue.js](./vue-database.md), and Svelte. This integration allows developers to leverage the power of a database while working within the familiar environment of their preferred framework, enhancing productivity and ease of development.
### Store local data with encryption
Security is a crucial aspect of data storage, especially when handling sensitive information. Browser databases, like RxDB, offer the capability to store local data with [encryption](../encryption.md), ensuring the confidentiality and protection of sensitive user data.
### Using a local database for state management
Utilizing a local browser database for state management eliminates the need for traditional state management libraries like Redux or NgRx. This approach simplifies the application's architecture by leveraging the database's capabilities to handle state-related operations efficiently.
### Data is portable and always accessible by the user
When data is stored in the browser, it becomes portable and always accessible by the user. This ensures that users have control and ownership of their data, enhancing data privacy and accessibility.
## Why SQL databases like SQLite are not a good fit for the browser
While SQL databases, such as [SQLite](../rx-storage-sqlite.md), excel in server-side scenarios, they are not always the optimal choice for browser-based applications. Here are some reasons why SQL databases may not be the best fit for the browser:
### Push/Pull based vs. reactive
SQL databases typically rely on a push/pull mechanism, where the server pushes updates to the client or the client pulls data from the server. This approach is not inherently reactive and requires additional effort to implement real-time data updates. In contrast, browser databases like [RxDB](https://rxdb.info/) provide built-in reactive mechanisms, allowing the application to react to data changes seamlessly.
### Build size of server-side databases
Server-side databases, designed to handle large-scale applications, often have significant build sizes that are unsuitable for browser applications. In contrast, browser databases are specifically optimized for browser environments and leverage browser APIs like [IndexedDB](../rx-storage-indexeddb.md), [OPFS](../rx-storage-opfs.md), and [Webworker](../rx-storage-worker.md), resulting in smaller build sizes.
### Initialization time and performance
The initialization time and performance of server-side databases can be suboptimal in browser applications. Browser databases, on the other hand, are designed to provide fast initialization and efficient performance within the browser environment, ensuring a smooth user experience.
## Why RxDB is a good fit for the browser
RxDB stands out as an excellent choice for implementing a browser database solution. Here's why RxDB is a perfect fit for browser applications:
### Observable Queries (rxjs) to automatically update the UI on changes
RxDB provides Observable Queries, powered by RxJS, enabling automatic UI updates when data changes occur. This reactive approach eliminates the need for manual data synchronization and ensures a real-time and responsive user interface.
```typescript
const query = myCollection.find({
selector: {
age: {
$gt: 21
}
}
});
const querySub = query.$.subscribe(results => {
console.log('got results: ' + results.length);
});
```
### NoSQL [JSON](./json-database.md) documents are a better fit for UIs
RxDB utilizes NoSQL [JSON documents](./json-database.md), which align naturally with UI development in JavaScript. JavaScript's native handling of JSON objects makes working with NoSQL documents more intuitive, simplifying UI-related operations.
### NoSQL has better TypeScript support compared to SQL
TypeScript is widely used in modern JavaScript development. [NoSQL databases](./in-memory-nosql-database.md), including RxDB, offer excellent TypeScript support, making it easier to build type-safe applications and leverage the benefits of static typing.
### Observable document fields
RxDB allows observing individual document fields, providing granular [reactivity](../reactivity.md). This feature enables efficient tracking of specific data changes and fine-grained UI updates, optimizing performance and responsiveness.
### Made in JavaScript, optimized for JavaScript applications
RxDB is built entirely in JavaScript, optimized for JavaScript applications. This ensures seamless integration with JavaScript codebases and maximizes performance within the browser environment.
### Optimized observed queries with the EventReduce Algorithm
RxDB employs the EventReduce Algorithm to optimize observed queries. This algorithm intelligently reduces unnecessary data transmissions, resulting in efficient query execution and improved performance.
### Built-in multi-tab support
RxDB natively supports multi-tab applications, allowing data synchronization and replication across different tabs or instances of the same application. This feature ensures consistent data across the application and enhances collaboration and real-time experiences.
### Handling of schema changes
RxDB excels in handling schema changes, even when data is stored on multiple client devices. It provides mechanisms to handle schema migrations seamlessly, ensuring data integrity and compatibility as the application evolves.
### Storing documents compressed
To optimize [storage](./browser-storage.md) space, RxDB allows the [compression](../key-compression.md) of documents. Storing compressed documents reduces storage requirements and improves overall performance, especially in scenarios with large data volumes.
### Flexible storage layer for various platforms
RxDB offers a flexible storage layer, enabling code reuse across different platforms, including [Electron.js](../electron-database.md), React Native, hybrid apps (e.g., Capacitor.js), and web browsers. This flexibility streamlines development efforts and ensures consistent data management across multiple platforms.
### Replication Algorithm for compatibility with any backend
RxDB incorporates a [Replication Algorithm](../replication.md) that is open-source and can be made compatible with various backend systems. This compatibility allows seamless data synchronization with different backend architectures, such as own servers, [Firebase](../replication-firestore.md), [CouchDB](../replication-couchdb.md), [NATS](../replication-nats.md) or [WebSocket](../replication-websocket.md).
## Follow Up
To explore more about RxDB and leverage its capabilities for browser database development, check out the following resources:
- [RxDB GitHub Repository](https://github.com/pubkey/rxdb): Visit the official GitHub repository of RxDB to access the source code, documentation, and community support.
- [RxDB Quickstart](../quickstart.md): Get started quickly with RxDB by following the provided quickstart guide, which provides step-by-step instructions for setting up and using RxDB in your projects.
[RxDB](https://rxdb.info/) empowers developers to unlock the power of browser databases, enabling efficient data management, real-time applications, and enhanced user experiences. By leveraging RxDB's features and benefits, you can take your browser-based applications to the next level of performance, scalability, and responsiveness.
---
## Browser Storage - RxDB as a Database for Browsers
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# Browser Storage - RxDB as a Database for Browsers
**Storing Data in the Browser**
When it comes to building web applications, one essential aspect is the storage of data. Two common methods of storing data directly within the user's web browser are LocalStorage and [IndexedDB](../rx-storage-indexeddb.md). These browser-based storage options serve various purposes and cater to different needs in web development.
### LocalStorage
[LocalStorage](./localstorage.md) is a straightforward way to store small amounts of data in the user's web browser. It operates on a simple key-value basis and is relatively easy to use. While it has limitations, it is suitable for basic data storage requirements.
### IndexedDB
IndexedDB, on the other hand, offers a more robust and structured approach to browser-based data storage. It can handle larger datasets and complex queries, making it a valuable choice for more advanced web applications.
## Why Store Data in the Browser
Now that we've explored the methods of storing data in the browser, let's delve into why this is a beneficial strategy for web developers:
1. **Caching**:
Storing data in the browser allows you to cache frequently used information. This means that your web application can access essential data more quickly because it doesn't need to repeatedly fetch it from a server. This results in a smoother and more responsive user experience.
2. **Offline Access**:
One significant advantage of browser storage is that data becomes portable and remains accessible even when the user is offline. This feature ensures that users can continue to use your application, view their saved information, and make changes, irrespective of their internet connection status.
3. **Faster Real-time Applications**:
For real-time applications, having data stored locally in the browser significantly enhances performance. Local data allows your application to respond faster to user interactions, creating a more seamless and responsive user interface.
4. **Low-Latency Queries**:
When you run queries locally within the browser, you minimize the latency associated with network requests. This results in near-instant access to data, which is particularly crucial for applications that require rapid data retrieval.
5. **Faster Initial Application Start Time**:
By preloading essential data into browser storage, you can reduce the initial load time of your web application. Users can start using your application more swiftly, which is essential for making a positive first impression.
6. **Store Local Data with Encryption**:
For applications that deal with sensitive data, browser storage allows you to implement [encryption](../encryption.md) to secure the stored information. This ensures that even if data is stored on the user's device, it remains confidential and protected.
In summary, storing data in the browser offers several advantages, including improved performance, offline access, and enhanced user experiences. LocalStorage and IndexedDB are two valuable tools that developers can utilize to leverage these benefits and create web applications that are more responsive and user-friendly.
## Browser Storage Limitations
While browser storage, such as LocalStorage and IndexedDB, offers many advantages, it's important to be aware of its limitations:
- **Slower Performance Compared to Native Databases**: Browser-based storage solutions can't match the [performance](../rx-storage-performance.md) of native server-side databases. They may experience slower data retrieval and processing, especially for large datasets or complex operations.
- **Storage Space Limitations**: Browsers [impose restrictions on the amount of data that can be stored locally](./indexeddb-max-storage-limit.md). This limitation can be problematic for applications with extensive data storage requirements, potentially necessitating creative solutions to manage data effectively.
## Why SQL Databases Like SQLite Aren't a Good Fit for the Browser
SQL databases like [SQLite](../rx-storage-sqlite.md), while powerful in server environments, may not be the best choice for browser-based applications due to various reasons:
### Push/Pull Based vs. Reactive
SQL databases often use a push/pull model for data synchronization. This approach is less reactive and may not align well with the real-time nature of web applications, where immediate updates to the user interface are crucial.
### Build Size of Server-Side Databases
Server-side databases like SQLite have a significant build size, which can increase the initial load time of web applications. This can result in a suboptimal user experience, particularly for users with slower internet connections.
### Initialization Time and Performance
SQL databases are optimized for server environments, and their initialization processes and performance characteristics may not align with the needs of web applications. They might not offer the swift performance required for seamless user interactions.
## Why RxDB Is a Good Fit as Browser Storage
RxDB is an excellent choice for browser-based storage due to its numerous features and advantages:
### Flexible Storage Layer for Various Platforms
RxDB offers a flexible storage layer that can seamlessly integrate with different platforms, making it versatile and adaptable to various application needs.
### NoSQL JSON Documents Are a Better Fit for UIs
NoSQL [JSON documents](./json-database.md), used by [RxDB](https://rxdb.info/), are well-suited for user interfaces. They provide a natural and efficient way to structure and display data in web applications.
### NoSQL Has Better TypeScript Support Compared to SQL
RxDB boasts robust TypeScript support, which is beneficial for developers who prefer type safety and code predictability in their projects.
### Observable Document Fields
RxDB enables developers to observe individual document fields, offering fine-grained control over data tracking and updates.
### Made in JavaScript, Optimized for JavaScript Applications
Being built in JavaScript and optimized for JavaScript applications, RxDB seamlessly integrates into web development stacks, minimizing compatibility issues.
### Observable Queries (rxjs) to Automatically Update the UI on Changes
RxDB's support for Observable Queries allows the user interface to update automatically in real-time when data changes. This [reactivity](../reactivity.md) enhances the user experience and simplifies UI development.
```typescript
const query = myCollection.find({
selector: {
age: {
$gt: 21
}
}
});
const querySub = query.$.subscribe(results => {
console.log('got results: ' + results.length);
});
```
### Optimized Observed Queries with the EventReduce Algorithm
RxDB's [EventReduce Algorithm](https://github.com/pubkey/event-reduce) ensures efficient data handling and rendering, improving overall performance and responsiveness.
### Handling of Schema Changes
RxDB provides built-in support for [handling schema changes](../migration-schema.md), simplifying database management when updates are required.
### Built-In Multi-Tab Support
For applications requiring multi-tab support, RxDB natively handles data consistency across different browser tabs, streamlining data synchronization.
### Storing Documents Compressed
Efficient data storage is achieved through [document compression](../key-compression.md), reducing storage space requirements and enhancing overall performance.
### Replication Algorithm for Compatibility with Any Backend
RxDB's [Replication Algorithm](../replication.md) facilitates compatibility with various backend systems, ensuring seamless data synchronization between the browser and server.
## Summary
In conclusion, RxDB is a powerful and feature-rich solution for browser-based storage. Its adaptability, real-time capabilities, TypeScript support, and optimization for JavaScript applications make it an ideal choice for modern web development projects, addressing the limitations of traditional SQL databases in the browser. Developers can harness RxDB to create efficient, responsive, and user-friendly web applications that leverage the full potential of browser storage.
## FAQ
Yes, in modern browser extension development (Manifest V3), the `chrome.storage.local` methods, including `.get()`, `.set()`, and `.remove()`, natively return JavaScript Promises. This allows developers to use clean `await` syntax. This is a significant improvement over standard `localStorage`, which is fully synchronous, and over legacy Chrome APIs that only supported callback functions.
No. Data stored in standard browser native technologies like [LocalStorage](./localstorage.md), [IndexedDB](../rx-storage-indexeddb.md), or Cookies is stored in plain text on the user's hard drive. It can be easily accessed by anyone with physical access to the device or by malicious scripts executing under the same origin (XSS attacks). To store sensitive user information, you must implement [Encryption](../encryption.md) at the application layer before writing data to the browser storage APIs. See [IndexedDB encryption](./indexeddb/indexeddb-encryption.md) for how to do this on top of IndexedDB.
Browser storage has evolved from merely storing tiny session tokens in Cookies into the backbone of **[Local-First](../offline-first.md)** architecture. Modern web applications utilize powerful APIs like [IndexedDB](../rx-storage-indexeddb.md) or [OPFS](../rx-storage-opfs.md) to store gigabytes of application state directly on the client. This allows applications to offer zero-latency UI interactions, fully function offline, and systematically synchronize changes to a backend only when network conditions permit.
While traditional enterprise databases like PostgreSQL or SQL Server run on backend servers, you can achieve enterprise-grade JSON storage natively in the browser using robust client-side databases. **[RxDB](../rx-database.md)** is specifically engineered to provide a fully reactive, NoSQL [document-oriented JSON database](./json-database.md) directly in the browser, capable of interacting seamlessly with various storage endpoints like [IndexedDB](../rx-storage-indexeddb.md) or even bridging to [SQLite via WebAssembly](../rx-storage-sqlite.md).
## Follow Up
To explore more about RxDB and leverage its capabilities for browser storage, check out the following resources:
- [RxDB GitHub Repository](https://github.com/pubkey/rxdb): Visit the official GitHub repository of RxDB to access the source code, documentation, and community support.
- [RxDB Quickstart](../quickstart.md): Get started quickly with RxDB by following the provided quickstart guide, which provides step-by-step instructions for setting up and using RxDB in your projects.
---
## Empower Web Apps with Reactive RxDB Data-base
# RxDB as a data base: Empowering Web Applications with Reactive Data Handling
In the world of web applications, efficient data management plays a crucial role in delivering a seamless user experience. As mobile applications continue to dominate the digital landscape, the importance of robust data bases becomes evident. In this article, we will explore RxDB as a powerful data base solution for web applications. We will delve into its features, advantages, and advanced techniques, highlighting its ability to handle reactive data and enable an [offline-first](../offline-first.md) approach.
## Overview of Web Applications that can benefit from RxDB
Before diving into the specifics of RxDB, let's take a moment to understand the scope of web applications that can leverage its capabilities. Any web application that requires real-time data updates, offline functionality, and synchronization between clients and servers can greatly benefit from RxDB. Whether it's a collaborative document editing tool, a task management app, or a chat application, RxDB offers a robust foundation for building these types of applications.
## Importance of data bases in Mobile Applications
Mobile applications have become an integral part of our lives, providing us with instant access to information and services. Behind the scenes, data bases play a pivotal role in storing and managing the data that powers these applications. data bases enable efficient data retrieval, updates, and synchronization, ensuring a smooth user experience even in challenging network conditions.
## Introducing RxDB as a data base Solution
RxDB, short for Reactive data base, is a client-side data base solution designed specifically for web and mobile applications. Built on the principles of reactive programming, RxDB brings the power of observables and event-driven architecture to data management. With RxDB, developers can create applications that are responsive, offline-ready, and capable of seamless data synchronization between clients and servers.
## Getting Started with RxDB
### What is RxDB?
RxDB is an open-source JavaScript data base that leverages reactive programming and provides a seamless API for handling data. It is built on top of existing popular data base technologies, such as [IndexedDB](../rx-storage-indexeddb.md), and adds a layer of reactive features to enable real-time data updates and synchronization.
### Reactive Data Handling
One of the standout features of RxDB is its reactive data handling. It utilizes observables to provide a stream of data that automatically updates whenever a change occurs. This reactive approach allows developers to build applications that respond instantly to data changes, ensuring a highly interactive and real-time user experience.
### Offline-First Approach
RxDB embraces an offline-first approach, enabling applications to work seamlessly even when there is no internet connectivity. It achieves this by caching data locally on the client-side and synchronizing it with the server when the connection is available. This ensures that users can continue working with the application and have their data automatically synchronized when they come back online.
### Data Replication
RxDB simplifies the process of data [replication](../replication.md) between clients and servers. It provides replication plugins that handle the synchronization of data in real-time. These plugins allow applications to keep data consistent across multiple clients, enabling collaborative features and ensuring that each client has the most up-to-date information.
### Observable Queries
RxDB introduces the concept of observable queries, which are powerful tools for efficiently querying data. With observable queries, developers can subscribe to specific data queries and receive automatic updates whenever the underlying data changes. This eliminates the need for manual polling and ensures that applications always have access to the latest data.
### Multi-Tab support
RxDB offers multi-tab support, allowing applications to function seamlessly across multiple [browser](./browser-database.md) tabs. This feature ensures that data changes in one tab are immediately reflected in all other open tabs, enabling a consistent user experience across different browser windows.
### RxDB vs. Other data base Options
When considering data base options for web applications, developers often encounter choices like IndexedDB, [OPFS](../rx-storage-opfs.md), and Memory-based solutions. RxDB, while built on top of IndexedDB, stands out due to its reactive data handling capabilities and advanced synchronization features. Compared to other options, RxDB offers a more streamlined and powerful approach to managing data in web applications.
### Different RxStorage layers for RxDB
RxDB provides various [storage layers](../rx-storage.md), known as RxStorage, that serve as interfaces to different underlying [storage](./browser-storage.md) technologies. These layers include:
- [LocalStorage RxStorage](../rx-storage-localstorage.md): Built on top of the browsers [localStorage API](./localstorage.md).
- [IndexedDB RxStorage](../rx-storage-indexeddb.md): This layer directly utilizes IndexedDB as its backend, providing a robust and widely supported storage option.
- [OPFS RxStorage](../rx-storage-opfs.md): OPFS (Operational Transformation File System) is a file system-like storage layer that allows for efficient [conflict resolution](../transactions-conflicts-revisions.md) and real-time collaboration.
- [Memory RxStorage](../rx-storage-memory.md): Primarily used for testing and development, this storage layer keeps data in memory without persisting it to disk.
Each RxStorage layer has its strengths and is suited for different scenarios, enabling developers to choose the most appropriate option for their specific use case.
## Synchronizing Data with RxDB between Clients and Servers
### Offline-First Approach
As mentioned earlier, RxDB adopts an offline-first approach, allowing applications to function seamlessly in disconnected environments. By caching data locally, applications can continue to operate and make updates even without an internet connection. Once the connection is restored, RxDB's replication plugins take care of synchronizing the data with the server, ensuring consistency across all clients.
### RxDB Replication Plugins
RxDB provides a range of replication plugins that simplify the process of synchronizing data between clients and servers. These plugins enable real-time replication using various protocols, such as WebSocket or HTTP, and handle conflict resolution strategies to ensure data integrity. By leveraging these replication plugins, developers can easily implement robust and scalable synchronization capabilities in their applications.
### Advanced RxDB Features and Techniques
Indexing and Performance Optimization
To achieve optimal performance, RxDB offers indexing capabilities. Indexing allows for efficient data retrieval and faster query execution. By strategically defining indexes on frequently accessed fields, developers can significantly enhance the overall performance of their RxDB-powered applications.
### Encryption of Local Data
In scenarios where data security is paramount, RxDB provides options for encrypting local data. By encrypting the data base contents, developers can ensure that sensitive information remains secure even if the underlying storage is compromised. RxDB integrates seamlessly with [encryption](../encryption.md) libraries, making it easy to implement end-to-end encryption in applications.
### Change Streams and Event Handling
RxDB offers change streams and event handling mechanisms, enabling developers to react to data changes in real-time. With change streams, applications can listen to specific collections or documents and trigger custom logic whenever a change occurs. This capability opens up possibilities for building real-time collaboration features, notifications, or other reactive behaviors.
### JSON Key Compression
In scenarios where storage size is a concern, RxDB provides JSON [key compression](../key-compression.md). By applying compression techniques to JSON keys, developers can significantly reduce the storage footprint of their data bases. This feature is particularly beneficial for applications dealing with large datasets or [limited storage capacities](./indexeddb-max-storage-limit.md).
## Conclusion
RxDB provides an exceptional data base solution for web and mobile applications, empowering developers to create reactive, offline-ready, and synchronized applications. With its reactive data handling, offline-first approach, and replication plugins, RxDB simplifies the challenges of building real-time applications with data synchronization requirements. By embracing advanced features like indexing, encryption, change streams, and JSON key compression, developers can optimize performance, enhance security, and reduce storage requirements. As web and [mobile applications](./mobile-database.md) continue to evolve, RxDB proves to be a reliable and powerful
---
## Electron SQLite Database - Reactive Local Data with RxDB
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# Electron SQLite - Building reactive desktop apps with RxDB and SQLite
[Electron](https://www.electronjs.org/) apps run on the user's device, so storing data locally is the natural way to build them. [SQLite](https://www.sqlite.org/) is the most proven embedded database and a great fit for Electron because it runs inside the app process, needs no server and stores everything in a single file.
But plain SQLite alone is not enough for a modern desktop application. It has no way to observe queries, no sync to a backend, no encryption and it can only run in the Electron **main** process, not in the **renderer** where your UI lives. This article shows how to combine SQLite with [RxDB](https://rxdb.info/) to get the reliability of SQLite together with reactive queries, [replication](../replication.md) and [encryption](../encryption.md), while keeping all heavy database work out of the UI process.
## Why SQLite is a good fit for Electron
- **No server process**: SQLite is embedded. Your Electron app opens a database file directly, there is no port to expose and no binary to manage. Shipping a server database like PostgreSQL or MySQL inside an Electron bundle is not practical, as explained in the [Electron database comparison](../electron-database.md).
- **Single file storage**: All data lives in one `.sqlite` file inside the app's user-data folder. Backups and debugging are simple.
- **Built into Node.js**: Since Node.js version 22, the [node:sqlite](https://nodejs.org/api/sqlite.html) module ships with Node itself. Recent Electron versions include a Node.js runtime with this module, so you can use SQLite **without native module rebuilds**. Packages like `sqlite3` or `better-sqlite3` require [@electron/rebuild](https://github.com/electron/rebuild) to compile against the Electron headers on every Electron upgrade. With `node:sqlite` this whole step disappears.
- **Proven at scale**: SQLite is the most deployed database in the world and Chromium itself uses it internally.
## The two Electron processes and where the database belongs
An Electron app consists of two kinds of JavaScript runtimes:
- The **main process**: a Node.js process without a UI. It has full filesystem access and can load `node:sqlite`.
- One or more **renderer processes**: Chromium browser windows that render your UI. They have no direct SQLite access.
SQLite must run in the main process. Your UI code in the renderer then needs a way to read and write data across the process boundary via [IPC](https://www.electronjs.org/de/docs/latest/api/ipc-renderer). Doing this by hand means writing an IPC handler for every query, serializing results, and inventing your own change-notification system so that windows update when data changes. RxDB ships this wiring as a plugin, as shown below.
## The problem with using SQLite directly
A hand-rolled setup looks like this: you open the database in the main process and answer queries over IPC.
```ts
// main process, without RxDB
import { DatabaseSync } from 'node:sqlite';
import { ipcMain } from 'electron';
const db = new DatabaseSync('/path/to/users.db');
db.exec('CREATE TABLE IF NOT EXISTS users(id TEXT PRIMARY KEY, name TEXT)');
ipcMain.handle('db-query', (event, sql, params) => {
return db.prepare(sql).all(...params);
});
```
```ts
// renderer process, without RxDB
const rows = await ipcRenderer.invoke(
'db-query',
'SELECT * FROM users WHERE name = ?',
['alice']
);
```
This works for a prototype, but several problems show up as the app grows:
- **No reactivity**: When one part of the app writes data, other components and other windows keep showing stale state. You have to build your own event system on top.
- **Multiple windows**: Each `BrowserWindow` is its own renderer process. Keeping their UI state consistent requires broadcasting every change to every window.
- **No sync**: Desktop users expect their data on other devices too. SQL gives you no replication protocol, no conflict handling and no offline queue.
- **Manual schema and migration handling**: Table definitions, indexes and migrations are all your responsibility.
- **No type safety**: Query results are untyped rows, so TypeScript cannot help you.
## RxDB on top of SQLite
[RxDB](https://rxdb.info/) is a local-first, NoSQL database for JavaScript applications. It stores documents in [collections](../rx-collection.md), validates them against a [JSON schema](../rx-schema.md) and exposes queries as RxJS observables. Through its swappable [storage layer](../rx-storage.md) it can persist data in the [SQLite RxStorage](../rx-storage-sqlite.md), which means you keep SQLite as the storage engine and gain:
- **Reactive queries**: Subscribe to a query and get a new result set each time the underlying data changes, across components and across windows.
- **Realtime replication**: The RxDB [Sync Engine](../replication.md) replicates with [CouchDB](../replication-couchdb.md), [Firestore](../replication-firestore.md), [GraphQL](../replication-graphql.md) or any [custom HTTP endpoint](../replication-http.md), including offline-first conflict handling.
- **Encryption**: Store sensitive fields encrypted on disk with the [encryption plugins](../encryption.md).
- **Compression**: Reduce storage size with [key compression](../key-compression.md).
- **TypeScript support**: Typed documents, typed queries and typed results out of the box.
## Why you should use the RxDB SQLite storage instead of SQLite by itself
With the RxDB SQLite storage you keep everything that makes SQLite attractive. The data still lives in a normal SQLite file on disk and the [Premium SQLite RxStorage](../rx-storage-sqlite.md) runs queries inside SQLite itself, using its JSON functions and real indexes. What changes is the layer your application code talks to:
| Concern | SQLite by itself | RxDB SQLite storage |
|---|---|---|
| Observe queries | Not possible | RxJS observables on any query |
| Access from renderer | Hand-written IPC handlers | [Electron plugin](../electron.md) |
| Multiple windows | Manual change broadcasting | Built in |
| Backend sync | Not included | [Sync Engine](../replication.md) |
| Conflict handling | Not included | [Conflict handlers](../transactions-conflicts-revisions.md) |
| Schema and migrations | Hand-written SQL | [JSON schema](../rx-schema.md) and [migration plugin](../migration-schema.md) |
| Encryption on disk | Requires paid SQLite extensions | [Encryption plugin](../encryption.md) |
| Query results | Untyped rows | Typed documents |
Each row of the left column is code you would write and maintain yourself. The IPC layer alone tends to grow into a custom protocol with one handler per use case, and the change-notification logic that keeps several windows in sync is hard to get right. RxDB replaces all of it with a documented, tested API while SQLite keeps doing what it does best: storing bytes reliably in one file.
## Setup: RxDB with SQLite in Electron
The recommended architecture runs the SQLite storage in the main process and connects each renderer window to it with the [RxDB Electron plugin](../electron.md). The renderer creates a normal `RxDatabase`, while all SQLite operations happen in the main process and never block the UI.
RxDB ships a free **trial version** of the SQLite storage that you can use to evaluate the setup. For production apps, the full [SQLite RxStorage](../rx-storage-sqlite.md) is part of the [RxDB Premium π](/premium/) plugins.
### 1. Install RxDB
```bash
npm install rxdb rxjs
```
### 2. Open the SQLite storage in the main process
Create the SQLite storage with the `node:sqlite` module and expose it to the renderer processes with `exposeIpcMainRxStorage`:
```ts
// main.js
import { app, ipcMain } from 'electron';
import { exposeIpcMainRxStorage } from 'rxdb/plugins/electron';
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsNodeNative
} from 'rxdb/plugins/storage-sqlite';
import { DatabaseSync } from 'node:sqlite';
app.on('ready', () => {
exposeIpcMainRxStorage({
key: 'main-storage',
storage: getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
}),
ipcMain
});
/* ... open your BrowserWindow ... */
});
```
The `sqliteBasics` adapter tells RxDB how to talk to the given SQLite library. Adapters for `sqlite3`, `better-sqlite3` and others exist as well, see the [SQLite RxStorage documentation](../rx-storage-sqlite.md).
### 3. Create the database in the renderer
```ts
// renderer.js
import { createRxDatabase } from 'rxdb';
import { getRxStorageIpcRenderer } from 'rxdb/plugins/electron';
import { ipcRenderer } from 'electron';
const db = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageIpcRenderer({
key: 'main-storage',
ipcRenderer
})
});
await db.addCollections({
heroes: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
color: { type: 'string' }
},
required: ['id', 'name', 'color']
}
}
});
```
:::note
`nodeIntegration` must be enabled on the `BrowserWindow` so that the renderer can use `ipcRenderer`, see the [Electron plugin docs](../electron.md).
:::
### 4. Read and write data
```ts
// insert a document
await db.heroes.insert({
id: 'sqlite-hero',
name: 'Alice',
color: 'red'
});
// query once
const redHeroes = await db.heroes.find({
selector: { color: 'red' }
}).exec();
// observe a query
db.heroes.find({
selector: { color: 'red' }
}).$.subscribe(heroes => {
// emits on every change to the result set,
// also when the write came from another window
renderHeroList(heroes);
});
```
This last snippet is the reason RxDB and SQLite work so well together in Electron. The write goes over IPC into SQLite in the main process, and every subscribed query in every window updates on its own. There is no custom IPC protocol to maintain and no stale UI state.
## Production: the Premium SQLite storage
The trial storage passes the full RxDB test suite but is limited to 500 non-deleted documents, skips indexes and runs queries in memory. For production, switch to the [Premium SQLite RxStorage](../rx-storage-sqlite.md), which uses real SQLite indexes and runs the queries inside SQLite with its JSON functions. The switch is a two-line change:
```ts
import {
getRxStorageSQLite,
getSQLiteBasicsNodeNative
} from 'rxdb-premium/plugins/storage-sqlite';
import { DatabaseSync } from 'node:sqlite';
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
});
```
Because the [storage layer](../rx-storage.md) is swappable, none of your application code changes. You can also start development with the [memory storage](../rx-storage-memory.md) for fast test runs and use SQLite only in the packaged app.
## Syncing the Electron SQLite database with a backend
Local SQLite data becomes more useful when it replicates. With the RxDB [Sync Engine](../replication.md) your Electron app pulls and pushes changes in realtime and keeps working offline. Writes land in SQLite first, the UI updates instantly and the replication catches up whenever the network allows it. Conflicts are detected and resolved with a [conflict handler](../transactions-conflicts-revisions.md) that you control.
A basic [HTTP replication](../replication-http.md) needs two endpoints on your server: one to pull document changes after a given checkpoint and one to push local change rows. On the client you wire them into `replicateRxCollection`:
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
collection: db.heroes,
replicationIdentifier: 'my-http-replication',
pull: {
async handler(checkpointOrNull, batchSize) {
const updatedAt = checkpointOrNull
? checkpointOrNull.updatedAt : 0;
const id = checkpointOrNull ? checkpointOrNull.id : '';
const response = await fetch(
'https://example.com/api/pull' +
`?updatedAt=${updatedAt}&id=${id}&limit=${batchSize}`
);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
},
push: {
async handler(changeRows) {
const response = await fetch(
'https://example.com/api/push',
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(changeRows)
}
);
// the server responds with an array of conflicts
return await response.json();
}
}
});
```
The pull handler fetches batches of changed documents from the server until the client is up to date. The push handler sends local writes and receives conflicting server states back, which RxDB then resolves with your conflict handler. For realtime updates from the server and the full server-side implementation, see the [HTTP replication tutorial](../replication-http.md).
## SQLite vs. the Node Filesystem storage
RxDB offers a second persistent storage for the Electron main process: the [Filesystem Node RxStorage](../rx-storage-filesystem-node.md). Instead of one SQLite file it stores documents as plain JSON text files in a folder via the Node.js filesystem API.
In the [performance comparison](../rx-storage-performance.md) the filesystem storage is a bit faster than the SQLite storage. Wrapping SQLite adds overhead, and every operation pays latency for moving data between the JavaScript process and the SQLite engine. The filesystem storage skips that boundary and writes JSON directly to disk.
Reasons to still pick SQLite:
- **Single file**: One portable `.sqlite` file is easier to back up, copy and inspect than a folder tree of JSON files, and many tools can open it.
- **Same storage across platforms**: If you also ship mobile apps with [Capacitor](../capacitor-database.md) or [React Native](../react-native-database.md), the SQLite storage works there too, so all your apps behave the same.
Reasons to pick the filesystem storage:
- **Speed**: Lower per-operation overhead, see the [performance measurements](../rx-storage-performance.md).
- **Simpler setup**: No SQLite library and no `sqliteBasics` adapter needed.
Using it looks like this, again combined with the Electron IPC plugin in the main process:
```ts
import {
getRxStorageFilesystemNode
} from 'rxdb-premium/plugins/storage-filesystem-node';
import { app } from 'electron';
import path from 'path';
const storage = getRxStorageFilesystemNode({
basePath: path.join(app.getPath('userData'), 'database')
});
```
Because the application code only sees the RxDB API, you can start with SQLite and move to the filesystem storage later (or the other way around) by exchanging the storage and running the [storage migration](../migration-storage.md).
## Alternatives to SQLite in Electron
SQLite and the filesystem storage in the main process are the recommended defaults, but RxDB supports other storages that can make sense in specific setups:
- The [IndexedDB RxStorage](../rx-storage-indexeddb.md) or [LocalStorage RxStorage](../rx-storage-localstorage.md) run directly in the renderer without any main-process code. Good for quick prototypes, slower for large datasets because [IndexedDB has performance limits](../slow-indexeddb.md).
- The [memory storage](../rx-storage-memory.md) keeps everything in RAM, useful for tests or caches.
A broader comparison of the options is in the [Electron database overview](../electron-database.md).
## FAQ
No. SQLite needs Node.js APIs that the renderer does not provide. The database must run in the main process. With the RxDB [Electron plugin](../electron.md) the renderer still gets a full database API because all operations are forwarded over IPC to the SQLite storage in the main process.
Not when you use the `node:sqlite` module that ships with Node.js 22 and newer, which recent Electron versions include. Third-party packages like `sqlite3` or `better-sqlite3` are native addons and must be rebuilt with [@electron/rebuild](https://github.com/electron/rebuild) whenever the Electron version changes.
The database name you pass to `createRxDatabase()` maps to a SQLite file on disk. Place it inside Electron's user-data folder (from `app.getPath('userData')`) so it survives app updates and follows platform conventions on Windows, macOS and Linux.
Yes. Each `BrowserWindow` connects to the same main-process storage through `getRxStorageIpcRenderer` with the same `key`. Query subscriptions in one window emit new results when another window writes to the database.
## Follow up
- Start with the RxDB [Quickstart](../quickstart.md)
- Read the full [SQLite RxStorage documentation](../rx-storage-sqlite.md)
- Check out the [RxDB Electron example project](https://github.com/pubkey/rxdb/tree/master/examples/electron)
- Compare [other databases for Electron](../electron-database.md)
---
## Embedded Database, Real-time Speed - RxDB
# Using RxDB as an Embedded Database
In modern UI applications, efficient data storage is a crucial aspect for seamless user experiences. One powerful solution for achieving this is by utilizing an embedded database. In this article, we will explore the concept of an embedded database and delve into the benefits of using [RxDB](https://rxdb.info/) as an embedded database in UI applications. We will also discuss why RxDB stands out as a robust choice for real-time applications with embedded database functionality.
## What is an Embedded Database?
An embedded database refers to a client-side database system that is integrated directly within an application. It is designed to operate within the client environment, such as a web browser or a [mobile](./mobile-database.md) app. This approach eliminates the need for a separate database server and allows the database to run locally on the client device.
## Embedded Database in UI Applications
In the context of UI applications, an embedded database serves as a local data storage solution. It enables applications to efficiently manage data, facilitate real-time updates, and enhance performance. Let's explore some of the benefits of using an embedded database compared to a traditional server database:
- Replicating the database state becomes easier: Implementing real-time data synchronization and [replication](../replication.md) is simpler with an embedded database compared to complex REST routes. The embedded nature allows for efficient replication of the database state across multiple instances of the application.
- Using the database for caching: An embedded database can be utilized for caching frequently accessed data. This caching mechanism enhances performance and reduces the need for repeated network requests, resulting in faster data retrieval.
- Building real-time applications is easier with local data: By leveraging local data storage, real-time applications can easily update the user interface in response to data changes. This approach simplifies the development of real-time features and enhances the responsiveness of the application.
- Store local data with [encryption](../encryption.md): Embedded databases, like RxDB, offer the ability to store local data with encryption. This ensures that sensitive information remains protected even when stored locally on the client device.
- Data is accessible offline: With an embedded database, data remains accessible even when the application is offline. Users can continue to interact with the application and access their data seamlessly, irrespective of their internet connectivity.
- Faster initial application start time: Since the data is already stored locally, there is no need for initial data fetching from a remote server. This significantly reduces the application's startup time and allows users to engage with the application more quickly.
- Improved scalability with local queries: Embedded databases, such as RxDB, perform queries locally on the client device instead of relying on server round-trips. This reduces latency and enhances scalability, particularly when dealing with large datasets or high query volumes.
- Seamless integration with JavaScript frameworks: Embedded databases, including RxDB, integrate seamlessly with popular JavaScript frameworks like Angular, React.js, [Vue.js](./vue-database.md), and Svelte. This compatibility allows developers to leverage the capabilities of these frameworks while benefiting from embedded database functionality.
- Running queries locally has low latency: With an embedded database, queries are executed locally on the client device, resulting in minimal latency. This improves the overall performance and responsiveness of the application.
- Data is portable and always accessible by the user: Embedded databases enable data portability, allowing users to seamlessly transition between devices while maintaining their data and application state. This ensures that data is always accessible and available to the user.
- Using a [local database](./local-database.md) for state management: Instead of relying on additional state management libraries like Redux or NgRx, an embedded database can be used for local state management. This simplifies state management and ensures data consistency within the application.
## Why RxDB as an Embedded Database for Real-time Applications
RxDB is a JavaScript-based embedded database that offers numerous advantages for building real-time applications. Let's explore why RxDB is a compelling choice:
- [Observable Queries](../rx-query.md) (RxJS): RxDB leverages the power of Observables through RxJS, enabling developers to create queries that automatically update the user interface on data changes. This reactive approach simplifies UI updates and ensures real-time synchronization of data.
- [NoSQL JSON Documents](./json-database.md) for UIs: RxDB utilizes NoSQL (JSON) documents as its data model, aligning seamlessly with the requirements of modern UI development. JavaScript's native support for JSON objects makes NoSQL documents a natural fit for UI-driven applications.
- Better TypeScript Support Compared to SQL: RxDB's NoSQL approach provides excellent TypeScript support. The flexibility of working with JSON objects enables robust typing and enhanced development experiences, ensuring type safety and reducing runtime errors.
- [Observable Document Fields](../rx-document.md): RxDB allows developers to observe individual fields within documents. This granularity enables efficient tracking of specific data changes and facilitates targeted UI updates, enhancing performance and responsiveness.
- Made in JavaScript, Optimized for JavaScript Applications: Being built entirely in JavaScript, RxDB is optimized for JavaScript applications. It leverages JavaScript's capabilities and integrates seamlessly with JavaScript frameworks and libraries, making it a natural choice for JavaScript developers.
- Optimized Observed Queries with the [EventReduce Algorithm](https://github.com/pubkey/event-reduce): RxDB incorporates the EventReduce algorithm to optimize observed queries. This algorithm reduces the number of emitted events during query execution, resulting in enhanced query performance and reduced overhead.
- Built-in Multi-tab Support: RxDB provides built-in multi-tab support, allowing multiple instances of an application to share and synchronize data seamlessly. This feature enables collaborative and real-time scenarios across multiple browser tabs or windows.
- Handling of Schema Changes across Multiple Client Devices: With RxDB, handling schema changes across multiple client devices becomes straightforward. RxDB's schema [migration capabilities](../migration-schema.md) ensure that applications can seamlessly adapt to evolving data structures, providing a consistent experience across different devices.
- Storing Documents Compressed: RxDB offers the ability to store documents in a compressed format. This reduces the storage footprint and improves performance, especially when dealing with large datasets.
- Flexible Storage Layer and Cross-Platform Compatibility: RxDB provides a flexible storage layer that can be reused across various platforms, including [Electron.js](../electron-database.md), [React Native](../react-native-database.md), hybrid apps (via Capacitor.js), and browsers. This cross-platform compatibility simplifies development and enables code reuse across different environments.
- Replication Algorithm for Backend Compatibility: RxDB's replication algorithm is open-source and can be made compatible with various backend solutions, such as self-hosted servers, Firebase, [CouchDB](../replication-couchdb.md), NATS, WebSockets, and more. This flexibility allows developers to choose their preferred backend infrastructure while benefiting from RxDB's embedded database capabilities.
## Follow Up
To further explore [RxDB](https://rxdb.info/) and leverage its capabilities as an embedded database, the following resources can be helpful:
- [RxDB GitHub Repository](https://github.com/pubkey/rxdb): Visit the official GitHub repository of RxDB to access the source code, documentation, and community support.
- [RxDB Quickstart](../quickstart.md): Get started quickly with RxDB by following the provided quickstart guide, which offers step-by-step instructions for setting up and using RxDB in your projects.
By utilizing [RxDB](https://rxdb.info/) as an embedded database in UI applications, developers can harness the power of efficient data management, real-time updates, and enhanced user experiences. RxDB's features and benefits make it a compelling choice for building modern, responsive, and scalable applications.
---
## RxDB - Firebase Realtime Database Alternative to Sync With Your Own Backend
import {Faq, FaqItem} from '@site/src/components/faq';
# RxDB - The Firebase Realtime Database Alternative That Can Sync With Your Own Backend
Are you on the lookout for a **Firebase Realtime Database alternative** that gives you greater freedom, deeper offline capabilities, and allows you to seamlessly integrate with any backend? **RxDB** (Reactive Database) might be the perfect choice. This [local-first](./local-first-future.md), NoSQL data store runs entirely on the client while supporting real-time updates and robust syncing with any server environment, making it a strong contender against Firebase Realtime Database's limitations and potential vendor lock-in.
## Why RxDB Is an Excellent Firebase Realtime Database Alternative
### 1. Complete Offline-First Experience
Unlike Firebase Realtime Database, which relies on central infrastructure to process data, RxDB is fully embedded within your client application (including [browsers](./browser-database.md), [Node.js](../nodejs-database.md), [Electron](../electron-database.md), and [React Native](../react-native-database.md)). This design means your app stays completely functional offline, since all data reads and writes happen locally. When connectivity is restored, RxDB's syncing framework automatically reconciles local changes with your remote backend.
### 2. Freedom to Use Any Server or Cloud
While Firebase Realtime Database ties you into Google's ecosystem, RxDB allows you to choose any hosting environment. You can:
- Host your data on your own servers or private cloud.
- Integrate with relational databases like [PostgreSQL](../replication-http.md) or other NoSQL options such as [CouchDB](../replication-couchdb.md).
- Build custom endpoints using [REST](../replication-http.md), [GraphQL](../replication-graphql.md), or any other protocol.
This flexibility ensures you're not locked into a single vendor and can adapt your backend strategy as your project evolves.
### 3. Advanced Conflict Handling
Firebase Realtime Database typically updates data with a simple last-in-wins approach. RxDB, on the other hand, lets you implement more sophisticated conflict resolution logic. Using [revisions and conflict handlers](../transactions-conflicts-revisions.md#custom-conflict-handler), RxDB can merge concurrent edits or preserve multiple versions to ensure your application remains consistent even when multiple clients modify the same data at the same time.
### 4. Lower Cloud Costs for Read-Heavy Apps
When you rely on Firebase Realtime Database, each query or listener can translate into ongoing reads, potentially running up your monthly bill. With RxDB, all queries are performed [locally](../offline-first.md). Your app only communicates with the backend to sync document changes, significantly reducing bandwidth and hosting expenses for applications that frequently read data.
### 5. Powerful Local Queries
If you've hit Firebase Realtime Database's querying limits, RxDB offers a far more robust approach to data retrieval. You can:
- Define custom indexes for faster local lookups.
- Perform sophisticated filters, joins, or full-text searches right on the client.
- Subscribe to real-time data updates through RxDB's [reactive query engine](../reactivity.md).
Because these operations happen locally, your [UI updates](./optimistic-ui.md) instantly, providing a snappy user experience.
### 6. True Offline Initialization
While Firebase offers some offline caching, it often requires an initial connection for authentication or to seed local data. RxDB, however, is built to handle an **offline-start** scenario. Users can begin working with the application immediately, regardless of connectivity, and any modifications they make will sync once the network is available again.
### 7. Works Everywhere JavaScript Runs
One of RxDB's core strengths is its ability to run in **any JavaScript environment**. Whether you're building a web app that uses IndexedDB in the browser, an [Electron](../electron-database.md) desktop program, or a [React Native](../react-native-database.md) mobile application, RxDB's **swappable storage** adapts to your runtime of choice. This consistency makes code-sharing and cross-platform development far simpler than being tied to a single backend system.
---
## How RxDB's Syncing Mechanism Operates
RxDB employs its own [Sync Engine](../replication.md) to manage data flow between your client and remote [servers](../rx-server.md). Replication revolves around:
1. **Pull**: Retrieving updated or newly created documents from the server.
2. **Push**: Sending local changes to the backend for persistence.
3. **Live Updates**: Continuously streaming changes to and from the backend for real-time synchronization.
## Sample Code: Sync RxDB With a Custom Endpoint
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { replicateRxCollection } from 'rxdb/plugins/replication';
async function initDB() {
const db = await createRxDatabase({
name: 'localdb',
storage: getRxStorageLocalstorage(),
multiInstance: true,
eventReduce: true
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
complete: { type: 'boolean' }
}
}
}
});
// Start a custom replication
replicateRxCollection({
collection: db.tasks,
replicationIdentifier: 'custom-tasks-api',
push: {
handler: async (docs) => {
// post local changes to your server
const resp = await fetch('https://yourapi.com/tasks/push', {
method: 'POST',
body: JSON.stringify({ changes: docs })
});
return await resp.json(); // return conflicting documents if any
}
},
pull: {
handler: async (lastCheckpoint, batchSize) => {
// fetch new/updated items from your server
const response = await fetch(
`https://yourapi.com/tasks/pull?checkpoint=${JSON.stringify(
lastCheckpoint
)}&limit=${batchSize}`
);
return await response.json();
}
},
live: true
});
return db;
}
```
### Setting Up P2P Replication Over WebRTC
In addition to using a centralized backend, RxDB supports peer-to-peer synchronization through WebRTC, enabling devices to share data directly.
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer,
createSimplePeerWrtc
} from 'rxdb/plugins/replication-webrtc';
const webrtcPool = await replicateWebRTC({
collection: db.tasks,
topic: 'p2p-topic-123',
connectionHandlerCreator: getConnectionHandlerSimplePeer({
signalingServerUrl: 'wss://signaling.rxdb.info/',
wrtc: createSimplePeerWrtc(require('node-datachannel/polyfill')),
webSocketConstructor: require('ws').WebSocket
})
});
webrtcPool.error$.subscribe((error) => {
console.error('P2P error:', error);
});
```
Here, any client that joins the same topic communicates changes to other peers, all without requiring a traditional client-server model.
## Quick Steps to Get Started
1. Install RxDB
```bash
npm install rxdb rxjs
```
2. Create a Local Database
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'myLocalDB',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
notes: {
schema: {
title: 'notes schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
content: { type: 'string' }
}
}
}
});
```
3. Synchronize
Use one of the [Replication Plugins](../replication.md) to connect with your preferred backend.
### Is RxDB the Right Solution for You?
- **Long Offline Use**: If your users need to work without an internet connection, RxDB's built-in offline-first design stands out compared to Firebase Realtime Database's partial offline approach.
- **Custom or Complex Queries**: RxDB lets you perform your [queries](../rx-query.md) locally, define [indexing](../rx-schema.md#indexes), and handle even complex [transformations](../rx-pipeline.md) locally - no extra call to an external API.
- **Avoid Vendor Lock-In**: If you anticipate needing to move or adapt your backend later, you can do so without rewriting how your client manages its data.
- **Peer-to-Peer Collaboration**: Whether you need quick demos or real production use, [WebRTC replication](../replication-webrtc.md) can link your users directly without central coordination of data storage.
## FAQ
You should use Firebase if your primary goal is to offload all backend infrastructure to a fully managed Google Cloud service and your application relies almost entirely on constant internet connectivity. However, if your application requires heavy, complex offline capabilities, true data ownership, or the flexibility to integrate with any existing REST/GraphQL backend, you should opt for an open-source, local-first database alternative like **[RxDB](../rx-database.md)**, which provides Firebase-like real-time UI reactivity without the vendor lock-in.
No, the Firebase Realtime Database and Cloud Firestore are both strict NoSQL, document-oriented data stores. They do not support strict relational schemas or native SQL `JOIN` operations. Developers must manually denormalize data across multiple JSON branches to establish relationships, a pattern perfectly mirrored by local-first NoSQL solutions like **[RxDB](../rx-database.md)** which map the same JSON topologies securely to client-side storage architectures.
---
## RxDB - Firestore Alternative to Sync with Your Own Backend
import {Faq, FaqItem} from '@site/src/components/faq';
# RxDB - The Firestore Alternative That Can Sync with Your Own Backend
If you're seeking a **Firestore alternative**, you're likely looking for a way to:
- **Avoid vendor lock-in** while still enjoying real-time replication.
- **Reduce cloud usage costs** by reading data locally instead of constantly fetching from the server.
- **Customize** how you store, query, and secure your data.
- **Implement advanced conflict resolution** strategies beyond Firestore's last-write-wins approach.
Enter **RxDB** (Reactive Database) - a [local-first](./local-first-future.md), NoSQL database for JavaScript applications that can sync in real time with **any** backend of your choice. Whether you're tired of the limitations and fees associated with Firebase Cloud Firestore or simply need more flexibility, RxDB might be the Firestore alternative you've been searching for.
## What Makes RxDB a Great Firestore Alternative?
Firestore is convenient for many projects, but it does lock you into Google's ecosystem. Below are some of the key advantages you gain by choosing RxDB:
### 1. Fully Offline-First
RxDB runs directly in your client application ([browser](./browser-database.md), [Node.js](../nodejs-database.md), [Electron](../electron-database.md), [React Native](../react-native-database.md), etc.). Data is stored locally, so your application **remains fully functional even when offline**. When the device returns online, RxDB's flexible replication protocol synchronizes your local changes with any remote endpoint.
### 2. Freedom to Use Any Backend
Unlike Firestore, RxDB doesn't require a proprietary hosting service. You can:
- Host your data on your own server (Node.js, Go, Python, etc.).
- Use existing databases like [PostgreSQL](../replication-http.md), [CouchDB](../replication-couchdb.md), or [MongoDB with custom endpoints](../replication.md).
- Implement a [custom GraphQL](../replication-graphql.md) or [REST-based](../replication-http.md) API for syncing.
This **backend-agnostic** approach protects you from vendor lock-in. Your application's client-side data storage remains consistent; only your replication logic (or plugin) changes if you switch servers.
### 3. Advanced Conflict Resolution
Firestore enforces a [last-write-wins](https://stackoverflow.com/a/47781502/3443137) conflict resolution strategy. This might cause issues if multiple users or devices update the same data in complex ways.
RxDB lets you:
- Implement **custom conflict resolution** via [revisions](../transactions-conflicts-revisions.md#custom-conflict-handler).
- Store partial merges, track versions, or preserve multiple user edits.
- Fine-tune how your data merges to ensure consistency across distributed systems.
### 4. Reduced Cloud Costs
Firestore queries often count as billable reads. With RxDB, queries run **locally** against your local state - no repeated network calls or extra charges. You pay only for the data actually synced, not every read. For **read-heavy** apps, using RxDB as a Firestore alternative can significantly reduce costs.
### 5. No Limits on Query Features
Firestore's query engine is limited by certain constraints (e.g., no advanced joins, limited indexing). With RxDB:
- **NoSQL** data is stored locally, and you can define any indexes you need.
- Perform [complex queries](../rx-query.md), run [full-text search](../fulltext-search.md), or do aggregated transformations or even [vector search](./javascript-vector-database.md).
- Use [RxDB's reactivity](../rx-query.md#observe) to subscribe to query results in real time.
### 6. True Offline-Start Support
While Firestore does have offline caching, it often requires an online check at app initialization for authentication. RxDB is [truly offline-first](../offline-first.md); you can launch the app and write data even if the device never goes online initially. It's ready whenever the user is.
### 7. Cross-Platform: Any JavaScript Runtime
RxDB is designed to run in **any environment** that can execute JavaScript. Whether youβre building a web app in the browser, an [Electron](../electron-database.md) desktop application, a [React Native](../react-native-database.md) mobile app, or a command-line tool with [Node.js](../nodejs-database.md), RxDBβs storage layer is swappable to fit your runtimeβs capabilities.
- In the **browser**, store data in [IndexedDB](../rx-storage-indexeddb.md) or [OPFS](../rx-storage-opfs.md).
- In [Node.js](../nodejs-database.md), use LevelDB or other supported storages.
- In [React Native](../react-native-database.md), pick from a range of adapters suited for mobile devices.
- In [Electron](../electron-database.md), rely on fast local storage with zero changes to your application code.
## FAQ
While Firestore provides basic offline caching, it is fundamentally a cloud-first database. True [Offline-First](../offline-first.md) architectures demand that the [local database](./local-database.md) acts as the single source of truth, capable of advanced local querying, custom indexing, and deterministic conflict resolution without ever contacting a server. Firestore's heavy reliance on Google Cloud connections makes it unsuitable for applications that must operate reliably in zero-connectivity environments for extended periods.
---
## How Does RxDB's Sync Work?
RxDB replication is powered by its own [Sync Engine](../replication.md). This simple yet robust protocol enables:
1. **Pull**: Fetch new or updated documents from the server.
2. **Push**: Send local changes back to the server.
3. **Live Real-Time**: Once you're caught up, you can opt for event-based streaming instead of continuous polling.
Code Example: Sync RxDB with a Custom Backend
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { replicateRxCollection } from 'rxdb/plugins/replication';
async function initDB() {
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
multiInstance: true,
eventReduce: true
});
await db.addCollections({
tasks: {
schema: {
title: 'task schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' }
}
}
}
});
// Start a custom REST-based replication
replicateRxCollection({
collection: db.tasks,
replicationIdentifier: 'my-tasks-rest-api',
push: {
handler: async (documents) => {
// Send docs to your REST endpoint
const res = await fetch('https://myapi.com/push', {
method: 'POST',
body: JSON.stringify({ docs: documents })
});
// Return conflicts if any
return await res.json();
}
},
pull: {
handler: async (lastCheckpoint, batchSize) => {
// Fetch from your REST endpoint
const url = 'https://myapi.com/pull' +
`?checkpoint=${JSON.stringify(lastCheckpoint)}` +
`&limit=${batchSize}`;
const res = await fetch(url);
return await res.json();
}
},
live: true // keep watching for changes
});
return db;
}
```
By swapping out the handler implementations or using an official plugin (e.g., [GraphQL](../replication-graphql.md), [CouchDB](../replication-couchdb.md), [Firestore replication](../replication-firestore.md), etc.), you can adapt to any backend or data source. RxDB thus becomes a flexible alternative to Firestore while maintaining [real-time capabilities](./realtime-database.md).
## Getting Started with RxDB as a Firestore Alternative
### Install RxDB:
```bash
npm install rxdb rxjs
```
### Create a Database:
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage()
});
```
### Define Collections:
```ts
await db.addCollections({
items: {
schema: {
title: 'items schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' }
}
}
}
});
```
### Sync
Use a [Replication Plugin](../replication.md) to connect with a custom backend or existing database.
For a Firestore-specific approach, RxDB [Firestore Replication](../replication-firestore.md) also exists if you want to combine local indexing and advanced queries with a Cloud Firestore backend. But if you really want to replace Firestore entirely - just point RxDB to your new backend.
### Example: Start a WebRTC P2P Replication
In addition to syncing with a central server, RxDB also supports pure peer-to-peer replication using [WebRTC](../replication-webrtc.md). This can be invaluable for scenarios where clients need to sync data directly without a master server.
```ts
import {
replicateWebRTC,
getConnectionHandlerSimplePeer,
createSimplePeerWrtc
} from 'rxdb/plugins/replication-webrtc';
const replicationPool = await replicateWebRTC({
collection: db.tasks,
topic: 'my-p2p-room', // Clients with the same topic will sync with each other.
connectionHandlerCreator: getConnectionHandlerSimplePeer({
// Use your own or the official RxDB signaling server
signalingServerUrl: 'wss://signaling.rxdb.info/',
// Node.js requires a polyfill for WebRTC & WebSocket
wrtc: createSimplePeerWrtc(require('node-datachannel/polyfill')),
webSocketConstructor: require('ws').WebSocket
}),
pull: {}, // optional pull config
push: {} // optional push config
});
// The replicationPool manages all connected peers
replicationPool.error$.subscribe(err => {
console.error('P2P Sync Error:', err);
});
```
This example sets up a live **P2P replication** where any new peers joining the same topic automatically sync local data with each other, eliminating the need for a dedicated central server for the actual data exchange.
## Is RxDB Right for Your Project?
- **You want offline-first**: If you need an offline-first app that starts offline, RxDB's local database approach and sync protocol excel at this.
- **Your project is read-heavy**: Reading from Firestore for every query can get expensive. With RxDB, reads are free and local; you only pay for writes or sync overhead.
- **You need advanced queries**: Firestore's query constraints may not suit complex data. With RxDB, you can define your own indexing logic or run arbitrary queries locally.
- **You want no vendor lock-in**: Easily transition from Firestore to your own server or another vendor - just change the replication layer.
## Follow Up
If you've been searching for a Firestore alternative that gives you the freedom to sync your data with any backend, offers robust offline-first capabilities, and supports truly customizable conflict resolution and queries, RxDB is worth exploring. You can adopt it seamlessly, ensure local reads, reduce costs, and stay in complete control of your data layer.
Ready to dive in? Check out the RxDB Quickstart Guide, join our Discord community, and experience how RxDB can be the perfect local-first, real-time database solution for your next project.
More resources:
- [RxDB Sync Engine](../replication.md)
- [Firestore Replication Plugin](../replication-firestore.md)
- [Custom Conflict Resolution](../transactions-conflicts-revisions.md)
- [RxDB GitHub Repository](/code/)
---
## Supercharge Flutter Apps with the RxDB Database
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB as a Database in a Flutter Application
In the world of mobile application development, Flutter has gained significant popularity due to its cross-platform capabilities and rich UI framework. When it comes to building feature-rich Flutter applications, the choice of a robust and efficient database is crucial. In this article, we will explore [RxDB](https://rxdb.info/) as a database solution for Flutter applications. We'll delve into the core features of RxDB, its benefits over other database options, and how to integrate it into a Flutter app.
:::note
You can find the source code for an example RxDB Flutter Application [at the github repo](https://github.com/pubkey/rxdb/tree/master/examples/flutter)
:::
### Overview of Flutter Mobile Applications
Flutter is an open-source UI software development kit created by Google that allows developers to build high-performance [mobile](./mobile-database.md) applications for iOS and Android platforms using a single codebase. Flutter's framework provides a wide range of widgets and tools that enable developers to create visually appealing and responsive applications.
### Importance of Databases in Flutter Applications
Databases play a vital role in Flutter applications by providing a persistent and reliable storage solution for storing and retrieving data. Whether it's user profiles, app settings, or complex data structures, a database helps in efficiently managing and organizing the application's data. Choosing the right database for a Flutter application can significantly impact the performance, scalability, and user experience of the app.
### Introducing RxDB as a Database Solution
RxDB is a powerful NoSQL database solution that is designed to work seamlessly with JavaScript-based frameworks, such as Flutter. It stands for Reactive Database and offers a variety of features that make it an excellent choice for building Flutter applications. RxDB combines the simplicity of JavaScript's document-based database model with the reactive programming paradigm, enabling developers to build real-time and [offline-first](../offline-first.md) applications with ease.
## Getting Started with RxDB
To understand how RxDB can be utilized in a Flutter application, let's explore its core features and advantages.
### What is RxDB?
[RxDB](https://rxdb.info/) is a client-side database built on top of [IndexedDB](../rx-storage-indexeddb.md), which is a low-level [browser-based database](./browser-database.md) API. It provides a simple and intuitive API for performing CRUD operations (Create, Read, Update, Delete) on documents. RxDB's underlying architecture allows for efficient handling of data synchronization between multiple clients and servers.
### Reactive Data Handling
One of the key strengths of RxDB is its reactive data handling. It leverages the power of Observables, a concept from reactive programming, to automatically update the UI in response to data changes. With RxDB, developers can define queries and subscribe to their results, ensuring that the UI is always in sync with the database.
### Offline-First Approach
RxDB follows an offline-first approach, making it ideal for building Flutter applications that need to function even without an internet connection. It allows data to be stored locally and seamlessly synchronizes it with the server when a connection is available. This ensures that users can access and interact with their data regardless of network availability.
### Data Replication
Data replication is a critical aspect of building distributed applications. RxDB provides robust replication capabilities that enable synchronization of data between different clients and servers. With its replication plugins, RxDB simplifies the process of setting up real-time data synchronization, ensuring consistency across all connected devices.
### [Observable Queries](../rx-query.md)
RxDB introduces the concept of observable queries, which are queries that automatically update when the underlying data changes. This feature is particularly useful for keeping the UI up to date with the latest data. By subscribing to an observable query, developers can receive real-time updates and reflect them in the user interface without manual intervention.
### RxDB vs. Other Flutter Database Options
When considering database options for Flutter applications, developers often come across alternatives such as SQLite or LokiJS. While these databases have their merits, RxDB offers several advantages over them. RxDB's seamless integration with Flutter, its offline-first approach, reactive data handling, and built-in data replication make it a compelling choice for building feature-rich and scalable Flutter applications.
## Using RxDB in a Flutter Application
Now that we understand the core features of RxDB, let's explore how to integrate it into a Flutter application.
## How RxDB can run in Flutter
RxDB is written in TypeScript and compiled to JavaScript. To run it in a Flutter application, the `flutter_qjs` library is used to spawn a QuickJS JavaScript runtime. RxDB itself runs in that runtime and communicates with the flutter dart runtime. To store data persistent, the [LokiJS RxStorage](../rx-storage-lokijs.md) is used together with a custom storage adapter that persists the database inside of the `shared_preferences` data.
To use RxDB, you have to create a compatible JavaScript file that creates your [RxDatabase](../rx-database.md) and starts some connectors which are used by Flutter to communicate with the JavaScript RxDB database via setFlutterRxDatabaseConnector().
```javascript
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageLoki
} from 'rxdb/plugins/storage-lokijs';
import {
setFlutterRxDatabaseConnector,
getLokijsAdapterFlutter
} from 'rxdb/plugins/flutter';
// do all database creation stuff in this method.
async function createDB(databaseName) {
// create the RxDatabase
const db = await createRxDatabase({
// the database.name is variable so we can change it on the flutter side
name: databaseName,
storage: getRxStorageLoki({
adapter: getLokijsAdapterFlutter()
}),
multiInstance: false
});
await db.addCollections({
heroes: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string',
maxLength: 100
},
color: {
type: 'string',
maxLength: 30
}
},
indexes: ['name'],
required: ['id', 'name', 'color']
}
}
});
return db;
}
// start the connector so that flutter can communicate with the JavaScript process
setFlutterRxDatabaseConnector(
createDB
);
```
Before you can use the JavaScript code, you have to bundle it into a single .js file. In this example we do that with webpack in a npm script here which bundles everything into the `javascript/dist/index.js` file.
To allow Flutter to access that file during runtime, add it to the assets inside of your pubspec.yaml:
```yaml
flutter:
assets:
- javascript/dist/index.js
```
Also you need to install RxDB in your flutter part of the application. First you have to use the rxdb dart package as a flutter dependency. Currently the package is not published at the dart pub.dev. Instead you have to install it from the local filesystem inside of your RxDB npm installation.
```yaml
# inside of pubspec.yaml
dependencies:
rxdb:
path: path/to/your/node_modules/rxdb/src/plugins/flutter/dart
```
Afterwards you can import the rxdb library in your dart code and connect to the JavaScript process from there. For reference, check out the lib/main.dart file.
```dart
import 'package:rxdb/rxdb.dart';
// start the javascript process and connect to the database
RxDatabase database = await getRxDatabase("javascript/dist/index.js", databaseName);
// get a collection
RxCollection collection = database.getCollection('heroes');
// insert a document
RxDocument document = await collection.insert({
"id": "zflutter-${DateTime.now()}",
"name": nameController.text,
"color": colorController.text
});
// create a query
RxQuery query = RxDatabaseState.collection.find();
// create list to store query results
List> documents = [];
// subscribe to a query
query.$().listen((results) {
setState(() {
documents = results;
});
});
```
### Different RxStorage layers for RxDB
RxDB offers multiple storage options, known as [RxStorage](../rx-storage.md) layers, to store data locally. These options include:
- [LokiJS RxStorage](../rx-storage-lokijs.md): LokiJS is an in-memory database that can be used as a [storage](./browser-storage.md) layer for RxDB. It provides fast and efficient in-memory data management capabilities.
- [SQLite RxStorage](../rx-storage-sqlite.md): SQLite is a popular and widely used [embedded database](./embedded-database.md) that offers robust storage capabilities. RxDB utilizes SQLite as a storage layer to persist data on the device.
- [Memory RxStorage](../rx-storage-memory.md): As the name suggests, Memory RxStorage stores data [in memory](./in-memory-nosql-database.md). While this option does not provide persistence, it can be useful for temporary or cache-based data storage.
By choosing the appropriate RxStorage layer based on the specific requirements of the application, developers can optimize performance and storage efficiency.
## Synchronizing Data with RxDB between Clients and Servers
One of the key strengths of RxDB is its ability to synchronize data between multiple clients and servers seamlessly. Let's explore how this synchronization can be achieved.
### Offline-First Approach
RxDB's offline-first approach ensures that data can be accessed and modified even when there is no internet connection. Changes made offline are automatically synchronized with the server once a connection is reestablished. This ensures data consistency across all devices, providing a seamless user experience.
### RxDB Replication Plugins
RxDB provides replication plugins that simplify the process of setting up data [synchronization between clients and servers](../replication.md). These plugins offer various synchronization strategies, such as one-way replication, two-way replication, and [conflict resolution](../transactions-conflicts-revisions.md) mechanisms. By configuring the appropriate replication plugin, developers can easily establish real-time data synchronization in their Flutter applications.
## Advanced RxDB Features and Techniques
RxDB offers a range of advanced features and techniques that enhance its functionality and performance. Let's explore a few of these features:
### Indexing and Performance Optimization
Indexing is a technique used to optimize query performance by creating indexes on specific fields. RxDB allows developers to define indexes on document fields, improving the efficiency of queries and data retrieval.
### Encryption of Local Data
To ensure data privacy and security, RxDB supports [encryption of local data](../encryption.md). By encrypting the data stored on the device, developers can protect sensitive information and prevent unauthorized access.
### Change Streams and Event Handling
RxDB provides change streams, which emit events whenever data changes occur. By leveraging change streams, developers can implement custom event handling logic, such as updating the UI or triggering background processes, in response to specific data changes.
### JSON Key Compression
To minimize storage requirements and optimize performance, RxDB offers [JSON key compression](../key-compression.md). This feature reduces the size of keys used in the database, resulting in more efficient storage and improved query performance.
## Conclusion
RxDB offers a powerful and flexible database solution for Flutter applications. With its offline-first approach, real-time data synchronization, and reactive data handling capabilities, RxDB simplifies the development of feature-rich and scalable Flutter applications. By integrating RxDB into your Flutter projects, you can leverage its advanced features and techniques to build responsive and data-driven applications that provide an exceptional user experience.
## FAQ
RxDB provides the best local-first database for Flutter applications. You gain full reactive data handling where observable queries automatically update your Flutter UI. The system stores data locally to ensure complete application functionality without an internet connection. Replication plugins handle background synchronization with your server effortlessly. You eliminate complex state management while maintaining consistent data across platforms.
:::note
You can find the source code for an example RxDB Flutter Application [at the github repo](https://github.com/pubkey/rxdb/tree/master/examples/flutter)
:::
---
## RxDB - The Ultimate JS Frontend Database
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB JavaScript Frontend Database: Efficient Data Storage in Frontend Applications
In modern web development, managing data on the front end has become increasingly important. Storing data in the frontend offers numerous advantages, such as offline accessibility, caching, faster application startup, and improved state management. Traditional SQL databases, although widely used on the server-side, are not always the best fit for frontend applications. This is where [RxDB](https://rxdb.info/), a frontend JavaScript database, emerges as a powerful solution. In this article, we will explore why storing data in the frontend is beneficial, the limitations of SQL databases in the frontend, and how [RxDB](https://rxdb.info/) addresses these challenges to become an excellent choice for frontend data storage.
## Why you might want to store data in the frontend
### Offline accessibility
One compelling reason to store data in the frontend is to enable [offline accessibility](../offline-first.md). By leveraging a frontend database, applications can cache essential data locally, allowing users to continue using the application even when an internet connection is unavailable. This feature is particularly useful for [mobile](./mobile-database.md) applications or web apps with limited or intermittent connectivity.
### Caching
Frontend databases also serve as efficient caching mechanisms. By storing frequently accessed data locally, applications can minimize network requests and reduce latency, resulting in faster and more responsive user experiences. Caching is particularly beneficial for applications that heavily rely on remote data or perform computationally intensive operations.
### Decreased initial application start time
Storing data in the frontend decreases the initial application start time because the data is already present locally. By eliminating the need to fetch data from a server during startup, applications can quickly render the UI and provide users with an immediate interactive experience. This is especially advantageous for applications with large datasets or complex data retrieval processes.
### Password encryption for local data
Security is a crucial aspect of data storage. With a front end database, developers can [encrypt](../encryption.md) sensitive local data, such as user credentials or personal information, using encryption algorithms. This ensures that even if the device is compromised, the data remains securely stored and protected.
### Local database for state management
Frontend databases provide an alternative to traditional state management libraries like Redux or NgRx. By using a [local database](./local-database.md), developers can store and manage application state directly in the frontend, eliminating the need for additional libraries. This approach simplifies the codebase, reduces complexity, and provides a more straightforward data flow within the application.
### Low-latency local queries
Frontend databases enable low-latency queries that run entirely on the client's device. Instead of relying on server round-trips for each query, the database executes queries locally, resulting in faster response times. This is particularly beneficial for applications that require real-time updates or frequent data retrieval.
### Building realtime applications with local data
Realtime applications often require immediate updates based on data changes. By storing data locally and utilizing a frontend database, developers can build [realtime applications](./realtime-database.md) more easily. The database can observe data changes and automatically update the UI, providing a seamless and responsive user experience.
### Easier integration with JavaScript frameworks
Frontend databases, including RxDB, are designed to integrate seamlessly with popular JavaScript frameworks such as [Angular](./angular-database.md), [React.js](./react-database.md), [Vue.js](./vue-database.md), and Svelte. These databases offer well-defined APIs and support that align with the specific requirements of these frameworks, enabling developers to leverage the full potential of the frontend database within their preferred development environment.
### Simplified replication of database state
Replicating database state between the frontend and backend can be challenging, especially when dealing with complex REST routes. Frontend databases, however, provide simple mechanisms for replicating database state. They offer intuitive replication algorithms that facilitate data synchronization between the frontend and backend, reducing the complexity and potential pitfalls associated with complex REST-based replication.
### Improved scalability
Frontend databases offer improved scalability compared to traditional SQL databases. By leveraging the computational capabilities of client devices, the burden on server resources is reduced. Queries and operations are performed locally, minimizing the need for server round-trips and enabling applications to scale more efficiently.
## Why SQL databases are not a good fit for the front end of an application
While SQL databases excel in server-side scenarios, they pose limitations when used on the frontend. Here are some reasons why SQL databases are not well-suited for frontend applications:
### Push/Pull based vs. reactive
SQL databases typically rely on a push/pull model, where the server pushes data to the client upon request. This approach is not inherently reactive, as it requires explicit requests for data updates. In contrast, frontend applications often require reactive data flows, where changes in data trigger automatic updates in the UI. Frontend databases, like [RxDB](https://rxdb.info/), provide reactive capabilities that seamlessly integrate with the dynamic nature of frontend development.
### Initialization time and performance
SQL databases designed for server-side usage tend to have larger build sizes and initialization times, making them less efficient for [browser-based](./browser-database.md) applications. Frontend databases, on the other hand, directly leverage browser APIs like [IndexedDB](../rx-storage-indexeddb.md), [OPFS](../rx-storage-opfs.md), and [WebWorker](../rx-storage-worker.md), resulting in leaner builds and faster initialization times. Often the queries are such fast, that it is not even necessary to implement a loading spinner.
### Build size considerations
Server-side SQL databases typically come with a significant build size, which can be impractical for browser applications where code size optimization is crucial. Frontend databases, on the other hand, are specifically designed to operate within the constraints of browser environments, ensuring efficient resource utilization and smaller build sizes.
For example the [SQLite](../rx-storage-sqlite.md) Webassembly file alone has a size of over 0.8 Megabyte with an additional 0.2 Megabyte in JavaScript code for connection.
## Why RxDB is a good fit for the frontend
RxDB is a powerful frontend JavaScript database that addresses the limitations of SQL databases and provides an optimal solution for frontend [data storage](./browser-storage.md). Let's explore why RxDB is an excellent fit for frontend applications:
### Made in JavaScript, optimized for JavaScript applications
RxDB is designed and optimized for JavaScript applications. Built using JavaScript itself, RxDB offers seamless integration with JavaScript frameworks and libraries, allowing developers to leverage their existing JavaScript knowledge and skills.
### NoSQL (JSON) documents for UIs
RxDB adopts a [NoSQL approach](./in-memory-nosql-database.md), using [JSON documents as its primary data structure](./json-database.md). This aligns well with the JavaScript ecosystem, as JavaScript natively works with JSON objects. By using NoSQL documents, RxDB provides a more natural and intuitive data model for UI-centric applications.
### Better TypeScript support compared to SQL
TypeScript has become increasingly popular for building frontend applications. RxDB provides excellent [TypeScript support](../tutorials/typescript.md), allowing developers to leverage static typing and benefit from enhanced code quality and tooling. This is particularly advantageous when compared to SQL databases, which often have limited TypeScript support.
### [Observable Queries](../rx-query.md) for automatic UI updates
RxDB introduces the concept of observable queries, powered by RxJS. Observable queries automatically update the UI whenever there are changes in the underlying data. This reactive approach eliminates the need for manual UI updates and ensures that the frontend remains synchronized with the database state.
### Optimized observed queries with the EventReduce Algorithm
RxDB optimizes observed queries with its EventReduce Algorithm. This algorithm intelligently reduces redundant events and ensures that UI updates are performed efficiently. By minimizing unnecessary re-renders, RxDB significantly improves performance and responsiveness in frontend applications.
```typescript
const query = myCollection.find({
selector: {
age: {
$gt: 21
}
}
});
const querySub = query.$.subscribe(results => {
console.log('got results: ' + results.length);
});
```
### Observable document fields
RxDB supports observable document fields, enabling developers to track changes at a granular level within documents. By observing specific fields, developers can reactively update the UI when those fields change, ensuring a responsive and synchronized frontend interface.
```typescript
myDocument.firstName$.subscribe(newName => console.log('name is: ' + newName));
```
### Storing Documents Compressed
RxDB provides the option to store documents in a [compressed format](../key-compression.md), reducing storage requirements and improving overall database performance. Compressed storage offers benefits such as reduced disk space usage, faster data read/write operations, and improved network transfer speeds, making it an essential feature for efficient frontend data storage.
### Built-in Multi-tab support
RxDB offers built-in multi-tab support, allowing data synchronization and state management across multiple browser tabs. This feature ensures consistent data access and synchronization, enabling users to work seamlessly across different tabs without conflicts or data inconsistencies.
### Replication Algorithm can be made compatible with any backend
RxDB's [realtime replication algorithm](../replication.md) is designed to be flexible and compatible with various backend systems. Whether you're using your own servers, [Firebase](../replication-firestore.md), [CouchDB](../replication-couchdb.md), [NATS](../replication-nats.md), [WebSocket](../replication-websocket.md), or any other backend, RxDB can be seamlessly integrated and synchronized with the backend system of your choice.
### Flexible storage layer for code reuse
RxDB provides a [flexible storage layer](../rx-storage.md) that enables code reuse across different platforms. Whether you're building applications with [Electron.js](../electron-database.md), [React Native](../react-native-database.md), hybrid apps using [Capacitor.js](../capacitor-database.md), or traditional web browsers, RxDB allows you to reuse the same codebase and leverage the power of a frontend database across different environments.
### Handling schema changes in distributed environments
In distributed environments where data is stored on multiple client devices, handling schema changes can be challenging. RxDB tackles this challenge by providing robust mechanisms for [handling schema changes](../migration-schema.md). It ensures that schema updates propagate smoothly across devices, maintaining data integrity and enabling seamless schema evolution.
## Follow Up
To further explore RxDB and get started with using it in your frontend applications, consider the following resources:
- [RxDB Quickstart](../quickstart.md): A step-by-step guide to quickly set up RxDB in your project and start leveraging its features.
- [RxDB GitHub Repository](https://github.com/pubkey/rxdb): The official repository for RxDB, where you can find the code, examples, and community support.
By adopting [RxDB](https://rxdb.info/) as your frontend database, you can unlock the full potential of frontend data storage and empower your applications with offline accessibility, caching, improved performance, and seamless data synchronization. RxDB's JavaScript-centric approach and powerful features make it an ideal choice for frontend developers seeking efficient and scalable data storage solutions.
---
## Generic Prompts for GitHub Repos - Reusable Agent Prompts for Any Project
# Generic Prompts for GitHub Repos
Prompts that are useful for any GitHub or open source project which you can give your agent to improve the project.
## Clean up
Stuff to cleanup build and installations so we have less errors and warnings.
### Remove warnings
Removing warnings is good to reduce the context size during agent runs because the terminal has less noisy output.
```txt
Run the installation and build and find the first warning. Fix that and make a pull request.
```
### Remove unused dependencies
-------------
Search for dependencies that you no longer need to reduce install times, noise and token usage. Also reduces risk of supply-chain attacks.
```txt
Run the installation and find dependencies that are not used anymore. Ensure that these are really not used, not on the core and not in the tests.
Remove these unused dependencies and make a pull request.
```
Example resulting PRs:
- https://github.com/pubkey/rxdb/pull/8351
-------------
## Correctness
### Find a bug and fix it
```txt
Find a bug in FEATURE_NAME and make a test case for it. First run the test case without a fix and show me the output. Then apply a fix and run the test case again and show me the output. To reproduce the bug, you can only use the public API and correct TypeScript type usage. Using the API wrongly or with different types does not count as a bug. Also in the test case you can only use the public API and correct TypeScript type usage, you cannot check for internal APIs or behavior.
- Ensure all other tests run successful.
- Run the performance tests before and after the fix and show me the difference.
- Add the fix to the changelog.
- Ensure the linting is ok.
```
Example resulting PRs:
https://github.com/pubkey/rxdb/pull/8275
-------------
## Performance
```txt
Improve the performance of FEATURE_NAME.
Run the performance tests before and after
the improvement and show me the difference.
When you find multiple ways to improve
performance, make a performance difference
comparison table to show me which of the
improvements are the most effective. Only keep
the improvements that make a significant
performance improvement. After each improvement
ensure all tests still work and that the
linting is ok.
Add the improvement to the changelog.
```
-------------
## Code Quality
## Security
## SEO
-------------
```txt
Go through all documentation pages and make a list of them with a short description of what they are about and their internal link url. Then go through the content of each page and check which keywords could should be linked internally to other pages of the table. Only add links if that page does not already have a link to the target page.
```
-------------
---
## ideas for articles
- storing and searching through 1mio emails in a browser database
- Finding the optimal way to shorten vector embeddings
- Performance and quality of vector comparison functions (euclideanDistance etc)
- performance and quality of vector indexing methods
- What is new in IndexedDB 3.0
- how progressive syncing beats client-server architecture
- beating expo sqlite performce with the new expo filesystem and rxdb
- linkedin post: How RxDB is optimized for LLMs to make you a 100x developer
- how WebMCP & Local-First gives your app superpowers
- How Local-First and WebMCP make your app accessible to agents
- DX for LLMs
- vibe-coding is the killer-app for local-first
- agent-first with RxDB
- "why the indexeddb API is almost perfekt"
- "how to do auth with RxDB"
- "Where to store that JWT token?"
## Seo keywords:
X- "optimistic ui"
X- "local database" (rddt done)
X- "react-native [encryption](../encryption.md)"
X- "vue database" (rddt done)
X- "jquery database"
X- "vue indexeddb"
X- "firebase realtime database alternative" (rddt done)
X- "firestore alternative" (rddt done)
X- "ionic storage" (rddt done)
X- "local database"
X- "offline database"
X- "zero local first"
X- "webrtc p2p" - 390 http://localhost:3000/[replication](../replication.md)-webrtc.html
X- "indexeddb storage limit" - 590 https://rxdb.info/articles/indexeddb-max-storage-limit.html
X- "indexeddb size limit" - 260 https://rxdb.info/articles/indexeddb-max-storage-limit.html
X- "indexeddb max size" - 590 https://rxdb.info/articles/indexeddb-max-storage-limit.html
X- "indexeddb limits" - 170 https://rxdb.info/articles/indexeddb-max-storage-limit.html
X- "json based database"
X- "json vs database"
X- "reactjs storage"
## Seo
- "supabase alternative"
- "store local storage"
- "react localstorage"
- "react-native storage"
- "supabase offline" - 260
- "store array in localstorage", "localStorage array of objects"
- "real time web apps" - 170
- "reactive database" - 210
- "electron sqlite"
- "in browser database" - 90
- "[offline first](../offline-first.md) app" - 260
- "react native sql" - 110
- "sqlite electron"
- "localstorage vs indexeddb"
- "react native nosql database" - 30
- "indexeddb library" - 260
- "indexeddb encryption" - 90
- "client side database" - 140
- "webtransport vs websocket"
- "local first development" - 210
- "local storage examples"
- "local vector database" - 590
- "mobile app database" - 590
- "web based database"
- "livequery" - 210
- "expo database" - 390
- "database sync" - 8100
- "p2p database" - 170
- "reactive app" - 260
- "offline web app" - 320
- "offline sync" - 320
- "react native encrypted storage" - 1000
- "firestore vs firebase" - 1300
- "ionic alternatives" - 480
- "react native backend" - 720
- "react native alternative" - 1000
- "react native sqlite" - 1900
- "flutter vs react native" - 5400
- "react native redux" - 3600
- "redux alternative" - 1300
- "Awesome local first" - 10
- "tauri database" - 170
- "capacitor embedded database"
- "Node.js embedded database"
- "sqlite javascript" - 2900
- "sqlite typescript" - 260
- "sync engine" - 390
- "indexeddb alternative" - 70
---
## RxDB In-Memory NoSQL - Supercharge Real-Time Apps
# RxDB as In-memory NoSQL Database: Empowering Real-Time Applications
Real-time applications have become increasingly popular in today's digital landscape. From instant messaging to collaborative editing tools, the demand for responsive and interactive software is on the rise. To meet these requirements, developers need powerful and efficient database solutions that can handle large amounts of data in real-time. [RxDB](https://rxdb.info/), an javascript NoSQL database, is revolutionizing the way developers build and scale their applications by offering exceptional speed, flexibility, and scalability.
## Speed and Performance Benefits
One of the key advantages of using RxDB as an in-memory NoSQL database is its ability to leverage in-memory storage for faster database operations. By storing data directly in memory, database operations can be performed significantly faster compared to traditional disk-based databases. This is especially important for real-time applications where every millisecond counts. With RxDB, developers can achieve near-instantaneous data access and manipulation, enabling highly responsive user experiences.
Additionally, RxDB eliminates disk I/O bottlenecks that are typically associated with traditional databases. In traditional databases, disk reads and writes can become a bottleneck as the amount of data grows. In contrast, an in-memory database like RxDB keeps the entire dataset in RAM, eliminating disk access overhead. This makes it an excellent choice for applications dealing with real-time analytics, high-throughput data processing, and caching.
## Persistence Options
While RxDB offers an [in-memory](../rx-storage-memory.md) storage adapter, it also offers [persistence storages](../rx-storage.md). Adapters such as [IndexedDB](../rx-storage-indexeddb.md), [SQLite](../rx-storage-sqlite.md), and [OPFS](../rx-storage-opfs.md) enable developers to persist data locally in the browser, making applications accessible even when [offline](../offline-first.md). This hybrid approach combines the benefits of in-memory performance with data durability, providing the best of both worlds. Developers can choose the adapter that best suits their needs, balancing the speed of in-memory storage with the long-term data persistence required for certain applications.
```javascript
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageMemory
} from 'rxdb/plugins/storage-memory';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageMemory()
});
```
Also the [memory mapped RxStorage](../rx-storage-memory-mapped.md) exists as a wrapper around any other RxStorage. The wrapper creates an in-memory storage that is used for query and write operations. This memory instance is replicated with the underlying storage for persistence. The main reason to use this is to improve initial page load and query/write times. This is mostly useful in browser based applications.
## Use Cases for RxDB
RxDB's capabilities make it well-suited for various real-time applications. Some notable use cases include:
- Chat Applications and Real-Time Messaging: RxDB's in-memory performance and real-time synchronization capabilities make it an excellent choice for building chat applications and real-time messaging systems. Developers can ensure that messages are delivered and synchronized across multiple clients in real-time, providing a seamless and responsive chat experience.
- Collaborative Document Editors: RxDB's ability to handle data streams and propagate changes in real-time makes it ideal for collaborative document editing. Multiple users can simultaneously edit a document, and their changes are instantly synchronized, allowing for real-time collaboration and ensuring that everyone has the most up-to-date version of the document.
- Real-Time Analytics Dashboards: RxDB's speed and scalability make it a valuable tool for real-time analytics dashboards. It can handle high volumes of data and perform complex analytics operations in real-time, providing instant insights and visualizations to users.
In conclusion, RxDB serves as a powerful in-memory NoSQL database that empowers developers to build real-time applications with exceptional speed, flexibility, and scalability. Its ability to leverage in-memory storage, eliminate disk I/O bottlenecks, and provide persistence options make it an attractive choice for a wide range of real-time use cases. Whether it's chat applications, collaborative document editors, or real-time analytics dashboards, RxDB provides the foundation for building responsive and interactive software that meets the demands of today's users.
---
## Best IndexedDB Wrapper - Compare Dexie, idb, localForage, PouchDB and RxDB
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {Faq, FaqItem} from '@site/src/components/faq';
# Best IndexedDB Wrapper
[IndexedDB](../../rx-storage-indexeddb.md) is the standard [browser storage](../browser-storage.md) API for structured data. Every modern browser ships it, it can store megabytes to gigabytes of JSON and binary data, and it works offline. But the native API is low-level and verbose. It relies on event callbacks, forces you to open a transaction for every read and write, and gives you nothing to query with beyond simple key ranges.
That is why almost nobody uses raw IndexedDB directly. Instead you pick an **IndexedDB wrapper**: a library that hides the callbacks, adds promises, and often layers queries, schemas, and reactivity on top.
This page lists the most used IndexedDB wrappers, compares their features and performance, and ends with a feature table so you can pick the right one for your app.
## Why You Need a Wrapper
The native IndexedDB API was designed as a building block for library authors, not for application code. Writing directly against it bites back quickly:
- **Callback based**: You handle `onsuccess` and `onerror` on every request, which nests control flow and makes error handling hard.
- **Manual transactions**: Every operation needs an explicit transaction and object store lookup, which is repetitive boilerplate.
- **No real queries**: You can only match by key or key range. Anything like "find users older than 18 sorted by name" means iterating a cursor by hand. See [Slow IndexedDB](../../slow-indexeddb.md) for why this also hurts performance.
- **No schema**: IndexedDB stores anything. That sounds flexible until inconsistent documents crash your app at runtime.
- **No change events**: There is no way to subscribe to data changes, so you build your own event bus to keep the UI in sync.
A wrapper solves some or all of these. The wrappers below sit on a spectrum. On one end are thin promise shims that only remove the callback pain. On the other end are full databases that happen to use IndexedDB as one of several storage backends.
## The IndexedDB Wrappers
### Dexie.js
[Dexie.js](https://dexie.org/) is the most popular minimalist wrapper. It gives you a clean promise based API, a fluent query builder (`db.users.where('age').above(18).toArray()`), and a schema declaration for indexes. It is small, well documented, and battle-tested in production.
Keep in mind that Dexie's queries are indexed range queries, not full NoSQL-style queries. The `where()` builder works on fields you declared as indexes, with operators like `above()`, `below()`, `between()`, `anyOf()`, and `startsWith()`. Anything beyond an indexed field falls back to `.filter()`, which runs a linear in-memory scan over the matched rows. There is no Mango-style selector language with `$or`, `$gt` on arbitrary fields, or nested field conditions like RxDB has. So Dexie is ergonomic for index-driven lookups, not for rich ad-hoc queries.
Dexie stays close to IndexedDB. It does not add its own document format on top, so writes go almost straight through to the store. It also offers `liveQuery()` for reactive results and a paid add-on for server sync.
**Good for**: apps that want ergonomic IndexedDB access with indexed range queries and a small footprint.
**Falls short when**: you need rich NoSQL queries, built-in replication, conflict handling, or schema validation beyond index declarations.
### idb
[idb](https://github.com/jakearchibald/idb) by Jake Archibald is the thinnest wrapper of all. It is a tiny promise based mirror of the raw IndexedDB API, under 1 KB. It does not add queries, schemas, or reactivity. It only replaces the callback style with promises and async iterators.
**Good for**: library authors and developers who want full control over IndexedDB with almost no abstraction and no bundle cost.
**Falls short when**: you want queries, indexes as a first-class concept, or any database feature. You still write low-level store and transaction code.
### localForage
[localForage](https://localForage.github.io/localForage/) is a key-value wrapper with automatic fallbacks. It picks IndexedDB when available and falls back to WebSQL or localStorage. The API is `getItem`, `setItem`, `removeItem`, so it feels like localStorage but async and with larger limits.
**Good for**: caching blobs or JSON values by key when you do not need queries.
**Falls short when**: you need to query by anything other than the key. There are no indexes, no filtering, and no sorting. We wrote a dedicated [localForage alternative](../alternatives/localforage-alternative.md) comparison.
### PouchDB
[PouchDB](https://pouchdb.com/) is a full document database that runs in the browser on top of IndexedDB and syncs with [CouchDB](../alternatives/couchdb-alternative.md). It brings a document model, map/reduce views, and a proven replication protocol.
PouchDB is capable but heavy. The revision tree it keeps for conflict handling grows the storage size, and its performance on large datasets in IndexedDB is a known pain point. See the [PouchDB alternative](../alternatives/pouchdb-alternative.md) page for details.
**Good for**: apps that sync with a CouchDB backend and want offline replication out of the box.
**Falls short when**: you care about bundle size and write performance, or you do not use CouchDB on the server.
### JsStore
[JsStore](https://jsstore.net/) wraps IndexedDB with an SQL-like query API and runs the work inside a Web Worker so heavy queries do not block the main thread. You write queries as JSON objects that resemble SQL statements.
**Good for**: teams that prefer an SQL mental model and want queries off the main thread.
**Falls short when**: you want a promise-native document API or a large ecosystem. The community is smaller than Dexie's.
### LokiJS
[LokiJS](https://github.com/techfort/LokiJS) is an in-memory document store with an optional IndexedDB persistence adapter. Because it queries in memory, reads are fast once loaded, but the whole dataset must fit in RAM and gets serialized to IndexedDB in bulk.
LokiJS is no longer actively maintained, which matters for a dependency at the core of your app.
**Good for**: small datasets that fit in memory and need fast in-memory filtering.
**Falls short when**: your data grows past what fits in RAM, or you need an actively maintained library. See the [LokiJS alternative](../alternatives/lokijs-alternative.md) page.
### RxDB
[RxDB](https://rxdb.info/) (Reactive Database) is a local-first, NoSQL database for JavaScript applications. It runs in the browser, Node.js, Electron, React Native, Capacitor, Deno, and Bun. It uses IndexedDB (or faster storages like [OPFS](../../rx-storage-opfs.md)) under the hood through its [RxStorage](../../rx-storage.md) layer, and adds a full database on top.
RxDB is not only a wrapper. It gives you [JSON Schema](../../rx-schema.md) validation, MongoDB-style (Mango) [queries](../../rx-query.md) with indexes, [reactive queries](../../reactivity.md) that re-emit when data changes, [multi-tab](../../rx-storage-indexeddb.md) coordination, schema [migrations](../../migration-schema.md), [encryption](./indexeddb-encryption.md), and a [Sync Engine](../../replication.md) for realtime replication with many backends.
The important part for performance is the storage abstraction. RxDB does not lock you to IndexedDB. Switching storages is a configuration change, not a rewrite, so you can start on IndexedDB and move to OPFS when you need more speed.
**Good for**: apps that need a real client-side database with queries, reactivity, and sync, on any JavaScript runtime.
**Falls short when**: you only want to store a handful of key-value flags. For that a thin wrapper like idb or localForage is enough.
## Performance Comparison
Raw IndexedDB is slow, and a thin wrapper cannot make the underlying store faster. It only removes the callback overhead. So for the thin wrappers (idb, Dexie, localForage) the performance is close to native IndexedDB, plus the small cost of the abstraction.
The chart below compares native IndexedDB, Dexie.js, and RxDB storages (IndexedDB-based and the OPFS storage) across common operations. Lower is better.
Two things stand out. First, Dexie's bulk insert is slower than writing straight to IndexedDB because of the extra work it does per document. Second, the [OPFS storage](../../rx-storage-opfs.md) that RxDB can use beats plain IndexedDB by a wide margin on most operations, which is impossible with an IndexedDB-only wrapper. You can reproduce all of these tests in the [RxStorage performance](../../rx-storage-performance.md) repo.
The takeaway: if your bottleneck is IndexedDB itself, no wrapper that only wraps IndexedDB will help. You need a library that can swap the storage engine.
## How to Choose
- Pick **idb** when you want the smallest possible promise shim and will write store logic yourself.
- Pick **Dexie.js** when you want ergonomic indexed queries with a small footprint and no sync.
- Pick **localForage** when you only store values by key and want localStorage-style code.
- Pick **PouchDB** when you sync with a CouchDB backend and accept the size and speed cost.
- Pick **JsStore** when you prefer SQL-style queries running in a Web Worker.
- Pick **RxDB** when you need a real database: schemas, reactive queries, multi-tab, migrations, and [replication](../../replication.md), with the option to run faster storages than IndexedDB.
## FAQ
It depends on the job. For a thin promise layer, **[idb](https://github.com/jakearchibald/idb)** is the smallest. For indexed queries with a small footprint, **[Dexie.js](https://dexie.org/)** is the popular choice. For a full client-side database with reactivity and sync, **[RxDB](../../rx-database.md)** does more than wrap IndexedDB and can run faster storages like OPFS underneath.
No. Dexie sits on top of IndexedDB and adds a small overhead per operation, so bulk writes are slower than writing straight to the store. It buys you a cleaner API and query builder, not more speed. To go faster than IndexedDB you need a different storage engine such as the **[OPFS storage](../../rx-storage-opfs.md)**.
You can use it directly, but the native API is callback based, needs a transaction per operation, and has no real query support. Most teams pick a wrapper to avoid that boilerplate. See the [IndexedDB alternative](../indexeddb-alternative.md) page for a full breakdown of the raw API's problems.
**PouchDB** syncs with CouchDB, and **[RxDB](../../replication.md)** ships a Sync Engine that replicates with many backends including CouchDB, GraphQL, HTTP, Supabase, Firestore, and its own replication server. Thin wrappers like idb, Dexie, and localForage do not include replication.
Yes. RxDB uses IndexedDB as one storage backend but abstracts it behind [RxStorage](../../rx-storage.md). You can switch to OPFS, in-memory, SQLite, or other storages without changing your application code, which is why RxDB is not tied to IndexedDB performance.
## Comparison Table
| Feature | RxDB | idb | Dexie.js | localForage | PouchDB | LokiJS |
| --- | --- | --- | --- | --- | --- | --- |
| Data model | Documents in collections | Raw stores | Tables + indexes | Key-value | Documents | Documents (in-memory) |
| Promise API | β | β | β | β | β | β οΈ callback |
| Queries | β Mango + indexes | β | β οΈ indexed range + `filter()` | β | β map/reduce | β in-memory |
| Schema validation | β JSON Schema | β | β οΈ indexes only | β | β | β |
| Reactivity | β observable queries | β | β οΈ liveQuery | β | β οΈ changes feed | β οΈ events |
| Multi-tab sync | β built in | β | β οΈ manual | β | β | β |
| Replication | β many backends | β | β οΈ paid add-on | β | β CouchDB | β |
| Migrations | β strategies | β | β versioned | β | β οΈ manual | β |
| Encryption | β plugin | β | β οΈ add-on | β | β οΈ plugin | β |
| Storage backends | IndexedDB, OPFS, SQLite, memory, more | IndexedDB | IndexedDB | IndexedDB, WebSQL, localStorage | IndexedDB | IndexedDB, memory, files |
| Bundle size | Medium | Tiny | Small | Small | Large | Small |
| Active development | Active | Low | Active | Low | Moderate | Inactive |
## Follow Up
- Start with the [RxDB Quickstart](../../quickstart.md)
- Read why [IndexedDB is slow](../../slow-indexeddb.md) and how to work around it
- Compare browser storage APIs in [LocalStorage vs. IndexedDB vs. OPFS](../localstorage-indexeddb-cookies-opfs-sqlite-wasm.md)
- Learn about the [RxStorage](../../rx-storage.md) layer and the [OPFS storage](../../rx-storage-opfs.md)
- Check the [RxDB code on GitHub](/code/) and leave a star β
---
## IndexedDB Encryption - How to Encrypt Data Stored in IndexedDB
import {Steps} from '@site/src/components/steps';
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_ENCRYPTION, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
# IndexedDB Encryption
**IndexedDB encryption** means protecting the data your app stores in the browser so that it cannot be read from disk without the right key. By default, [IndexedDB](../indexeddb-alternative.md) writes everything as plain text on the user's device, which is fine for a todo demo and a real problem for anything sensitive. This page explains what IndexedDB stores on disk, why the native API cannot encrypt it for you, and how [RxDB](https://rxdb.info/) adds transparent field-level encryption on top of IndexedDB.
## Is IndexedDB Encrypted by Default?
No. IndexedDB is not encrypted at rest. The browser writes your object stores to a database file on the user's disk in plain text, and the file format is documented. In Chrome the data lives inside a LevelDB directory, in Firefox inside a SQLite file, and in both cases anyone with read access to the profile folder can open the file and read your records.
This surprises many developers, because IndexedDB feels private. It runs inside the browser, it is scoped to a single [origin](../browser-storage.md), and other websites cannot touch it. But same-origin isolation is not encryption. It stops another website from reading your data through the browser. It does nothing against someone who reads the raw file from disk.
So the threat model is simple. If an attacker gets file-level access to the machine, a stolen laptop, a shared computer, a malicious desktop process, or a browser extension with the right permissions, they can read every unencrypted IndexedDB record you ever wrote. For credentials, tokens, health data, or financial records, that is not acceptable.
## Why IndexedDB Cannot Encrypt Data for You
The native IndexedDB API has no encryption feature. There is no option on `indexedDB.open()`, no flag on an object store, and no callback where you could plug in a cipher. The API was designed as a [low-level storage engine](../local-database.md), and encryption was left to the layers above it.
That leaves you with one native option: encrypt the values yourself with the [Web Crypto API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API) before you call `store.put()`, and decrypt them again after every `store.get()`. It works, but it is painful for a few concrete reasons.
- **You lose querying.** Once a field is a ciphertext string, an IndexedDB index over it is useless. You cannot ask the browser to "find all documents where `status` equals `active`", because on disk `status` is now random bytes. Every filter has to load and decrypt records by hand.
- **You hand-wire every read and write.** Web Crypto is asynchronous and returns an `ArrayBuffer`. You have to manage the key, the initialization vector, the encoding to and from a storable string, and wrap every single access point in the browser.
- **One missed spot leaks data.** The moment one code path writes a value without going through your encryption helper, that record lands on disk in plain text, and you will not notice until someone reads the file.
The naive fix is to encrypt the whole database blob as one string. This works until you have more than a handful of records, because now every read decrypts everything and every write re-encrypts everything. This will not scale.
## How RxDB Adds Encryption on Top of IndexedDB
[RxDB](https://rxdb.info/) (Reactive Database) is a local-first, NoSQL database for JavaScript applications. It runs on top of IndexedDB and other storages, and its [encryption plugin](../../encryption.md) wraps any [RxStorage](../../rx-storage.md) so that flagged fields are encrypted before they are written and decrypted when you read them back. Your data still lives in IndexedDB under the hood. Switching storages is a configuration change, not a rewrite.
The encryption is a wrapper around the storage, so the same setup works with the free [Dexie.js storage](../../rx-storage-dexie.md) and with the premium [IndexedDB RxStorage](../../rx-storage-indexeddb.md). RxDB handles the cipher, the key derivation, and the encoding for you. You only declare which fields are secret.
### 1. Wrap the IndexedDB Storage With Encryption
The encryption plugin takes your normal storage and returns an encrypted one. Everything else about the database stays the same.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
// wrap the Dexie/IndexedDB storage with the encryption plugin
const encryptedStorage = wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageDexie() // stores into IndexedDB under the hood
});
// the password decrypts the data, so keep it out of the source code
const db = await createRxDatabase({
name: 'mydatabase',
storage: encryptedStorage,
password: 'sudoLetMeIn'
});
```
### 2. Mark the Secret Fields in the Schema
You declare encrypted fields with the `encrypted` array in the [JSON schema](../../rx-schema.md). Only those fields are encrypted on disk. The primary key and any field you want to query stay readable.
```ts
await db.addCollections({
users: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
// the primary key must have a maxLength
id: { type: 'string', maxLength: 100 },
email: { type: 'string' },
// this field is stored as ciphertext on disk
secret: { type: 'string' }
},
required: ['id', 'email'],
encrypted: ['secret']
}
}
});
```
### 3. Read and Write Without Touching the Cipher
Encryption and decryption happen inside RxDB. You insert and query documents like normal, and the `secret` field is plain text in your code and ciphertext on disk.
```ts
await db.users.insert({
id: 'user1',
email: 'alice@example.com',
secret: 'my private token'
});
// query by a non-encrypted field
const doc = await db.users.findOne({
selector: { email: 'alice@example.com' }
}).exec(true);
console.log(doc.secret); // 'my private token' - decrypted for you
```
Keep in mind that encrypted fields cannot be used inside a query selector, because on disk they are ciphertext. You query by the primary key or by non-encrypted fields, and you keep the sensitive data in the encrypted fields. If you need to query encrypted values, you can replicate them into a non-encrypted [memory mapped storage](../../rx-storage-memory-mapped.md) and query that.
## The Performance Cost of Encryption
Encryption is not free. Every write has to encrypt the flagged fields before they reach the store, and every read has to decrypt them again. That is extra CPU work on top of the storage access, and [IndexedDB is already slow](../../slow-indexeddb.md), so the cipher adds to a cost that is high to begin with. On the `crypto-js` plugin the WebCrypto based plugin is about 5x faster, and document inserts are about 10x faster, which tells you how much the cipher itself can weigh.
The overhead comes from a few places you should keep in mind.
- **Encryption runs on the main thread by default.** The cipher is CPU-bound, so encrypting many documents on the main thread can block rendering and make the UI stutter. To fix this, move the storage into a [Worker](../../rx-storage-worker.md) or [SharedWorker](../../rx-storage-shared-worker.md) so encryption runs off the main thread.
- **Encrypted fields cannot use an index.** On disk an encrypted field is a random ciphertext string, so the browser cannot build a useful index over it and a query cannot filter on it. Anything you want to query has to stay unencrypted, and over-encrypting fields quietly pushes those queries into full scans.
- **A whole field is re-encrypted on every write.** RxDB encrypts each flagged field as one string, and there is no partial update inside it. If you keep a large object or a long text in an encrypted field, the full value is re-encrypted on every single write, even when you only changed one small property.
- **The build gets bigger with `crypto-js`.** The free plugin bundles the crypto-js module, which adds to your JavaScript bundle. The premium `encryption-web-crypto` plugin uses the browser's native API instead, so it ships less code.
You keep the cost low with a few habits. Encrypt only the fields that hold sensitive data, not the whole document. Keep encrypted fields small and store big encrypted blobs as [attachments](../../rx-attachment.md), because attachments are only decrypted on an explicit fetch and not while a query runs. And for anything heavy, use the WebCrypto plugin inside a worker.
The chart below shows the difference between the two plugins (lower is better). You can reproduce these numbers yourself in the RxDB repository.
## Choosing an Encryption Plugin
RxDB ships two encryption plugins, and both wrap the storage the same way.
- **`encryption-crypto-js`**: the free plugin, based on the `AES` algorithm of the [crypto-js](https://www.npmjs.com/package/crypto-js) library. It works everywhere and is a good default.
- **`encryption-web-crypto`**: a [π premium](/premium/) plugin built on the native [Web Crypto API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API). Document inserts are about 10x faster than with crypto-js, and the build size is smaller because it uses the browser API instead of bundling a module.
If encryption runs on many documents, the Web Crypto plugin is worth it. IndexedDB is already [slow](../../slow-indexeddb.md), so you do not want a slow cipher on top. For heavy workloads you can also move encryption into a [Worker storage](../../rx-storage-worker.md) so it does not block the main thread.
## FAQ
Is IndexedDB encrypted at rest by default?
No. IndexedDB writes your data to a file on the user's disk in plain text. Same-origin isolation stops other websites from reading it, but it does nothing against someone with file-level access to the machine. To protect data at rest you have to encrypt the values before they are written, which an encrypted [RxStorage](../../rx-storage.md) wrapper does for you.
How do I encrypt data in IndexedDB?
You have two options. Encrypt each value yourself with the [Web Crypto API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API) before every write and decrypt after every read, which breaks querying and is easy to get wrong. Or use **[RxDB](https://rxdb.info/)** on top of IndexedDB and flag the secret fields with `encrypted` in the schema, so encryption and decryption happen transparently. See [How RxDB Adds Encryption](#how-rxdb-adds-encryption-on-top-of-indexeddb).
Can I query encrypted IndexedDB fields?
No. An encrypted field is stored as ciphertext, so an index over it is meaningless and a selector on it cannot match. You query by the primary key or by non-encrypted fields and keep the sensitive data in the encrypted fields. To query encrypted values you can replicate them into a non-encrypted [memory mapped storage](../../rx-storage-memory-mapped.md) first.
Does encryption slow down IndexedDB?
Yes, a bit, because every write encrypts and every read decrypts. The cost depends on the plugin. The free `crypto-js` plugin is fine for small data, and the premium `encryption-web-crypto` plugin is about 10x faster on inserts. For heavy workloads you can run encryption inside a [Worker storage](../../rx-storage-worker.md) to keep it off the main thread.
Can I change the encryption password later?
Not directly. The password is set once per database, and opening the database with a different password throws an error. To rotate it you migrate the data into a new database with the [storage migration plugin](../../migration-storage.md), or you store a random meta-password encrypted with the user password and only change that outer layer. See the [encryption docs](../../encryption.md#changing-the-password).
## Follow Up
- Read the full [RxDB encryption plugin docs](../../encryption.md).
- Start building with the [RxDB Quickstart](../../quickstart.md).
- Learn the raw API in the [IndexedDB Tutorial](./indexeddb-tutorial.md).
- See why the native API is slow in [Slow IndexedDB](../../slow-indexeddb.md).
- Compare the storages in [IndexedDB Alternative](../indexeddb-alternative.md).
- Check the code on [GitHub](/code/) and leave a star β if RxDB helps you.
- Ask questions in the [community chat](/chat/).
---
## IndexedDB Sync - Replicate Browser Data Across Tabs, Devices and Servers
import {Faq, FaqItem} from '@site/src/components/faq';
# IndexedDB Sync
[IndexedDB](../../rx-storage-indexeddb.md) stores structured data inside a single browser, on a single device, in a single origin. That is the whole design. It has no concept of syncing that data to another tab, another device, or a backend server. As soon as your app needs the same data in more than one place, you have to build **IndexedDB sync** yourself, or use a library that ships it.
This page explains what IndexedDB sync means, why the native API gives you nothing for it, the different levels of sync you might need, and how [RxDB](https://rxdb.info/) adds realtime replication on top of IndexedDB.
## What IndexedDB Sync Means
"Sync" is used for three different problems, and it helps to keep them apart:
- **Multi-tab sync**: Two browser tabs of the same origin each open the same IndexedDB database. A write in one tab must become visible in the other tab.
- **Client-server sync**: The browser holds a local copy in IndexedDB and a backend server holds the source of truth. Changes flow both ways so the client works [offline](../../offline-first.md) and catches up when it reconnects.
- **Peer-to-peer sync**: Two or more clients exchange changes directly, without a central server, over [WebRTC](../../replication-webrtc.md) or a relay.
Native IndexedDB solves none of these. It only stores and reads data in one place.
## Why Raw IndexedDB Has No Sync
IndexedDB was designed as a low-level storage building block, not a database engine. It gives you object stores, indexes, and transactions. It does not give you:
- **Change events across tabs**: There is no built-in event when another tab writes to the store. You have to broadcast changes yourself.
- **A network layer**: IndexedDB never talks to a server. It has no push, no pull, no protocol.
- **Change tracking**: There is no log of what changed since the last sync. To replicate you need to know which documents are new or updated, and raw IndexedDB does not record that.
- **Conflict handling**: When the same document is edited in two places while offline, something has to decide the winner. IndexedDB has no notion of document revisions.
So syncing IndexedDB is not a small helper on top of the API. You end up rebuilding change feeds, a revision system, and a replication protocol. That is a database.
## Multi-Tab Sync
The first level of sync happens inside one browser. When a user opens your app in two tabs, both tabs read and write the same IndexedDB database, and a write in one tab should update the UI in the other.
The browser primitive for this is the `BroadcastChannel` API, which sends messages between tabs of the same origin. You can send a message on every write and have other tabs re-read the changed data. Doing this by hand is error-prone, because you also have to avoid running the same background work in every tab at once.
RxDB handles this out of the box. With `multiInstance: true`, writes in one tab are visible to [reactive queries](../../reactivity.md) in every other tab, change events propagate over a `BroadcastChannel`, and [leader election](../../leader-election.md) picks a single tab to run the server replication so you do not open one connection per tab.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageIndexedDB(),
// Coordinate the same database across all tabs of this origin.
multiInstance: true
});
```
## Client-Server Sync
The second level is syncing the browser's IndexedDB copy with a backend. This is what most people mean by "IndexedDB sync". The client keeps working on the local database, and a replication process moves changes to and from the server.
To do this correctly you need three things that raw IndexedDB lacks:
1. **A checkpoint**: a marker of the last successfully synced state, so a reconnect only sends what changed since then instead of everything.
2. **Change detection**: a way to list documents modified since that checkpoint. RxDB stores an internal `_meta` field and revision on every document for exactly this.
3. **Conflict handling**: a rule for when the same document was changed on both sides. RxDB uses per-document revisions and a [conflict handler](../../transactions-conflicts-revisions.md) you can customize.
RxDB packages this into its [Sync Engine](../../replication.md). The backend does not have to run RxDB. You can replicate against any infrastructure through the general [replication protocol](../../replication.md) or one of the ready-made plugins:
- [GraphQL replication](../../replication-graphql.md) against a GraphQL endpoint
- [HTTP replication](../../replication-http.md) against a plain REST server on top of PostgreSQL or MongoDB
- [CouchDB](../../replication-couchdb.md), [Firestore](../../replication-firestore.md), [Supabase](../../replication-supabase.md), [NATS](../../replication-nats.md), [WebSocket](../../replication-websocket.md), and more
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = replicateRxCollection({
collection: db.todos,
replicationIdentifier: 'my-todos-http-replication',
pull: {
async handler(checkpointOrNull, batchSize) {
// Ask the server for documents changed since the last checkpoint.
const response = await fetch(`/api/pull?since=${/* checkpoint */ ''}`);
const data = await response.json();
return { documents: data.documents, checkpoint: data.checkpoint };
}
},
push: {
async handler(changeRows) {
// Send local writes to the server and return conflicts, if any.
const response = await fetch('/api/push', {
method: 'POST',
body: JSON.stringify(changeRows)
});
return response.json();
}
}
});
```
Because the replication runs on top of the [local database](../local-database.md), reads and writes stay [zero-latency](../zero-latency-local-first.md). The user never waits for the network. The sync happens in the background and continues where it left off after the client goes offline and back online.
## Peer-to-Peer Sync
The third level skips the central server. Clients exchange changes directly with each other. RxDB supports this with [WebRTC replication](../../replication-webrtc.md), where peers connect through a signaling server and then sync documents directly. This suits collaborative apps where a backend is optional or where devices on the same network should sync without the cloud.
## Sync Approaches Compared
There are a few ways to get sync onto IndexedDB. They differ in how much they hand you.
- **Do it yourself**: Use raw IndexedDB or a thin wrapper like [Dexie.js](./best-indexeddb-wrapper.md) and write the change tracking, checkpoint, and protocol by hand. Full control, but you are building and maintaining a replication engine.
- **A syncing document store**: [PouchDB](../alternatives/pouchdb-alternative.md) syncs IndexedDB with CouchDB. It works well but ties you to the CouchDB protocol and its revision-tree overhead grows the on-disk size.
- **RxDB**: A local-first database that uses IndexedDB (or faster storages) and ships multi-tab, client-server, and peer-to-peer sync with pluggable backends.
## Feature Comparison
| Sync capability | Raw IndexedDB | Dexie.js | PouchDB | RxDB |
| --- | --- | --- | --- | --- |
| Multi-tab change events | β | β οΈ manual | β οΈ manual | β built in |
| Leader election across tabs | β | β | β | β built in |
| Client-server replication | β | β οΈ paid add-on | β CouchDB only | β many backends |
| Offline then catch up | β | β | β | β |
| Change tracking / checkpoints | β | β | β | β |
| Conflict handling | β | β | β revision tree | β revisions + custom handler |
| Peer-to-peer sync | β | β | β οΈ via CouchDB | β WebRTC |
| Backend requirement | none | none | CouchDB | any (GraphQL, HTTP, more) |
## FAQ
No. IndexedDB is scoped to one browser on one device and has no network layer. To sync across devices you need a replication process that moves changes through a server or a peer connection. RxDB provides this with its **[Sync Engine](../../replication.md)**.
Use the `BroadcastChannel` API to notify other tabs of writes, or let a database handle it. With RxDB and `multiInstance: true`, writes in one tab reach [reactive queries](../../reactivity.md) in every other tab automatically, and [leader election](../../leader-election.md) keeps a single tab responsible for the server connection.
Yes, when the database tracks changes. The client reads and writes to the local IndexedDB copy while offline, and the replication sends the queued changes once the connection returns. This is the core of the [offline-first](../../offline-first.md) approach that RxDB is built for.
The sync layer needs per-document revisions to detect that both sides changed. RxDB attaches a revision to every document and runs a [conflict handler](../../transactions-conflicts-revisions.md) that you can customize, so you decide whether the local write, the remote write, or a merge wins.
No. RxDB replicates against any infrastructure. There are plugins for [GraphQL](../../replication-graphql.md), plain [HTTP](../../replication-http.md), [CouchDB](../../replication-couchdb.md), [Firestore](../../replication-firestore.md), [Supabase](../../replication-supabase.md), and others, and you can implement the [replication protocol](../../replication.md) against your own server.
## Follow Up
- Read how the [RxDB Sync Engine](../../replication.md) works
- Start with the [RxDB Quickstart](../../quickstart.md)
- Learn about [offline-first](../../offline-first.md) apps and [zero-latency](../zero-latency-local-first.md) interactions
- Compare the [best IndexedDB wrappers](./best-indexeddb-wrapper.md)
- Check the [RxDB code on GitHub](/code/) and leave a star β
---
## IndexedDB Tutorial - How to Use IndexedDB, Its Limits, and RxDB
import {Steps} from '@site/src/components/steps';
# IndexedDB Tutorial
**IndexedDB** is the standard [browser storage](../browser-storage.md) API for storing larger amounts of structured data on the client, including files and blobs. It runs in every modern browser and keeps your data on disk, so it survives page reloads and works offline. This tutorial teaches you how to use IndexedDB from scratch with the raw API, then shows where the native API falls short, and how [RxDB](https://rxdb.info/) adds the missing database layer on top of it.
We start with plain IndexedDB and no libraries, so you understand what the browser gives you on its own. If you already know the API and only want the parts that hurt, jump to [The Limits of IndexedDB](#the-limits-of-indexeddb).
## What is IndexedDB?
**IndexedDB** is a transactional database built into every browser. It stores JavaScript objects on disk, inside the user's browser, and lets you look them up by a primary key or by secondary indexes. It is a [low-level building block](../local-database.md), not a developer-facing database engine, so a minute on the core ideas pays off before you write any code.
### How IndexedDB Was Invented
In the early days of the web, the browser could only keep small strings in cookies. The first attempt at a real client-side database was **Web SQL Database**, a spec that put a full SQL engine (SQLite) into the browser. It was deprecated in 2010, because every browser would have had to ship the exact same SQLite build and there was no independent standard to implement against.
IndexedDB was the answer to that problem. Instead of SQL, it defines a storage engine that each browser can build on its own, and it became a W3C standard in 2015. The goals were clear:
- Store more structured data than cookies or [localStorage](../../rx-storage-localstorage.md) allow.
- Keep the API **asynchronous**, so a large read or write never blocks the UI thread.
- Make every change **transactional**, so a failed write does not leave half-written data behind.
- Support **indexes** for fast lookups by fields other than the primary key.
- Stay **low-level**, so libraries (like RxDB) can build friendlier APIs on top.
The last goal is why raw IndexedDB feels so bare. It was designed as a foundation for libraries, not as the thing you use directly.
### The Core Concepts
- **Database**: a named, versioned container. You open it by name, and the version number controls when its structure is allowed to change.
- **Object store**: the place where records live, similar to a table in SQL or a collection in a document database. One database can hold many object stores.
- **Record**: this is the part that trips people up. A record is one complete JavaScript object, stored under a key. It is not a single value like in localStorage, and it is not a row of separate columns like in SQL. Whatever object you put in is the object you get back. So `{ id: 'todo1', name: 'Learn IndexedDB', category: 'work', done: false }` goes in and comes out as one whole record.
- **Primary key**: the value that uniquely identifies a record inside its store. In the tutorial the key is the `id` field (`keyPath: 'id'`), so every todo needs its own unique `id`.
- **Index**: a secondary lookup path. By default you can only find a record by its primary key. An index lets you also find records by another field, for example every todo where `category` equals `work`, without reading the whole store. You define indexes once, at the moment the store is created, and the browser keeps them up to date on every write.
- **Transaction**: every read and write runs inside a transaction. It groups operations and commits them as one unit, and it commits on its own as soon as the browser is done with it.
### Callbacks and Why They Are Harder Than Promises
IndexedDB is older than Promises, so it reports results through **callbacks**. A callback is a function you hand to the API, and the API calls it back later, once the work is done. IndexedDB gives you two on every request: `onsuccess` when the operation worked and `onerror` when it failed. The result never comes back as a return value, it arrives inside the callback.
A **Promise** represents a value that is not ready yet, and you read it with `await` or `.then()`. The difference in practice:
```js
// callback style: the result lives inside onsuccess
const request = store.get('todo1');
request.onsuccess = () => {
console.log(request.result);
};
// promise style: the result is the return value
const todo = await getTodo('todo1');
console.log(todo);
```
Callbacks are harder to work with for a few concrete reasons:
- **Nesting**: when one step depends on the previous one, callbacks nest inside callbacks, the code drifts to the right, and it gets hard to follow.
- **No await**: you cannot pause on a callback. With Promises you write straight-line code and `await` each step in order.
- **Scattered errors**: every request needs its own `onerror`, instead of one `try/catch` around the whole flow.
- **Hard to compose**: combining several async steps by hand is error-prone, while Promises chain and combine cleanly.
Keep this in mind while reading the tutorial below. Every `onsuccess` you see is a place where a Promise-based database would let you `await` the result instead.
## How to Use IndexedDB - Step by Step
The example below builds a small `todos` store, adds an index, and runs the full create, read, update, and delete cycle. Every record has the shape `{ id, name, category, done }`. You can paste each block into the browser console and follow along.
### Open a Database and Create an Object Store
You open a database by name and version. The `onupgradeneeded` event fires only on the first open or when you raise the version number, and it is the only place where you are allowed to create object stores and indexes.
```js
const request = indexedDB.open('todos-db', 1);
request.onupgradeneeded = (event) => {
const db = event.target.result;
// create the store on first open, keyed by the 'id' field
const store = db.createObjectStore('todos', { keyPath: 'id' });
// add a secondary index so we can query by category later
store.createIndex('category', 'category', { unique: false });
};
let db;
request.onsuccess = (event) => {
db = event.target.result; // the database is ready to use here
};
request.onerror = (event) => {
console.error('Could not open the database', event.target.error);
};
```
### Add a Record
Every write needs a `readwrite` transaction. You open the transaction, get the object store, and call `add`. The transaction commits on its own once all its requests are done.
```js
function addTodo(todo) {
const tx = db.transaction('todos', 'readwrite');
const store = tx.objectStore('todos');
const request = store.add(todo);
request.onsuccess = () => console.log('added key', request.result);
tx.onerror = () => console.error('write failed', tx.error);
}
addTodo({ id: 'todo1', name: 'Learn IndexedDB', category: 'work', done: false });
```
### Read a Record by Key
A `readonly` transaction is enough for reads. `store.get(key)` returns a request, and the result arrives on its `onsuccess` event, never as a return value.
```js
const tx = db.transaction('todos', 'readonly');
const store = tx.objectStore('todos');
const request = store.get('todo1');
request.onsuccess = () => {
console.log(request.result);
// > { id: 'todo1', name: 'Learn IndexedDB', category: 'work', done: false }
};
```
### Query Records With an Index
To fetch many records by a field, you go through the index you created. `IDBKeyRange.only('work')` limits the result to records where `category` equals `work`.
```js
const tx = db.transaction('todos', 'readonly');
const index = tx.objectStore('todos').index('category');
const request = index.getAll(IDBKeyRange.only('work'));
request.onsuccess = () => {
console.log(request.result); // all todos where category === 'work'
};
```
### Update a Record
There is no partial update. You read the record, change it in memory, and write the whole object back with `put`, which overwrites the record that has the same key.
```js
const tx = db.transaction('todos', 'readwrite');
const store = tx.objectStore('todos');
const getRequest = store.get('todo1');
getRequest.onsuccess = () => {
const todo = getRequest.result;
todo.done = true;
store.put(todo);
};
```
### Delete a Record
Deleting is a single call inside a `readwrite` transaction.
```js
const tx = db.transaction('todos', 'readwrite');
tx.objectStore('todos').delete('todo1');
```
That is the whole raw API. Notice that nothing here returns a Promise, so you cannot `await` a read or write, and you have to keep every transaction alive inside its own callback chain.
## The Limits of IndexedDB
The tutorial above works, but as soon as your app grows past a demo, the native API starts to bite back. These are the limits you will run into.
- **Callback-based API**: IndexedDB is built on events, not Promises. You cannot `await` a request, and a transaction auto-commits as soon as control returns to the event loop, so it cannot survive an `await` in the middle. Real code turns into nested `onsuccess` handlers or a pile of manual Promise wrappers.
- **No reactivity**: The native API has no way to tell you when data changes. If you want your UI to update after a write, you have to build your own event bus and call it from every place that touches the store.
- **Limited querying**: You can only query by a key or a single-field index range. Anything like "find all not-done todos in category 'work', sorted by name, skip 20, limit 10" means opening a cursor and iterating and filtering by hand. There is no combined filter, sort, skip, and limit. Booleans and plain objects are also not valid IndexedDB keys, so a `done: true/false` field cannot be indexed directly.
- **No schema or validation**: Object stores are schemaless. You can write any shape into them, which feels convenient until one wrong write puts a malformed record on disk and a later read crashes your app.
- **Slow for many operations**: IndexedDB is not fast, and bulk reads and writes have a lot of overhead. See [Slow IndexedDB](../../slow-indexeddb.md) for the numbers and the reasons.
- **Storage limits and eviction**: The browser controls the quota, and it can evict your data under storage pressure. Safari wipes script-writable storage after 7 days of inactivity. See [IndexedDB max storage limit](../indexeddb-max-storage-limit.md).
- **No sync**: IndexedDB stores data in one tab, on one device, in one origin. It has no concept of syncing to another tab, another device, or a backend server. See [IndexedDB sync](./indexeddb-sync.md).
None of this makes IndexedDB a bad storage engine. It is a good place to keep data on disk. The trouble starts when you expect it to behave like a database, because it does not. That is the gap [RxDB](https://rxdb.info/) fills.
## How RxDB Overcomes the Limits of IndexedDB
RxDB (Reactive Database) is a local-first, NoSQL database for JavaScript applications. It runs on top of IndexedDB (and other storages) and gives you the database features the raw API is missing, while your data still lives in IndexedDB under the hood. Switching storages is a configuration change, not a rewrite.
In the browser you have two common ways to store into IndexedDB with RxDB: the free [Dexie.js storage](../../rx-storage-dexie.md), which wraps IndexedDB and is a good default, or the [IndexedDB RxStorage](../../rx-storage-indexeddb.md) with [π premium access](/premium/), which is faster and ships a smaller build. The examples below use the Dexie storage.
### 1. A Promise-Based API With Schemas
RxDB gives you a clean async API and validates every write against a [JSON schema](../../rx-schema.md), so bad data never reaches disk. Compare this to the callback code from the tutorial above.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
const db = await createRxDatabase({
name: 'todos-db',
storage: getRxStorageDexie() // stores into IndexedDB under the hood
});
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
// the primary key must have a maxLength
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
category: { type: 'string' },
done: { type: 'boolean' }
},
required: ['id', 'name', 'category', 'done']
}
}
});
// a single awaitable line instead of an onupgradeneeded/onsuccess chain
const doc = await db.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
category: 'work',
done: false
});
```
### 2. Reactive Queries
Every RxDB query is observable. You subscribe once, and it emits a new result whenever a matching document changes, even when the change happens in another part of your app or another browser tab. This is the reactivity the native API leaves you to build yourself.
```ts
const observable = db.todos.find({
selector: { done: { $eq: false } }
}).$; // get the observable via RxQuery.$
observable.subscribe(openTodos => {
console.log('Currently have ' + openTodos.length + ' things to do');
// -> re-render your UI here with the updated list
});
```
### 3. MongoDB-Style Queries
Instead of opening a cursor and filtering by hand, you write [MongoDB-style (Mango) queries](../../rx-query.md) that combine filter, sort, skip, and limit in one object. And a boolean field like `done` is queryable, which a raw IndexedDB index cannot do.
```ts
const results = await db.todos.find({
selector: {
done: { $eq: false },
category: { $eq: 'work' }
},
sort: [{ name: 'asc' }],
skip: 0,
limit: 10
}).exec();
```
### 4. Sync Across Tabs, Devices, and Servers
RxDB ships a [Sync Engine](../../replication.md) that replicates your IndexedDB data to a backend over HTTP, WebSocket, or GraphQL, with conflict handling and offline queueing built in. Writes made offline sync later, and changes flow back to every connected client in realtime. Raw IndexedDB gives you none of this. See [IndexedDB sync](./indexeddb-sync.md) for the full picture.
### 5. Better Performance and Managed Storage
RxDB uses batched cursors and other techniques to work around the slow parts of IndexedDB, and it helps you handle quota and eviction instead of letting the browser surprise you. For the details, read [Slow IndexedDB](../../slow-indexeddb.md) and [IndexedDB max storage limit](../indexeddb-max-storage-limit.md).
## FAQ
How do I use IndexedDB in JavaScript?
You open a database with `indexedDB.open(name, version)`, create an **object store** inside the `onupgradeneeded` event, and then read and write records inside `readwrite` or `readonly` transactions. Every operation returns a request whose result arrives on an `onsuccess` callback. The [step-by-step tutorial above](#how-to-use-indexeddb---step-by-step) walks through the full create, read, update, and delete cycle.
What is a record in IndexedDB?
A record is one complete JavaScript object stored inside an **object store** under a key. It is not a single value like a localStorage string, and it is not a row of separate columns like in SQL. You put an object in and you get the same object back. See [The Core Concepts](#the-core-concepts) above.
What is an index in IndexedDB?
An index is a secondary lookup path. Without it you can only find a record by its primary key. An index lets you find records by another field, for example every todo where `category` equals `work`, without scanning the whole store. You create indexes once, inside `onupgradeneeded`, and the browser keeps them current on every write.
What is the difference between a callback and a promise?
A callback is a function you pass to an API that gets called later with the result, like IndexedDB's `onsuccess`. A Promise is a value you read with `await` or `.then()`. Promises let you write straight-line code with one `try/catch`, while callbacks nest and need error handling on every step. This is a large part of why raw IndexedDB feels harder than a modern database.
Is IndexedDB better than localStorage?
Yes, for structured data. [localStorage](../../rx-storage-localstorage.md) only stores strings, is synchronous, and is capped at a few megabytes. IndexedDB stores objects and blobs, works asynchronously, and holds much more data. For simple key-value flags, localStorage is fine.
Why is IndexedDB so hard to use?
The API predates Promises and is built around events and callbacks, so control flow is verbose and a transaction cannot survive an `await`. It also has no reactivity, no schema, and only single-field range queries. Most teams add a wrapper. See [the best IndexedDB wrapper](./best-indexeddb-wrapper.md) for a comparison.
Can IndexedDB sync data to a server or other devices?
No. IndexedDB stores data in one browser, on one device, in one origin, and has no built-in sync. You have to build replication yourself or use a library like **[RxDB](https://rxdb.info/)** that ships a [Sync Engine](../../replication.md). See [IndexedDB sync](./indexeddb-sync.md).
Does RxDB use IndexedDB?
Yes. **[RxDB](https://rxdb.info/)** can store its data in IndexedDB through the free [Dexie.js storage](../../rx-storage-dexie.md) or the premium [IndexedDB RxStorage](../../rx-storage-indexeddb.md). You keep IndexedDB as the storage and get a real database API, reactivity, and sync on top.
## Follow Up
- Start building with the [RxDB Quickstart](../../quickstart.md).
- Compare wrappers in [Best IndexedDB Wrapper](./best-indexeddb-wrapper.md).
- Read why the native API is slow in [Slow IndexedDB](../../slow-indexeddb.md).
- Learn about replication in [IndexedDB Sync](./indexeddb-sync.md).
- See the full picture in [IndexedDB Alternative](../indexeddb-alternative.md).
- Check the code on [GitHub](/code/) and leave a star β if RxDB helps you.
- Ask questions in the [community chat](/chat/).
---
## IndexedDB with TypeScript - Type-Safe Browser Storage
import {Faq, FaqItem} from '@site/src/components/faq';
# IndexedDB with TypeScript
[IndexedDB](../../rx-storage-indexeddb.md) is the standard [browser storage](../browser-storage.md) API for structured data. It works in every modern browser and can hold large amounts of JSON and binary data. But its TypeScript story is weak. The native API returns loosely typed values, so most reads come back as `any` and you lose the type safety that made you pick TypeScript in the first place.
This page explains where the native IndexedDB types fall short, how to add types by hand, and how libraries like [idb](https://github.com/jakearchibald/idb) and [RxDB](https://rxdb.info/) give you type-safe access, with RxDB deriving the types straight from your schema.
## The Problem with Native IndexedDB Types
The DOM type definitions ship with `lib.dom.d.ts`, so `IDBDatabase`, `IDBObjectStore`, and `IDBRequest` are typed. The trouble is what those types say. A read gives you back an `IDBRequest`, and its `result` property is typed as `any`.
```ts
const request = store.get('user-1');
request.onsuccess = () => {
// request.result is `any`. No autocomplete, no checking.
const user = request.result;
console.log(user.naem); // typo compiles fine, breaks at runtime
};
```
There are three problems here:
- **`result` is `any`**: Every value you read is untyped, so a typo in a field name compiles and fails only at runtime.
- **No schema link**: TypeScript does not know which object stores exist or what shape their records have. You get no error when you read from a store that does not exist.
- **Manual casts everywhere**: To get any safety back you cast every read (`request.result as User`), which is a promise you make to the compiler, not a check it performs.
TypeScript cannot infer types across the IndexedDB boundary. You have to add them.
## Typing the Native API by Hand
You can layer your own types on top with generics and casts. It removes some of the pain but not all of it.
```ts
type User = {
id: string;
name: string;
age: number;
};
function getUser(db: IDBDatabase, id: string): Promise {
return new Promise((resolve, reject) => {
const tx = db.transaction('users', 'readonly');
const request = tx.objectStore('users').get(id);
// The cast is a claim, not a guarantee.
request.onsuccess = () => resolve(request.result as User);
request.onerror = () => reject(request.error);
});
}
```
The value is still whatever IndexedDB stored. The `as User` cast tells the compiler to trust you. If the stored record does not match, TypeScript stays silent and the bug surfaces later. For real safety you also need runtime validation, which raw IndexedDB does not provide.
## Type-Safe Wrappers
Libraries close the gap in different ways. See the [best IndexedDB wrapper](./best-indexeddb-wrapper.md) comparison for the full feature picture. Here we look only at the typing.
### idb
The [idb](https://github.com/jakearchibald/idb) library accepts a `DBSchema` type parameter. You describe your stores once, and every read and write is typed against it.
```ts
import { openDB, DBSchema } from 'idb';
interface MyDB extends DBSchema {
users: {
key: string;
value: { id: string; name: string; age: number };
indexes: { 'by-age': number };
};
}
const db = await openDB('mydb', 1, {
upgrade(db) {
const store = db.createObjectStore('users', { keyPath: 'id' });
store.createIndex('by-age', 'age');
}
});
const user = await db.get('users', 'user-1');
// user is typed as { id: string; name: string; age: number } | undefined
```
This is a big step up. Reads are typed, store names are checked, and index names are validated. The type is still a claim about what is stored, not a runtime check, so a corrupted record slips through, but the developer experience is much better than the raw API.
### RxDB
[RxDB](https://rxdb.info/) is a local-first database that runs on IndexedDB (or faster storages) and takes a different approach. You define a [JSON Schema](../../rx-schema.md) once, and RxDB derives the TypeScript type from it. There is a single source of truth, so the type and the runtime validation cannot drift apart.
```ts
import {
toTypedRxJsonSchema,
ExtractDocumentTypeFromTypedRxJsonSchema,
RxJsonSchema
} from 'rxdb';
const userSchemaLiteral = {
title: 'user schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
age: { type: 'integer' }
},
required: ['id', 'name', 'age'],
indexes: ['age']
} as const; // <- 'as const' preserves the literal type
const schemaTyped = toTypedRxJsonSchema(userSchemaLiteral);
// The document type is derived from the schema, not written twice.
export type UserDocType =
ExtractDocumentTypeFromTypedRxJsonSchema;
export const userSchema: RxJsonSchema = userSchemaLiteral;
```
Once the collection is created, queries and documents are fully typed, and every write is validated against the schema at runtime:
```ts
const adults = await db.users.find({
selector: { age: { $gt: 17 } }
}).exec();
// adults is typed as RxDocument[]
const first = adults[0];
first.name; // string, checked
first.get('naem'); // TypeScript error, unknown field
```
The difference is that the type and the validation come from the same schema. With the wrappers above you write the type once and hope the stored data matches. With RxDB the schema drives both the compile-time type and the runtime [schema validation](../../schema-validation.md), so a document that does not fit is rejected on write. See the full [TypeScript tutorial](../../tutorials/typescript.md) for typing collections, ORM methods, and documents.
## How the Options Compare
| Typing aspect | Native IndexedDB | idb | RxDB |
| --- | --- | --- | --- |
| Reads typed | β `any` | β via `DBSchema` | β from schema |
| Store/collection names checked | β | β | β |
| Query results typed | β | β οΈ key ranges only | β |
| Single source of truth | β | β type only | β schema |
| Runtime validation | β | β | β schema validation |
| Type from schema | β | β manual | β derived |
## FAQ
Yes, but they are weak. The DOM library types `IDBDatabase` and friends, yet reads return an `IDBRequest` whose `result` is `any`. So you get no type safety on the data itself. A wrapper like **[idb](https://github.com/jakearchibald/idb)** or **[RxDB](../../rx-schema.md)** is needed for typed reads.
Use a library that carries types through the query. RxDB types query results as `RxDocument` where `T` is derived from your [schema](../../rx-schema.md), so filters and result arrays stay typed. The raw API only supports key-range lookups and returns `any`.
A typed wrapper checks your code at compile time. It does not check the data at runtime, so a record that does not match the type still loads. RxDB adds runtime [schema validation](../../schema-validation.md) from the same schema that produces the type, so invalid documents are rejected on write.
Yes. RxDB derives the document type from its schema with `ExtractDocumentTypeFromTypedRxJsonSchema`. If your schema lives in a `.json` file, you can also generate types at build time with a tool like [json-schema-to-typescript](https://www.npmjs.com/package/json-schema-to-typescript).
## Follow Up
- Read the full [RxDB TypeScript tutorial](../../tutorials/typescript.md)
- Learn about the [RxDB schema](../../rx-schema.md) and [schema validation](../../schema-validation.md)
- Compare the [best IndexedDB wrappers](./best-indexeddb-wrapper.md)
- Start with the [RxDB Quickstart](../../quickstart.md)
- Check the [RxDB code on GitHub](/code/) and leave a star β
---
## IndexedDB Alternative - Why RxDB is the Better Choice
import {ComparisonTable} from '@site/src/components/comparison-table';
# IndexedDB Alternatives
IndexedDB is the standard [browser storage](../articles/browser-storage.md) API for storing significant amounts of structured data, including files/blobs. It is available in every modern browser.
However, using the native IndexedDB API is **verbose**, **low-level**, and lacks many features modern applications need.
If you are looking for an **IndexedDB alternative**, you likely want a library that abstracts the complexity away and provides features like **Reactivity**, **[Schema Validation](../schema-validation.md)**, and **Sync**.
RxDB is the ultimate alternative because it gives you the speed of a [local database](../articles/local-database.md) with the ease of use of a modern [JSON-document store](../rx-collection.md).
## The Problem with Raw IndexedDB
IndexedDB was designed as a low-level building block, not a developer-facing database engine.
Because of that, relying on raw IndexedDB (or thin wrappers) often leads to significant friction:
1. **Callback Hell**: The API heavily relies on event handlers (`onsuccess`, `onerror`), making control flow difficult to read and maintain.
2. **Missing Observability**: Standard IndexedDB provides no way to listen to data changes. You have to build your own event bus to update the UI when data changes.
3. **Complex Transaction Management**: You must explicitly create transactions for every read or write, which is repetitive and error-prone.
4. **No Schema Enforcement**: IndexedDB is schema-less. You can store anything, which sounds good until your app crashes because of inconsistent data structures.
5. **Limited Querying**: You can only query by simple key ranges. Complex queries (like "find users older than 18 and sort by name") require manually iterating over cursors, which is slow and code-heavy. See [Slow IndexedDB](../slow-indexeddb.md).
RxDB solves all of these problems while maintaining the benefits of a local database.
## Why RxDB is the Best Alternative
RxDB is a NoSQL database for JavaScript applications. It uses [IndexedDB](../rx-storage-indexeddb.md) (or faster alternatives) under the hood but provides a rich, feature-complete API on top.
### 1. Developer Experience
RxDB offers a promise-based API that feels intuitive for JavaScript developers. It uses [JSON Schema](../rx-schema.md) to define your data structure, ensuring you never store invalid data.
**Raw IndexedDB:**
```js
const request = indexedDB.open('myDatabase', 1);
request.onupgradeneeded = (event) => { /* Handle versions */ };
request.onsuccess = (event) => {
const db = event.target.result;
const transaction = db.transaction(['users'], 'readonly');
const store = transaction.objectStore('users');
const getRequest = store.get('user1');
getRequest.onsuccess = () => {
console.log(getRequest.result); // Finally got the data
};
};
```
**RxDB:**
```js
const db = await createRxDatabase({
name: 'myDatabase',
storage: getRxStorageDexie() // Uses IndexedDB under the hood
});
// Define collection once
await db.addCollections({
users: { schema: myJsonSchema }
});
// Query data
const user = await db.users.findOne('user1').exec();
```
### 2. Reactivity (The "Rx" in RxDB)
Modern UIs ([React](../articles/react-database.md), [Vue](../articles/vue-database.md), [Angular](../articles/angular-database.md), Svelte) need to be reactive. When data changes, the view should update.
RxDB is built on **[RxJS](https://rxjs.dev/)**. Every query, document, or field can be observed.
```js
// Subscribe to a query -> UI updates automatically on change
db.users.find({
selector: { age: { $gt: 18 } }
}).$.subscribe(users => {
updateUI(users);
});
```
This works even across multiple browser tabs. If a user changes data in Tab A, Tab B updates instantly. Implementing this with raw IndexedDB and `BroadcastChannel` manually is a massive undertaking. See [Reactivity](../reactivity.md).
### 3. Advanced Query Engine
Searching for data in raw IndexedDB requires opening cursors and iterating over records manually, which is slow and complex.
RxDB includes [Mango Query](../rx-query.md) syntax (like MongoDB). You can filter, sort, and limit data with a simple JSON object.
```js
const results = await db.users.find({
selector: {
age: { $gt: 18 },
role: { $in: ['admin', 'moderator'] }
},
sort: [{ name: 'asc' }],
limit: 10
}).exec();
```
### 4. Synchronization
Raw IndexedDB is purely local. Usage in real-world apps usually requires syncing data with a backend.
Building a robust sync protocol (handling [offline](../offline-first.md) changes, [conflict resolution](../transactions-conflicts-revisions.md), delta updates) is one of the hardest problems in software engineering.
RxDB solves this out of the box. It has a robust [replication protocol](../replication.md) that supports:
- **[Real-time sync](../articles/realtime-database.md)**: Changes are pushed/pulled immediately.
- **Conflict Resolution**: Strategies to handle concurrent edits.
- **Multi-Backend**: Plugins for [CouchDB](../replication-couchdb.md), [GraphQL](../replication-graphql.md), [Firestore](../replication-firestore.md), [Supabase](../replication-supabase.md), and generic [HTTP](../replication-http.md).
### 5. Performance & Storage Engines
While IndexedDB is fast enough for simple tasks, it can be the bottleneck for high-performance apps due to serialization overhead and browser implementation details. See [RxStorage Performance](../rx-storage-performance.md).
RxDB abstracts the storage layer. You can start with IndexedDB and switch to unparalleled performance engines later without changing your application code:
- **[Dexie.js RxStorage](../rx-storage-dexie.md)**: A optimized wrapper around IndexedDB.
- **[OPFS RxStorage](../rx-storage-opfs.md)**: Uses the Origin Private File System, which is significantly faster than IDB for many operations.
- **[SQLite RxStorage](../rx-storage-sqlite.md)**: Uses SQLite via WebAssembly (or Native in [React Native](../react-native-database.md)/[Electron](../electron-database.md)) for robust SQL-based storage.
- **[Memory RxStorage](../rx-storage-memory.md)**: For ephemeral data or testing.
### 6. TypeScript Support
IndexedDB API is loosely typed. You often cast `any` or struggle with correct event types.
RxDB is written in TypeScript and provides first-class type safety. Your database schema generates TypeScript types, so you get autocomplete for every field in your documents and queries.
```ts
// TypeScript knows that 'age' is a number
const user = await db.users.findOne().exec();
console.log(user.age.toFixed(2));
```
### 7. Encryption & Compression
Storing sensitive data? Raw IndexedDB writes everything to disk in plain text, so RxDB adds [IndexedDB encryption](./indexeddb/indexeddb-encryption.md) with a built-in [encryption plugin](../encryption.md). You provide a password, and the flagged fields are stored encrypted at rest.
Storing lots of data? The [Key-Compression](../key-compression.md) plugin shrinks your JSON keys to minimize storage usage, often reducing database size by 40%+.
## Other Alternatives
There are other ways to store data in the browser, but they all have significant limitations compared to IndexedDB (and RxDB).
### LocalStorage
`localStorage` is a synchronous key-value store. See [Using localStorage](../articles/localstorage.md).
- **Why it fails**: It blocks the main thread (UI freezes on large reads/writes). It is capped at ~5MB. It only supports strings, so you must constantly `JSON.parse` and `JSON.stringify`.
- **Use case**: Simple settings like "dark mode: on".
### Cookies
Cookies are small pieces of data sent with every HTTP request.
- **Why it fails**: Extremely limited size (4KB). Wastes bandwidth by sending data to the server on every request.
- **Use case**: Session tokens, authentication.
### WebSQL
WebSQL was a wrapper around SQLite but is **deprecated** and removed from non-Google browsers.
- **Why it fails**: It is a dead standard. Do not use it.
- **Use case**: Legacy apps only.
### OPFS (Origin Private File System)
OPFS is a new high-performance file system API for the web.
- **Why it fails**: It is a file system, not a database. It has no indexing, no querying, and no document structure. It is extremely low-level.
- **Note**: RxDB *uses* OPFS in its [OPFS RxStorage](../rx-storage-opfs.md) to give you the performance of OPFS with the features of a real database.
## Comparison
| Feature | Raw IndexedDB | **RxDB** |
| :--- | :--- | :--- |
| **Api Style** | Event-based / Callback | Promise / Observable |
| **Reactivity** | β None | β [Observables](../rx-query.md) / [Signals](../reactivity.md) |
| **Sync** | β Manual Implementation | β Built-in & Backend Agnostic |
| **Query Engine** | β Basic Key-Range | β [MongoDB-style (Mango)](../rx-query.md) |
| **Transactions** | β Manual | β Automatic |
| **Schema** | β None | β [JSON Schema](../rx-schema.md) |
| **Migrations** | β οΈ Manual | β [Declarative](../migration-schema.md) |
| **Multi-Tab Sync**| β Manual | β Automatic |
| **Encryption** | β None | β [Built-in](./indexeddb/indexeddb-encryption.md) |
| **TypeScript** | β οΈ Partial | β Full Support |
## Conclusion
If you are building a toy project, `localStorage` or a simple wrappers like `idb-keyval` might suffice.
But if you are building a **production application** that needs to be fast, reliable, and maintainable, relying on raw IndexedDB is a premature optimization that costs you development time.
**RxDB** is the "Battery Included" alternative that handles the hard parts of local data (sync, reactivity, queries) so you can focus on building your product. For further reading, check out [Why Local-First Software Is the Future](../articles/local-first-future.md) or [RxDB as a Database for Browsers](../articles/browser-database.md).
---
## IndexedDB Max Storage Size Limit - Detailed Best Practices
{/*
SEO Keywords:
- "indexeddb storage limit" - 590
- "indexeddb size limit" - 260
- "indexeddb max size" - 590
- "indexeddb limits" - 170
*/}
import {VideoBox} from '@site/src/components/video-box';
import {CenteredImage} from '@site/src/components/centered-image';
# IndexedDB Max Storage Size Limit
IndexedDB is widely known as the primary browser-based storage API for large client-side data, particularly valuable for modern [offline-first](../offline-first.md) applications. These apps aim to keep everything functional and interactive even without an internet connection, which naturally demands substantial local storage. However, IndexedDB has various size limits depending on the browser, disk space, and user settings. Being aware of these constraints is crucial so you can avoid quota errors and deliver a seamless user experience without unexpected data loss.
Offline-first apps have grown in popularity because they provide immediate feedback, zero-latency interactions, and resilience in poor network conditions. Storing big data sets, or even entire data models, in IndexedDB has become far more common than in the era of small localStorage or cookie usage. But all this local data is subject to quotas, and thatβs exactly what this guide will help you understand and manage.
## Why IndexedDB Has a Storage Limit
Browsers need a way to curb runaway disk usage and safeguard user resources. This is accomplished through **quota management** policies, which can vary among Chrome, Firefox, Safari, Edge, and others. Some browsers use a percentage of your total disk space, while others rely on a fixed maximum or dynamic approach per origin. These policies are designed to prevent malicious or poorly optimized web pages from consuming an unreasonable amount of user storage.
Chrome (and Chromium-based browsers) typically allow you to use a percentage of the userβs free disk space, whereas Firefox historically prompts users to allow more than 5 MB in mobile or 50 MB in desktop. Safari often sets tighter maximum caps, especially on iOS devices. Edge aligns closely with Chromeβs rules but can also include enterprise or corporate policy overrides. Understanding these default or dynamic limits prepares you to plan your appβs storage needs appropriately.
## Browser-Specific IndexedDB Limits
IndexedDB size quotas differ significantly across browsers and platforms. While there isnβt a universal rule, the following table summarizes approximate limits and any notes or caveats you should be aware of:
| Browser | Approx. Limit | Notes |
|----------------|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|
| Chrome/Chromium | Up to ~80% of free disk, per origin cap | Often cited as 60 GB on a 100 GB drive. Shared pool approach. Quota usage can prompt partial or extended user approvals. |
| Firefox | ~2 GB (desktop) or ~5 MB initial for mobile | Older versions asked permission at 50 MB for desktop. Ephemeral/incognito sessions may require repeated user prompts. |
| Safari (iOS) | ~1 GB per origin (variable) | Historically stricter. iOS devices limit quotas further. Behavior can differ between iOS Safari versions or iPadOS. |
| Edge | Similar to Chromeβs 80% of free space | Can be influenced by Windows enterprise policies. Generally aligned with Chromium approach. |
| iOS Safari | Typically 1 GB, can be less on older iOS | Early iOS versions were known for more aggressive quotas and data eviction on low space. |
| Android Chrome | Similar to desktop Chrome | May exhibit warnings in especially low-storage devices. The same 80% free space logic generally applies. |
Historically, these limits have evolved. For instance, older Firefox versions included `dom.indexedDB.warningQuota`, showing a 50 MB prompt on desktop or a 5 MB prompt on mobile. Many developers wrote about these notifications on Stack Overflow. Since around 2015, Firefox has changed its quota approach significantly. Likewise, Safari used to limit data more aggressively on older iOS versions. Some older tutorials suggest comparing IndexedDB to localStorage, but modern browsers allow far larger and more flexible storage with IndexedDB than the old localStorage or cookie-based setups.
---
## Checking Your Current IndexedDB Usage
To assess where your app stands relative to these storage limits, you can use the **Storage Estimation API**. The snippet below shows how to estimate both your used storage and the total space allocated to your origin:
```js
const quota = await navigator.storage.estimate();
const totalSpace = quota.quota;
const usedSpace = quota.usage;
console.log('Approx total allocated space:', totalSpace);
console.log('Approx used space:', usedSpace);
```
[Some browsers (all modern ones)](https://developer.mozilla.org/en-US/docs/Web/API/StorageManager/persist#browser_compatibility) also provide a `navigator.storage.persist()` method to request persistent storage, preventing the browser from automatically clearing your data if the userβs device runs low on space. Note that users might deny such requests, or the request might fail silently on stricter environments. Always handle these outcomes gracefully and design your app to degrade if persistent storage is unavailable.
## Testing Your Appβs IndexedDB Quotas
The best way to handle real-world usage is to test for low storage conditions and large data sets in different environments. You can fill up the space manually by writing repetitive test data or running scripts that bulk-insert documents until an error occurs.
Real-time usage monitors or dashboards can keep track of your `navigator.storage.estimate()` results, letting you see how close you are to the max limit in production. Developer tools in Chrome or Firefox can simulate limited storage situations, which is crucial for QA:
This short tutorial shows how you can artificially reduce available storage in Google Chromeβs dev tools to see how your app behaves when nearing or exceeding the quota.
## Handling Errors When Limits Are Reached
When the userβs device is too full or your app exceeds the allotted quota, most browsers will throw a **QuotaExceededError** (or similarly named exception) when trying to store additional data. Often, the request to IndexedDB simply fails with an error event. Handling this gracefully is essential to avoid crashes or data corruption.
A typical approach is to wrap your write operations in try/catch blocks or in `onsuccess` / `onerror` event callbacks. If you detect a quota error, you can prompt the user to clear out old items or reduce the scope of offline data. Some apps implement a fallback system that removes less critical documents to free space and then retries the write.
```js
try {
const tx = db.transaction('largeStore', 'readwrite');
const store = tx.objectStore('largeStore');
await store.add(hugeData, someKey);
await tx.done;
} catch (error) {
if (error.name === 'QuotaExceededError') {
console.warn('IndexedDB quota exceeded. Cleanup or prompt user to free space.');
// Optionally remove older data or show a UI hint:
// removeOldDocuments();
// displayStorageFullDialog();
} else {
// handle other errors
console.error('IndexedDB write error:', error);
}
}
```
## Tricks to Exceed the Storage Size Limitation
Even if you plan well, your app might need more storage than a single origin typically allows. There are a few advanced tactics you can use:
If you store binary data such as images or videos, consider compressing them via the Compression Streams API. For textual or [JSON data](./json-based-database.md), a library like [RxDB](/) supports built-in [key-compression](../key-compression.md) to shorten field names or entire documents. This can be extremely helpful when storing large sets of objects:
```ts
// Example: How key-compression can transform your documents internally
const uncompressed = {
"firstName": "Corrine",
"lastName": "Ziemann",
"shoppingCartItems": [
{
"productNumber": 29857,
"amount": 1
},
{
"productNumber": 53409,
"amount": 6
}
]
};
const compressed = {
"|e": "Corrine",
"|g": "Ziemann",
"|i": [
{
"|h": 29857,
"|b": 1
},
{
"|h": 53409,
"|b": 6
}
]
};
```
Sharding data across multiple subdomains or iframes is another trick, though it complicates communication. When you need truly massive offline data, you might store part of the data under `sub1.yoursite.com` and another chunk under `sub2.yoursite.com`, using `postMessage()` to coordinate. This can circumvent single-origin limitations, but it introduces extra complexity. Another effective method is to let data expire automatically, for example, older records can be removed if they havenβt been accessed for a certain period.
## IndexedDB Max Size of a Single Object
There is no explicit cap on how large an individual object or record in IndexedDB can be, other than the overall disk quota. If you attempt to store one extremely large object, you will eventually hit browser memory constraints or the global storage quota. In practice, youβll encounter out-of-memory issues in JavaScript before IndexedDB itself refuses a single large write. A helpful test can be seen in [this JSFiddle experiment](https://jsfiddle.net/sdrqf8om/2/) where you see browsers can crash when creating massive in-memory objects.
## Is There a Time Limit for Data Stored in IndexedDB?
IndexedDB data can remain indefinitely as long as the user does not clear the browserβs data or the origin does not run afoul of automated eviction policies (e.g., Safari or Android might remove large caches for sites unused over a long period when space is needed). Typically, there is no βtime limit,β but ephemeral modes or incognito sessions have their own rules. If you rely on permanent offline data, request persistent storage and handle the possibility that the user or the OS could still remove your data under extreme conditions. Especially Safari is known to be very fast in deleting local data.
## Follow Up
Learn more by checking the [IndexedDB official docs](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API), which detail store design, error handling, and quota usage. If you need a straightforward way to manage large offline data with compression and [conflict resolution](../transactions-conflicts-revisions.md), explore the [RxDB Quickstart](../quickstart.md). You can also join the community on [GitHub](/code/) to share tips on overcoming the **IndexedDB max storage size limit** in production environments.
---
## RxDB - The Perfect Ionic Database
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# Ionic Storage - RxDB as database for hybrid apps
In the fast-paced world of mobile app development, **hybrid applications** have emerged as a versatile solution, offering the best of both worlds - the web and native app experiences. One key challenge these apps face is efficiently storing and querying data on the **client's device**. Enter [RxDB](https://rxdb.info/), a powerful client-side database tailored for ionic hybrid applications. In this article, we'll explore how RxDB addresses the requirements of storing and querying data in ionic apps, and why it stands out as a preferred choice.
## What are Ionic Hybrid Apps?
Ionic (aka Ionic 2 ) hybrid apps combine the strengths of web technologies (HTML, CSS, JavaScript) with native app development to deliver cross-platform applications. They are built using web technologies and then wrapped in a native container to be deployed on various platforms like iOS, Android, and the web. These apps provide a consistent user experience across devices while benefiting from the efficiency and familiarity of web development.
## Storing and Querying Data in an Ionic App
Storing and querying data is a fundamental aspect of any application, including hybrid apps. These apps often need to operate offline, store user-generated content, and provide responsive user interfaces. Therefore, having a reliable and efficient way to manage data on the client's device is crucial.
## Introducing RxDB as a Client-Side Database for Ionic Apps
RxDB steps in as a powerful solution to address the data management needs of ionic hybrid apps. It's a NoSQL client-side database that offers exceptional performance and features tailored to the unique requirements of client-side applications. Let's delve into the key features of RxDB that make it a great fit for these apps.
### Getting Started with RxDB
### What is RxDB?
At its core, [RxDB](https://rxdb.info/) is a **NoSQL** database that operates with a [local-first](../offline-first.md) approach. This means that your app's data is stored and processed primarily on the client's device, reducing the dependency on constant network connectivity. By doing so, RxDB ensures your app remains responsive and functional, even when offline.
### Local-First Approach
The [local-first](../offline-first.md) approach adopted by RxDB is a game-changer for hybrid applications. Storing data locally allows your app to function seamlessly without an internet connection, providing users with uninterrupted access to their data. When connectivity is restored, RxDB handles the synchronization of data, ensuring that any changes made offline are appropriately propagated.
### Observable Queries
One of RxDB's standout features is its implementation of [observable queries](../rx-query.md). This concept allows your app's user interface to be dynamically updated in real time as data changes within the database. RxDB's observables create a bridge between your database and user interface, keeping them in sync effortlessly.
### NoSQL Query Engine
RxDB's NoSQL query engine empowers you to perform powerful queries on your app's data, without the constraints imposed by traditional relational databases. This flexibility is particularly valuable when dealing with unstructured or semi-structured data. With the NoSQL query engine, you can retrieve, filter, and manipulate data according to your app's unique requirements.
```ts
const foundDocuments = await myDatabase.todos.find({
selector: {
done: {
$eq: false
}
}
}).exec();
```
### Great Observe Performance with EventReduce
RxDB introduces a concept called [EventReduce](https://github.com/pubkey/event-reduce), which optimizes the observation process. Instead of overwhelming your app's UI with every data change, EventReduce filters and batches these changes to provide a smooth and efficient experience. This leads to enhanced app performance, lower resource usage, and ultimately, happier users.
## Why NoSQL is a Better Fit for Client-Side Applications Compared to relational databases like SQLite
When it comes to choosing the right database solution for your client-side applications, NoSQL RxDB presents compelling advantages over traditional options like [SQLite](../rx-storage-sqlite.md). Let's delve into the key reasons why NoSQL RxDB is a superior fit for your ionic hybrid app development.
### Easier Document-Based Replication
NoSQL databases, like RxDB, inherently embrace a document-based approach to [data storage](./ionic-storage.md). This design choice simplifies data [replication](../replication.md) between clients and servers. With documents representing discrete units of data, you can easily synchronize individual pieces of information without the complexity that can arise when dealing with rows and tables in a relational database like SQLite. This document-centric replication model streamlines the synchronization process and ensures that your app's data remains consistent across devices.
### Offline Capable
One of the defining features of client-side applications is the ability to function even when offline. NoSQL RxDB excels in this area by supporting a local-first approach. Data is cached on the client's device, enabling the app to remain fully functional even without an internet connection. As connectivity is restored, RxDB handles data synchronization with the server seamlessly. This offline capability ensures a smooth user experience, critical for ionic hybrid apps catering to users in various network conditions.
### NoSQL Has Better TypeScript Support
TypeScript, a popular superset of JavaScript, is renowned for its static typing and enhanced developer experience. NoSQL databases like RxDB are inherently flexible, making them well-suited for TypeScript integration. With well-defined data structures and clear typings, NoSQL RxDB offers [improved type safety](../tutorials/typescript.md) and easier development when compared to traditional SQL databases like SQLite. This results in reduced debugging time and increased code reliability.
### Easier [Schema Migration](../migration-schema.md) with NoSQL Documents
Schema changes are a common occurrence in application development, and dealing with them can be challenging. NoSQL databases, including RxDB, are more forgiving in this aspect. Since documents in NoSQL databases don't enforce a rigid structure like tables in relational databases, schema changes are often simpler to manage. This flexibility makes it easier to evolve your app's data structure over time without the need for complex migration scripts, a notable advantage when compared to SQLite.
## Great Performance
RxDB's [excellent performance](../rx-storage-performance.md) stems from its advanced indexing capabilities, which streamline data retrieval and ensure swift query execution. Additionally, the [JSON key compression](../key-compression.md) employed by RxDB minimizes storage overhead, enabling efficient data transfer and quicker loading times. The incorporation of real-time updates through change streams and the **EventReduce mechanism** further enhances RxDB's performance, delivering a responsive user experience even as data changes are propagated seamlessly.
## Using RxDB in an Ionic Hybrid App
RxDB's integration into your ionic hybrid app opens up a world of possibilities for efficient data management. Let's explore how to set up RxDB, use it with popular JavaScript frameworks, and take advantage of its diverse storage options.
### Setup RxDB
Getting started with RxDB is a straightforward process. By including the RxDB library in your project, you can quickly start harnessing its capabilities. Begin by installing the [RxDB package](https://www.npmjs.com/package/rxdb) from the npm registry. Then, configure your database instance to suit your app's needs. This setup process paves the way for seamless data management in your ionic hybrid app.
For a full instruction, follow the [RxDB Quickstart](https://rxdb.info/quickstart.html).
### Using RxDB in Frameworks (React, Angular, Vue.js)
RxDB seamlessly integrates with various JavaScript frameworks, ensuring compatibility with your preferred development environment. Whether you're building your ionic hybrid app with [React](./react-database.md), [Angular](./angular-database.md), or [Vue.js](./vue-database.md), RxDB offers bindings and tools that enable you to leverage its features effortlessly. This compatibility allows you to stay within the comfort zone of your chosen framework while benefiting from RxDB's powerful data management capabilities.
### Different RxStorage Layers for RxDB
RxDB doesn't limit you to a single storage solution. Instead, it provides a range of [RxStorage](../rx-storage.md) layers to accommodate diverse use cases. These storage layers offer flexibility and customization, enabling you to tailor your data management strategy to match your app's requirements. Let's explore some of the available RxStorage options:
- [LocalStorage RxStorage](../rx-storage-localstorage.md): Based on the browsers [localStorage](./localstorage.md). Easy to set up and fast for small datasets.
- [IndexedDB RxStorage](../rx-storage-indexeddb.md): Leveraging the native browser storage, IndexedDB RxStorage offers reliable data persistence. This storage option is suitable for a wide range of scenarios and is supported by most modern browsers.
- [OPFS RxStorage](../rx-storage-opfs.md): Operating within the browser's file system, OPFS RxStorage is a unique choice that can handle larger data volumes efficiently. It's particularly useful for applications that require substantial data storage.
- [Memory RxStorage](../rx-storage-memory.md): Memory RxStorage is perfect for temporary or cache-like data storage. It keeps data in memory, which can result in rapid data access but doesn't provide long-term persistence.
- [SQLite RxStorage](../rx-storage-sqlite.md): SQLite is the goto database for mobile applications. It is build in on android and iOS devices. The SQLite RxDB storage layer is build upon SQLite and offers the best performance on hybrid apps, like ionic.
## Replication of Data with RxDB between Clients and Servers
Efficient data replication between clients and servers is the backbone of modern application development, ensuring that data remains consistent and up-to-date across various devices and platforms. RxDB provides a suite of replication methods that facilitate seamless communication between clients and servers, ensuring that your data is always in sync.
### RxDB Replication Algorithm
At the heart of RxDB's replication capabilities lies a sophisticated [algorithm](../replication.md) designed to manage data synchronization between clients and servers. This algorithm intelligently handles data changes, [conflict resolution](../transactions-conflicts-revisions.md), and network connectivity fluctuations, resulting in reliable and efficient data replication. With the RxDB replication algorithm, your application can maintain data consistency across devices without unnecessary complexities.
- [CouchDB Replication](../replication-couchdb.md):
RxDB's integration with CouchDB replication presents a powerful way to synchronize data between clients and servers. CouchDB, a well-established NoSQL database, excels at distributed and decentralized data scenarios. By utilizing RxDB's CouchDB replication, you can establish bidirectional synchronization between your RxDB-powered client and a CouchDB server. This synchronization ensures that data updates made on either end are seamlessly propagated to the other, facilitating collaboration and data sharing.
- [Firestore Replication](../replication-firestore.md):
Firestore, Google's cloud-hosted NoSQL database, offers another avenue for data replication in RxDB. With Firestore replication, you can establish a connection between your RxDB-powered app and Firestore's cloud infrastructure. This integration provides real-time updates to data across multiple instances of your application, ensuring that users always have access to the latest information. RxDB's support for Firestore replication empowers you to build dynamic and responsive applications that thrive in today's fast-paced digital landscape.
- [WebRTC Replication](../replication-webrtc.md):
Peer-to-peer (P2P) replication via WebRTC introduces a cutting-edge approach to data synchronization in RxDB. P2P replication allows devices to communicate directly with each other, bypassing the need for a central server. This method proves invaluable in scenarios where network connectivity is limited or unreliable. With WebRTC replication, devices can exchange data directly, enabling collaboration and information sharing even in challenging network conditions.
## RxDB as an Alternative for Ionic Secure Storage
When it comes to securing sensitive data in your Ionic applications, RxDB emerges as a powerful alternative to traditional secure storage solutions. Let's delve into why RxDB is an exceptional choice for safeguarding your data while providing additional benefits.
### RxDB On-Device Encryption Plugin
RxDB offers an [on-device encryption plugin](https://rxdb.info/encryption.html), adding an extra layer of security to your app's data. This means that data stored within the RxDB database can be encrypted, ensuring that even if the device falls into the wrong hands, the sensitive information remains inaccessible without the proper decryption key. This level of data protection is crucial for applications that deal with personal or confidential information. [Encryption](../encryption.md) runs either with `AES` on `crypto-js` or with the [Web Crypto API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API) which is faster and more secure.
### Works Offline
Security should never compromise functionality. RxDB excels in this area by allowing your application to operate seamlessly even when offline. The locally stored encrypted data remains accessible and functional, enabling users to interact with the app's features even without an active internet connection. This offline capability ensures that user data is secure, while the app continues to deliver a responsive and uninterrupted experience.
### Easy-to-Setup Replication with Your Backend
Ensuring data consistency between your client-side application and backend is a key concern for developers. RxDB simplifies this process with its straightforward replication setup. You can effortlessly configure data synchronization between your local RxDB instance and your backend server. This replication capability ensures that encrypted data remains up-to-date and aligned with the central database, enhancing data integrity and security.
### Compression of Client-Side Stored Data
In addition to security and offline capabilities, RxDB also offers [data compression](https://rxdb.info/key-compression.html). This means that the data stored on the client's device is efficiently compressed, reducing storage requirements and improving overall app performance. This compression ensures that your app remains responsive and efficient, even as data volumes grow.
### Cost-Effective Solution
In addition to its security features, RxDB offers cost-effective benefits. RxDB is [priced more affordably](/premium/) compared to some other secure storage solutions, making it an attractive option for developers seeking robust security without breaking the bank. For many users, the free version of RxDB provides ample features to meet their application's security and data management needs.
## FAQ
RxDB excels as a peer-to-peer syncing database for mobile applications. You build mobile applications using local storage on the user device. RxDB synchronizes changes directly between multiple clients utilizing WebRTC data channels. You connect devices locally without depending on a central server. This approach minimizes latency and ensures continuous data sharing even across isolated network environments.
## Follow Up
- Try out the [RxDB ionic example project](https://github.com/pubkey/rxdb/tree/master/examples/ionic)
- Try out the [RxDB Quickstart](https://rxdb.info/quickstart.html)
- Join the [RxDB Chat](https://rxdb.info/chat/)
---
## RxDB - Local Ionic Storage with Encryption, Compression & Sync
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB - Local Ionic Storage with Encryption, Compression & Sync
When building **Ionic** apps, developers face the challenge of choosing a robust **Ionic storage** mechanism that supports:
- **Offline-First** usage
- **Data Encryption** to protect sensitive content
- **Compression** to reduce storage usage and improve performance
- **Seamless Sync** with any backend for real-time updates
[RxDB](https://rxdb.info/) (Reactive Database) offers all these features in a single, [local-first](./local-first-future.md) database solution tailored to **Ionic** and other hybrid frameworks. Keep reading to learn how RxDB solves the most common storage pitfalls in hybrid app development while providing unmatched flexibility.
## Why RxDB for Ionic Storage?
### 1. Offline-Ready NoSQL Storage
[Offline functionality](../offline-first.md) is crucial for modern mobile applications, particularly when devices encounter unreliable or slow networks. RxDB stores all data **locally** so your Ionic app can run seamlessly without needing a continuous internet connection. When a network is available again, RxDB automatically synchronizes changes with your backend - no extra code required.
### 2. Powerful Encryption
Securing on-device data is paramount when handling sensitive information. RxDB includes [encryption plugins](../encryption.html) that let you:
- **Encrypt** data fields at rest with AES
- Invalidate data access by simply withholding the password
- Keep your users' data confidential, even if the device is stolen
This built-in encryption sets RxDB apart from many other Ionic storage options that lack integrated security.
### 3. Built-In Data Compression
Large or repetitive data can significantly slow down devices with minimal memory. RxDB's [key-compression](../key-compression.md) feature decreases document size stored on the device, improving overall performance by:
- Reducing disk usage
- Accelerating queries
- Minimizing network overhead when syncing
### 4. Real-Time Sync & Conflict Handling
In addition to functioning fully offline, RxDB supports advanced [replication](../replication.md) options. Your Ionic app can instantly sync updates with any backend ([CouchDB](../replication-couchdb.md), [Firestore](../replication-firestore.md), [GraphQL](../replication-graphql.md), or [custom REST](../replication-http.md)), maintaining a [real-time](./realtime-database.md) user experience. Plus, RxDB handles [conflicts](../transactions-conflicts-revisions.md) gracefully - meaning less worry about clashing user edits.
### 5. Easy to Adopt and Extend
RxDB runs with a **NoSQL** approach and integrates seamlessly into [Ionic Angular](https://ionicframework.com/docs/angular/overview) or other frameworks you might use with Ionic. You can extend or replace storage backends, add encryption, or build advanced offline-first features with minimal overhead.
## Quick Start: Implementing RxDB with LocalSTorage Storage
For a simple proof-of-concept or testing environment in [Ionic](./ionic-database.md), you can use [localstorage](../rx-storage-localstorage.md) as your underlying storage. Later, if you need better native performance, you can **switch to the SQLite storage** offered by the [RxDB Premium plugins](https://rxdb.info/premium/).
1. **Install RxDB**
```bash
npm install rxdb rxjs
```
2. **Initialize the Database**
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
async function initDB() {
const db = await createRxDatabase({
name: 'myionicdb',
storage: getRxStorageLocalstorage(),
multiInstance: false // or true if you plan multi-tab usage
// Note: If you need encryption, set `password` here
});
await db.addCollections({
notes: {
schema: {
title: 'notes schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
content: { type: 'string' },
timestamp: { type: 'number' }
},
required: ['id']
}
}
});
return db;
}
```
3. **Ready to Upgrade Later?**
When you need the best performance on mobile devices, purchase the RxDB [Premium](/premium/) [SQLite Storage](../rx-storage-sqlite.md) and replace `getRxStorageLocalstorage()` with `getRxStorageSQLite()` - your app logic remains largely the same. You only have to change the configuration.
## Encryption Example
To secure local data, add the crypto-js [encryption plugin](../encryption.md) (free version) or the [premium](/premium/) web-crypto plugin. Below is an example using the free crypto-js plugin:
```ts
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
import { createRxDatabase } from 'rxdb/plugins/core';
async function initEncryptedDB() {
const encryptedStorage = wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageLocalstorage()
});
const db = await createRxDatabase({
name: 'secureIonicDB',
storage: encryptedStorage,
password: 'myS3cretP4ssw0rd'
});
await db.addCollections({
secrets: {
schema: {
title: 'secret schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
text: { type: 'string' }
},
required: ['id'],
// all fields in this array will be stored encrypted:
encrypted: ['text']
}
}
});
return db;
}
```
With encryption enabled:
- `text` is automatically encrypted at rest.
- [Queries](../rx-query.md) on encrypted fields are not directly possible (since data is encrypted), but once a document is loaded, RxDB decrypts it for normal usage.
## Compression Example
To minimize the storage footprint, RxDB offers a [key-compression](../key-compression.md) feature. You can enable it in your schema:
```ts
await db.addCollections({
logs: {
schema: {
title: 'logs schema',
version: 0,
keyCompression: true, // enable compression
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
message: { type: 'string' },
createdAt: { type: 'string', format: 'date-time' }
}
}
}
});
```
With `keyCompression: true`, RxDB shortens field names internally, significantly reducing document size. This helps both stored data and network transport during replication.
## RxDB vs. Other Ionic Storage Options
**Ionic Native Storage** or **Capacitor-based** key-value stores may handle small amounts of data but lack advanced features like:
- Complex queries
- Full NoSQL document model
- [Offline-first](../offline-first.md) [sync](../replication.md)
- Encryption & key compression out of the box
- RxDB stands out by delivering all these capabilities in a unified library.
## Follow Up
For Ionic storage that supports offline-first operations, built-in encryption, optional data compression, and live syncing with any backend, RxDB provides a powerful solution. Start quickly with [localstorage](../rx-storage-localstorage.md) for local development and testing - then scale up to the premium SQLite storage for optimal performance on production mobile devices.
Ready to learn more?
- Explore the [RxDB Quickstart Guide](../quickstart.md)
- Check out [RxDB Encryption](../encryption.md) to protect user data
- Learn about [SQLite Storage](../rx-storage-sqlite.md) in [RxDB Premium](/premium/) for top [performance](../rx-storage-performance.md) on mobile.
- Join our community on the [RxDB Chat](/chat/)
**RxDB** - The ultimate toolkit for Ionic developers seeking offline-first, secure, and compressed local data, with real-time sync to any server.
---
## Local JavaScript Vector Database that works offline
# Local Vector Database with RxDB and transformers.js in JavaScript
The [local-first](../offline-first.md) revolution is here, changing the way we build apps! Imagine a world where your app's data lives right on the user's device, always available, even when there's no internet. That's the magic of local-first apps. Not only do they bring faster performance and limitless scalability, but they also empower users to work offline without missing a beat. And leading the charge in this space are local database solutions, like [RxDB](https://rxdb.info/).
But here's where things get even more exciting: when building [local-first](./local-first-future.md) apps, traditional databases often fall short. They're great at searching for exact matches, like `numbers` or `strings`, but what if you want to search by **meaning**, like sifting through emails to find a specific topic? Sure, you could use **RegExp**, but to truly unlock the power of semantic search and similarity-based queries, you need something more cutting-edge. Something that really understands the content of the data.
Enter **vector databases**, the game-changers for searching data by meaning! They have unlocked these new possibilities for storing and querying data, especially in tasks requiring **semantic search** and **similarity-based** queries. With the help of a **machine learning model**, data is transformed into a vector representation that can be stored, queried and compared in a database.
But unfortunately, most vector databases are designed for server-side use, typically running in large cloud clusters, not to run on a users device. To fix that, in this article, we will combine **RxDB** and **transformers.js** to create a local vector database running in the **browser** with **JavaScript**. It stores data in **[IndexedDB](../rx-storage-indexeddb.md)**, and uses a machine learning model with **WebAssembly** locally, without the need for external servers.
- [transformers.js](https://github.com/xenova/transformers.js) is a powerful framework that allows machine learning models to run directly within JavaScript using WebAssembly or WebGPU.
- [RxDB](https://rxdb.info/) is a [NoSQL](./in-memory-nosql-database.md), local-first database with a flexible storage layer that can run on any JavaScript runtime, including browsers and mobile environments. (You are reading this article on the RxDB docs).
A local vector database offers several key benefits:
- **Zero network latency**: Data is processed locally on the user's device, ensuring near-instant responses.
- **Offline functionality**: Data can be queried even without an internet connection.
- **Enhanced privacy**: Sensitive information remains on the device, never needing to leave for external processing.
- **Simple setup**: No backend servers are required, making deployment straightforward.
- **Cost savings**: By running everything locally, you avoid fees for API access or cloud services for large language models.
:::note
In this article only the important source code parts are shown. You can find the full open-source vector database implementation at the [github repository](https://github.com/pubkey/javascript-vector-database).
:::
## What is a Vector Database?
A vector database is a specialized database optimized for storing and querying data in the form of **high-dimensional** vectors, often referred to as **embeddings**. These embeddings are numerical representations of data, such as text, images, or audio, created by machine learning models like [MiniLM](https://huggingface.co/Xenova/all-MiniLM-L6-v2). Unlike traditional databases that work with exact matches on predefined fields, vector databases focus on **semantic similarity**, allowing you to query data based on meaning rather than exact values.
> A vector, or embedding, is essentially an array of numbers, like `[0.56, 0.12, -0.34, -0.90]`.
For example, instead of asking "Which document has the word 'database'?", you can query "Which documents discuss similar topics to this one?" The vector database compares embeddings and returns results based on how similar the vectors are to each other.
Vector databases handle multiple types of data beyond **text**, including **images**, **videos**, and **audio** files, all transformed into embeddings for efficient querying. Mostly you would not train a model by yourself and instead use one of the public available [transformer models](https://huggingface.co/models?pipeline_tag=feature-extraction&library=transformers.js).
Vector databases are highly effective in various types of applications:
- **Similarity Search**: Finds the closest matches to a query, even when the query doesn't contain the exact terms.
- **Clustering**: Groups similar items based on the proximity of their vector representations.
- **Recommendations**: Suggests items based on shared characteristics.
- **Anomaly Detection**: Identifies outliers that differ from the norm.
- **Classification**: Assigns categories to data based on its vector's nearest neighbors.
In this tutorial, we will build a vector database designed as a **Similarity Search** for **text**. For other use cases, the setup can be adapted accordingly. This flexibility is why [RxDB](https://rxdb.info/) doesn't provide a dedicated vector-database plugin, but rather offers utility functions to help you build your own vector search system.
## Generating Embeddings Locally in a Browser
For the first step to build a local-first vector database we need to compute embeddings directly on the user's device. This is where [transformers.js](https://github.com/xenova/transformers.js) from [huggingface](https://huggingface.co/docs/transformers.js/index) comes in, allowing us to run machine learning models in the browser with **WebAssembly**. Below is an implementation of a `getEmbeddingFromText()` function, which takes a piece of text and transforms it into an embedding using the [Xenova/all-MiniLM-L6-v2](https://huggingface.co/Xenova/all-MiniLM-L6-v2) model:
```js
import { pipeline } from "@xenova/transformers";
const pipePromise = pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2');
async function getEmbeddingFromText(text) {
const pipe = await pipePromise;
const output = await pipe(text, {
pooling: "mean",
normalize: true,
});
return Array.from(output.data);
}
```
This function creates an embedding by running the text through a pre-trained model and returning it in the form of an array of numbers, which can then be stored and further processed locally.
:::note
Vector embeddings from different machine learning models or versions are not compatible with each other. When you change your model, you have to recreate all embeddings for your data.
:::
## Storing the Embeddings in RxDB
To store the embeddings, first we have to create our [RxDB Database](../rx-database.md) with the [localstorage storage](../rx-storage-localstorage.md) that stores data in the browsers [localstorage](./localstorage.md). For more advanced projects, you can use any other [RxStorage](../rx-storage.md).
```ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageLocalstorage()
});
```
Then we add a `items` collection that stores our documents with the `text` field that stores the content.
```ts
await db.addCollections({
items: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 20
},
text: {
type: 'string'
}
},
required: ['id', 'text']
}
}
});
const itemsCollection = db.items;
```
In our [example repo](https://github.com/pubkey/javascript-vector-database), we use the [Wiki Embeddings](https://huggingface.co/datasets/Supabase/wikipedia-en-embeddings) dataset from supabase which was transformed and used to fill up the `items` collection with test data.
```ts
const imported = await itemsCollection.count().exec();
const response = await fetch('./files/items.json');
const items = await response.json();
const insertResult = await itemsCollection.bulkInsert(
items
);
```
Also we need a `vector` collection that stores our embeddings.
RxDB, as a NoSQL database, allows for the storage of flexible data structures, such as embeddings, within documents. To achieve this, we need to define a [schema](../rx-schema.md) that specifies how the embeddings will be stored alongside each document. The schema includes fields for an `id` and the `embedding` array itself.
```ts
await db.addCollections({
vector: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 20
},
embedding: {
type: 'array',
items: {
type: 'string'
}
}
},
required: ['id', 'embedding']
}
}
});
const vectorCollection = db.vector;
```
When storing documents in the database, we need to ensure that the embeddings for these documents are generated and stored automatically. This requires a handler that runs during every document write, calling the machine learning model to generate the embeddings and storing them in a separate vector collection.
Since our app runs in a browser, it's essential to avoid duplicate work when **multiple browser tabs** are open and ensure efficient use of resources. Furthermore, we want the app to resume processing documents from where it left off if it's closed or interrupted. To achieve this, RxDB provides a [pipeline plugin](../rx-pipeline.md), which allows us to set up a workflow that processes items and stores their embeddings. In our example, a pipeline takes batches of 10 documents, generates embeddings, and stores them in a separate vector collection.
```ts
const pipeline = await itemsCollection.addPipeline({
identifier: 'my-embeddings-pipeline',
destination: vectorCollection,
batchSize: 10,
handler: async (docs) => {
await Promise.all(docs.map(async(doc) => {
const embedding = await getVectorFromText(doc.text);
await vectorCollection.upsert({
id: doc.primary,
embedding
});
}));
}
});
```
However, processing data locally presents performance challenges. Running the handler with a batch size of 10 takes around **2-4 seconds per batch**, meaning processing 10k documents would take up to an hour. To improve performance, we can do parallel processing using [WebWorkers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers). A WebWorker runs on a different JavaScript process and we can start and run many of them in parallel.
Our worker listens for messages and performance the embedding generation on each request. It then sends the result embedding back to the main thread.
```ts
// worker.js
import { getVectorFromText } from './vector.js';
onmessage = async (e) => {
const embedding = await getVectorFromText(e.data.text);
postMessage({
id: e.data.id,
embedding
});
};
```
On the main thread we spawn one worker per core and send the tasks to the worker instead of processing them on the main thread.
```ts
// create one WebWorker per core
const workers = new Array(navigator.hardwareConcurrency)
.fill(0)
.map(() => new Worker(new URL("worker.js", import.meta.url)));
```
```ts
let lastWorkerId = 0;
let lastId = 0;
export async function getVectorFromTextWithWorker(text: string): Promise {
let worker = workers[lastWorkerId++];
if(!worker) {
lastWorkerId = 0;
worker = workers[lastWorkerId++];
}
const id = (lastId++) + '';
return new Promise(res => {
const listener = (ev: any) => {
if (ev.data.id === id) {
res(ev.data.embedding);
worker.removeEventListener('message', listener);
}
};
worker.addEventListener('message', listener);
worker.postMessage({
id,
text
});
});
}
const pipeline = await itemsCollection.addPipeline({
identifier: 'my-embeddings-pipeline',
destination: vectorCollection,
batchSize: navigator.hardwareConcurrency, // one per CPU core
handler: async (docs) => {
await Promise.all(docs.map(async (doc, i) => {
const embedding = await getVectorFromTextWithWorker(doc.body);
/* ... */
});
}
});
```
This setup allows us to utilize the full hardware capacity of the client's machine. By setting the batch size to match the number of logical processors available (using the [navigator.hardwareConcurrency](https://developer.mozilla.org/en-US/docs/Web/API/Navigator/hardwareConcurrency) API) and running one worker per processor, we can reduce the processing time for 10k embeddings to **about 5 minutes** on my developer laptop with 32 CPU cores.
## Comparing Vectors by calculating the distance
Now that we have stored our embeddings in the database, the next step is to compare these vectors to each other. Various methods are available to measure the similarity or difference between two vectors, such as [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance), [Manhattan distance](https://www.singlestore.com/blog/distance-metrics-in-machine-learning-simplfied/), [Cosine similarity](https://tomhazledine.com/cosine-similarity/), and **Jaccard similarity** (and more). RxDB provides utility functions for each of these methods, making it easy to choose the most suitable method for your application. In this tutorial, we will use **Euclidean distance** to compare vectors. However, the ideal algorithm may vary depending on your data's distribution and the specific type of query you are performing. To find the optimal method for your app, it is up to you to try out all of these and compare the results.
Each method gets two vectors as input and returns a single number. Here's how to calculate the Euclidean distance between two embeddings with the vector utilities from RxDB:
```ts
import { euclideanDistance } from 'rxdb/plugins/vector';
const distance = euclideanDistance(embedding1, embedding2);
console.log(distance); // 25.20443
```
With this we can sort multiple embeddings by how good they match our search query vector.
## Searching the Vector database with a full table scan
To find out if our embeddings have been stored correctly and that our vector comparison works as should, let's run a basic query to ensure everything functions as expected. In this query, we aim to find documents similar to a given user input text. The process involves calculating the embedding from the input text, fetching all documents, calculating the distance between their embeddings and the query embedding, and then sorting them based on their similarity.
```ts
import { euclideanDistance } from 'rxdb/plugins/vector';
import { sortByObjectNumberProperty } from 'rxdb/plugins/core';
const userInput = 'new york people';
const queryVector = await getEmbeddingFromText(userInput);
const candidates = await vectorCollection.find().exec();
const withDistance = candidates.map(doc => ({
doc,
distance: euclideanDistance(queryVector, doc.embedding)
}));
const queryResult = withDistance
.sort(sortByObjectNumberProperty('distance'))
.reverse();
console.dir(queryResult);
```
:::note
For **distance**-based comparisons, sorting should be in ascending order (smallest first), while for **similarity**-based algorithms, the sorting should be in descending order (largest first).
:::
If we inspect the results, we can see that the documents returned are ordered by relevance, with the most similar document at the top:
:::note
This demo page can be [run online here](https://pubkey.github.io/javascript-vector-database/).
:::
However our full-scan method presents a significant challenge: it does not scale well. As the number of stored documents increases, the time taken to fetch and compare embeddings grows proportionally. For example, retrieving embeddings from our [test dataset](https://huggingface.co/datasets/Supabase/wikipedia-en-embeddings) of 10k documents takes around **700 milliseconds**. If we scale up to 100k documents, this delay would rise to approximately **7 seconds**, making the search process inefficient for larger datasets.
## Indexing the Embeddings for Better Performance
To address the scalability issue, we need to store embeddings in a way that allows us to avoid fetching all of them from storage during a query. In traditional databases, you can sort documents by an **index field**, allowing efficient queries that retrieve only the necessary documents. An index organizes data in a structured, sortable manner, much **like a phone book**. However, with vector embeddings we are not dealing with simple, single values. Instead, we have large **lists of numbers**, which makes indexing more complex because we have more than one dimension.
### Vector Indexing Methods
Various methods exist for indexing these vectors to improve query efficiency and performance:
- [Locality Sensitive Hashing (LSH)](https://www.youtube.com/watch?v=Arni-zkqMBA): LSH hashes data so that similar items are likely to fall into the same bucket, optimizing approximate nearest neighbor searches in high-dimensional spaces by reducing the number of comparisons.
- [Hierarchical Small World](https://www.youtube.com/watch?v=77QH0Y2PYKg): HSW is a graph structure designed for efficient navigation, allowing quick jumps across the graph while maintaining short paths between nodes, forming the basis for HNSW's optimization.
- [Hierarchical Navigable Small Worlds (HNSW)](https://www.youtube.com/watch?v=77QH0Y2PYKg): HNSW builds a hierarchical graph for fast approximate nearest neighbor search. It uses multiple layers where higher layers represent fewer, more connected nodes, improving search efficiency in large datasetsβ.
- **Distance to samples**: While testing different indexing strategies, [I](https://github.com/pubkey) found out that using the distance to a sample set of items is a good way to index embeddings. You pick like 5 random items of your data and get the embeddings for them out of the model. These are your 5 index vectors. For each embedding stored in the vector database, we calculate the distance to our 5 index vectors and store that `number` as an index value. This seems to work good because similar things have similar distances to other things. For example the words "shoe" and "socks" have a similar distance to "boat" and therefore should have roughly the same index value.
When building **local-first** applications, performance is often a challenge, especially in JavaScript. With **IndexedDB**, certain operations, like many sequential `get by id` calls, [are slow](../slow-indexeddb.md), while bulk operations, such as `get by index range`, are fast. Therefore, it's essential to use an indexing method that allows embeddings to be stored in a sortable way, like **Locality Sensitive Hashing** or **Distance to Samples**. In this article, we'll use **Distance to Samples**, because for [me](https://github.com/pubkey) it provides the best default behavior for the sample dataset.
### Storing indexed embeddings in RxDB
The optimal way to store index values alongside embeddings in RxDB is to place them within the same [RxCollection](../rx-collection.md). To ensure that the index values are both sortable and precise, we convert them into strings with a fixed length of `10` characters. This standardization helps in managing values with many decimals and ensures proper sorting in the database.
Here's is our schema example schema where each document contains an embedding and corresponding index fields:
```ts
const indexSchema = {
type: 'string',
maxLength: 10
};
const schema = {
"version": 0,
"primaryKey": "id",
"type": "object",
"properties": {
"id": {
"type": "string",
"maxLength": 100
},
"embedding": {
"type": "array",
"items": {
"type": "number"
}
},
// index fields
"idx0": indexSchema,
"idx1": indexSchema,
"idx2": indexSchema,
"idx3": indexSchema,
"idx4": indexSchema
},
"required": [
"id",
"embedding",
"idx0",
"idx1",
"idx2",
"idx3",
"idx4"
],
"indexes": [
"idx0",
"idx1",
"idx2",
"idx3",
"idx4"
]
}
```
To populate these index fields, we modify the [RxPipeline](../rx-pipeline.md) handler accordingly to the **Distance to samples** method. We calculate the distance between the document's embedding and our set of `5` index vectors. The calculated distances are converted to `string` and stored in the appropriate index fields:
```ts
import { euclideanDistance } from 'rxdb/plugins/vector';
const sampleVectors: number[][] = [/* the index vectors */];
const pipeline = await itemsCollection.addPipeline({
handler: async (docs) => {
await Promise.all(docs.map(async(doc) => {
const embedding = await getEmbedding(doc.text);
const docData = { id: doc.primary, embedding };
// calculate distance to all samples
// and store them in the index fields
new Array(5).fill(0).map((_, idx) => {
const indexValue = euclideanDistance(sampleVectors[idx], embedding);
docData['idx' + idx] = indexNrToString(indexValue);
});
await vectorCollection.upsert(docData);
}));
}
});
```
## Searching the Vector database with utilization of the indexes
Once our embeddings are stored in an indexed format, we can perform searches much **more efficiently** than through a full table scan. While this indexing method boosts performance, it comes with a tradeoff: a slight loss in precision, meaning that the result set may not always be the optimal one. However, this is generally acceptable for **similarity search** use cases.
There are multiple ways to leverage indexes for faster queries. Here are two effective methods:
1. **Query for Index Similarity in Both Directions**: For each index vector, calculate the distance to the search embedding and fetch all relevant embeddings in both directions (sorted before and after) from that value.
```ts
async function vectorSearchIndexSimilarity(searchEmbedding: number[]) {
const docsPerIndexSide = 100;
const candidates = new Set();
await Promise.all(
new Array(5).fill(0).map(async (_, i) => {
const distanceToIndex = euclideanDistance(
sampleVectors[i], searchEmbedding
);
const [docsBefore, docsAfter] = await Promise.all([
vectorCollection.find({
selector: {
['idx' + i]: {
$lt: indexNrToString(distanceToIndex)
}
},
sort: [{ ['idx' + i]: 'desc' }],
limit: docsPerIndexSide
}).exec(),
vectorCollection.find({
selector: {
['idx' + i]: {
$gt: indexNrToString(distanceToIndex)
}
},
sort: [{ ['idx' + i]: 'asc' }],
limit: docsPerIndexSide
}).exec()
]);
docsBefore.map(d => candidates.add(d));
docsAfter.map(d => candidates.add(d));
})
);
const docsWithDistance = Array.from(candidates).map(doc => {
const distance = euclideanDistance((doc as any).embedding, searchEmbedding);
return {
distance,
doc
};
});
const sorted = docsWithDistance
.sort(sortByObjectNumberProperty('distance'))
.reverse();
return {
result: sorted.slice(0, 10),
docReads
};
}
```
2. **Query for an Index Range with a Defined Distance**: Set an `indexDistance` and retrieve all embeddings within a specified range from the index vector to the search embedding.
```ts
async function vectorSearchIndexRange(searchEmbedding: number[]) {
await pipeline.awaitIdle();
const indexDistance = 0.003;
const candidates = new Set();
let docReads = 0;
await Promise.all(
new Array(5).fill(0).map(async (_, i) => {
const distanceToIndex = euclideanDistance(
sampleVectors[i], searchEmbedding
);
const range = distanceToIndex * indexDistance;
const docs = await vectorCollection.find({
selector: {
['idx' + i]: {
$gt: indexNrToString(distanceToIndex - range),
$lt: indexNrToString(distanceToIndex + range)
}
},
sort: [{ ['idx' + i]: 'asc' }],
}).exec();
docs.map(d => candidates.add(d));
docReads = docReads + docs.length;
})
);
const docsWithDistance = Array.from(candidates).map(doc => {
const distance = euclideanDistance((doc as any).embedding, searchEmbedding);
return {
distance,
doc
};
});
const sorted = docsWithDistance
.sort(sortByObjectNumberProperty('distance'))
.reverse();
return {
result: sorted.slice(0, 10),
docReads
};
};
```
Both methods allow you to limit the number of embeddings fetched from storage while still ensuring a reasonably precise search result. However, they differ in how many embeddings are read and how precise the results are, with trade-offs between performance and accuracy. The first method has a known embedding read amount of `docsPerIndexSide * 2 * [amount of indexes]`. The second method reads out an unknown amount of embeddings, depending on the sparsity of the dataset and the value of `indexDistance`.
And that's it for the implementation. We now have a local first vector database that is able to store and query vector data.
## Performance benchmarks
In server-side databases, performance can be improved by scaling hardware or adding more servers. However, [local-first](../offline-first.md) apps face the unique challenge that the hardware is determined by the end user, making performance unpredictable. Some users may have **high-end gaming PCs**, while others might be using **outdated smartphones in power-saving mode**. Therefore, when building a local-first app that processes more than a few documents, performance becomes a critical factor and should be thoroughly tested upfront.
Let's run performance benchmarks on my **high-end gaming PC** to give you a sense of how long different operations take and what's achievable.
### Performance of the Query Methods
| Query Method | Time in milliseconds | Docs read from storage |
| ---------------- | -------------------- | ---------------------- |
| Full Scan | 765 | 10000 |
| Index Similarity | 1647 | 934 |
| Index Range | 88 | 2187 |
As shown, the **index similarity** query method takes significantly longer compared to others. This is due to the need for descending sort orders in some queries `sort: [{ ['idx' + i]: 'desc' }]`. While RxDB supports descending sorts, performance suffers because IndexedDB does not efficiently handle [reverse indexed bulk operations](https://github.com/w3c/IndexedDB/issues/130). As a result, the **index range method** performs much better for this use case and should be used instead. With its query time of only `88` milliseconds it is fast enough for all most things and likely such fast that you do not even need to show a loading spinner. Also it is faster compared to fetching the query result from a server-side vector database over the internet.
### Performance of the Models
Let's also look at the time taken to calculate a single embedding across various models from the [huggingface transformers list](https://huggingface.co/models?pipeline_tag=feature-extraction&library=transformers.js):
| Model Name | Time per Embedding in (ms) | Vector Size | Model Size (MB) |
| -------------------------------------------- | -------------------------- | ----------- | --------------- |
| Xenova/all-MiniLM-L6-v2 | 173 | 384 | 23 |
| Supabase/gte-small | 341 | 384 | 34 |
| Xenova/paraphrase-multilingual-mpnet-base-v2 | 1000 | 768 | 279 |
| jinaai/jina-embeddings-v2-base-de | 1291 | 768 | 162 |
| jinaai/jina-embeddings-v2-base-zh | 1437 | 768 | 162 |
| jinaai/jina-embeddings-v2-base-code | 1769 | 768 | 162 |
| mixedbread-ai/mxbai-embed-large-v1 | 3359 | 1024 | 337 |
| WhereIsAI/UAE-Large-V1 | 3499 | 1024 | 337 |
| Xenova/multilingual-e5-large | 4215 | 1024 | 562 |
From these benchmarks, it's evident that models with larger vector outputs **take longer to process**. Additionally, the model size significantly affects performance, with larger models requiring more time to compute embeddings. This trade-off between model complexity and performance must be considered when choosing the right model for your use case.
## Potential Performance Optimizations
There are multiple other techniques to improve the performance of your local vector database:
- **Shorten embeddings**: The storing and retrieval of embeddings can be improved by "shortening" the embedding. To do that, you just strip away numbers from your vector. For example `[0.56, 0.12, -0.34, 0.78, -0.90]` becomes `[0.56, 0.12]`. That's it, you now have a smaller embedding that is faster to read out of the storage and calculating distances is faster because it has to process less numbers. The downside is that you loose precision in your search results. Sometimes shortening the embeddings makes more sense as a pre-query step where you first compare the shortened vectors and later fetch the "real" vectors for the 10 most matching documents to improve their sort order.
- **Optimize the variables in our Setup**: In this examples we picked our variables in a non-optimal way. You can get huge performance improvements by setting different values:
- We picked 5 indexes for the embeddings. Using less indexes improves your query performance with the cost of less good results.
- For queries that search by fetching a specific embedding distance we used the `indexDistance` value of `0.003`. Using a lower value means we read less document from the storage. This is faster but reduces the precision of the results which means we will get a less optimal result compared to a full table scan.
- For queries that search by fetching a given amount of documents per index side, we set the value `docsPerIndexSide` to `100`. Increasing this value means you fetch more data from the storage but also get a better precision in the search results. Decreasing it can improve query performance with worse precision.
- **Use faster models**: There are many ways to improve performance of machine learning models. If your embedding calculation is too slow, try other models. **Smaller** mostly means **faster**. The model `Xenova/all-MiniLM-L6-v2` which is used in this tutorial is about [1 year old](https://huggingface.co/Xenova/all-MiniLM-L6-v2/tree/main). There exist better, more modern models to use. Huggingface makes these convenient to use. You only have to switch out the model name with any other model from [that site](https://huggingface.co/models?pipeline_tag=feature-extraction&library=transformers.js).
- **Narrow down the search space**: By utilizing other "normal" filter operators to your query, you can narrow down the search space and optimize performance. For example in an email search you could additionally use a operator that limits the results to all emails that are not older than one year.
- **Dimensionality Reduction** with an [autoencoder](https://www.youtube.com/watch?v=D16rii8Azuw): An autoencoder encodes vector data with minimal loss which can improve the performance by having to store and compare less numbers in an embedding.
- **Different RxDB Plugins**: RxDB has different storages and plugins that can improve the performance like the [IndexedDB RxStorage](../rx-storage-indexeddb.md), the [OPFS RxStorage](../rx-storage-opfs.md), the [sharding](../rx-storage-sharding.md) plugin and the [Worker](../rx-storage-worker.md) and [SharedWorker](../rx-storage-shared-worker.md) storages.
## Migrating Data on Model/Index Changes
When you change the index parameter or even update the whole model which was used to create the embeddings, you have to migrate the data that is already stored on your users devices. RxDB offers the [Schema Migration Plugin](../migration-schema.md) for that.
When the app is reloaded and the updated source code is started, RxDB detects changes in your [schema version](../rx-schema.md#version) and runs the [migration strategy](../migration-schema.md#providing-strategies) accordingly. So to update the stored data, increase the schema version and define a handler:
```ts
const schemaV1 = {
"version": 1, // <- increase schema version by 1
"primaryKey": "id",
"properties": {
/* ... */
},
/* ... */
};
```
In the migration handler we recreate the new embeddings and index values.
```ts
await myDatabase.addCollections({
vectors: {
schema: schemaV1,
migrationStrategies: {
1: function(docData){
const embedding = await getEmbedding(docData.body);
new Array(5).fill(0).map((_, idx) => {
docData['idx' + idx] = euclideanDistance(mySampleVectors[idx], embedding);
});
return docData;
},
}
}
});
```
## Possible Future Improvements to Local-First Vector Databases
For now our vector database works and we are good to go. However there are some things to consider for the future:
- **WebGPU** is [not fully supported](https://caniuse.com/webgpu) yet. When this changes, creating embeddings in the browser have the potential to become faster. You can check if your current chrome supports WebGPU by opening `chrome://gpu/`. Notice that WebGPU has been reported to sometimes be [even slower](https://github.com/xenova/transformers.js/issues/894#issuecomment-2323897485) compared to WASM but likely it will be faster in the long term.
- **Cross-Modal AI Models**: While progress is being made, AI models that can understand and integrate multiple modalities are still in development. For example you could query for an **image** together with a **text** prompt to get a more detailed output.
- **Multi-Step queries**: In this article we only talked about having a single query as input and an ordered list of outputs. But there is big potential in chaining models or queries together where you take the results of one query and input them into a different model with different embeddings or outputs.
## Follow Up
- Shared/Like my [announcement tweet](https://x.com/rxdbjs/status/1833429569434427494)
- Read the source code that belongs to this article [at github](https://github.com/pubkey/javascript-vector-database)
- Learn how to use RxDB with the [RxDB Quickstart](../quickstart.md)
- Check out the [RxDB github repo](https://github.com/pubkey/rxdb) and leave a star β
---
## RxDB as a Database in a jQuery Application
import {VideoBox} from '@site/src/components/video-box';
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB as a Database in a jQuery Application
In the early days of dynamic web development, **jQuery** emerged as a popular library that simplified DOM manipulation and AJAX requests. Despite the rise of modern frameworks, many developers still maintain or extend existing jQuery projects, or leverage jQuery in specific contexts. As jQuery applications grow in complexity, they often require efficient data handling, offline support, and synchronization capabilities. This is where [RxDB](https://rxdb.info/), a reactive JavaScript database for the browser, node.js, and [mobile devices](./mobile-database.md), steps in.
## jQuery Web Applications
jQuery provides a simple API for DOM manipulation, event handling, and AJAX calls. It has been widely adopted due to its ease of use and strong community support. Many projects continue to rely on jQuery for handling client-side functionality, UI interactions, and animations. As these applications evolve, the need for a robust database solution that can manage data locally (and offline) becomes increasingly important.
## Importance of Databases in jQuery Applications
Modern, data-driven jQuery applications often need to:
- **Store and retrieve data locally** for quick and responsive user experiences.
- **Synchronize data** between clients or with a [central server](../rx-server.md).
- **Handle offline scenarios** seamlessly.
- **Handle large or complex data structures** without repeatedly hitting the server.
Relying solely on server endpoints or basic browser storage (like `localStorage`) can quickly become unwieldy for larger or more complex use cases. Enter RxDB, a dedicated solution that manages data on the client side while offering real-time synchronization and offline-first capabilities.
## Introducing RxDB as a Database Solution
RxDB (short for Reactive Database) is built on top of [IndexedDB](./browser-database.md) and leverages [RxJS](https://rxjs.dev/) to provide a modern, reactive approach to handling data in the browser. With RxDB, you can store documents locally, query them in real-time, and synchronize changes with a remote server whenever an internet connection is available.
### Key Features
- **Reactive Data Handling**: RxDB emits real-time updates whenever your data changes, allowing you to instantly reflect these changes in the DOM with jQuery.
- **Offline-First Approach**: Keep your application usable even when the user's network is unavailable. Data is automatically synchronized once connectivity is restored.
- **Data Replication**: Enable multi-device or multi-tab synchronization with minimal effort.
- **[Observable Queries](../rx-query.md)**: Reduce code complexity by subscribing to queries instead of constantly polling for changes.
- **Multi-Tab Support**: If a user opens your jQuery application in multiple tabs, RxDB keeps data in sync across all sessions.
## Getting Started with RxDB
### What is RxDB?
[RxDB](https://rxdb.info/) is a client-side NoSQL database that stores data in the browser (or [node.js](../nodejs-database.md)) and synchronizes changes with other instances or servers. Its design embraces reactive programming principles, making it well-suited for real-time applications, offline scenarios, and multi-tab use cases.
### Reactive Data Handling
RxDB's use of observables enables an event-driven architecture where data mutations automatically trigger UI updates. In a jQuery application, you can subscribe to these changes and update DOM elements as soon as data changes occur - no need for manual refresh or complicated change detection logic.
### Offline-First Approach
One of RxDB's distinguishing traits is its emphasis on offline-first design. This means your jQuery application continues to function, display, and update data even when there's no network connection. When connectivity is restored, RxDB synchronizes updates with the server or other peers, ensuring consistency across all instances.
### Data Replication
RxDB supports real-time data [replication](../replication.md) with different backends. By enabling replication, you ensure that multiple clients - be they multiple [browser](./browser-database.md) tabs or separate devices - stay in sync. RxDB's conflict resolution strategies help keep the data consistent even when multiple users make changes simultaneously.
### Observable Queries
Instead of static queries, RxDB provides observable queries. Whenever data relevant to a query changes, RxDB re-emits the new result set. You can subscribe to these updates within your jQuery code and instantly reflect them in the UI.
### Multi-Tab Support
Running your jQuery app in multiple tabs? RxDB automatically synchronizes changes between those tabs. Users can freely switch windows without missing real-time updates.
### RxDB vs. Other jQuery Database Options
Historically, jQuery developers might use `localStorage` or raw `IndexedDB` for storing data. However, these solutions can require significant boilerplate, lack [reactivity](../reactivity.md), and offer no built-in sync or conflict resolution. RxDB fills these gaps with an out-of-the-box solution, abstracting away low-level database complexities and providing an event-driven, offline-capable approach.
## Using RxDB in a jQuery Application
### Installing RxDB
Install RxDB (and `rxjs`) via npm or yarn:
```bash
npm install rxdb rxjs
```
If your project isn't set up with a build process, you can still use bundlers like Webpack or Rollup, or serve RxDB as a UMD bundle. Once included, you'll have access to RxDB globally or via import statements.
## Creating and Configuring a Database
Below is a minimal example of how to create an RxDB instance and collection. You can call this when your page initializes, then store the `db` object for later use:
```js
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
async function initDatabase() {
const db = await createRxDatabase({
name: 'heroesdb',
storage: getRxStorageLocalstorage(),
password: 'myPassword', // optional encryption password
multiInstance: true, // multi-tab support
eventReduce: true // optimizes event handling
});
await db.addCollections({
hero: {
schema: {
title: 'hero schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
points: { type: 'number' }
}
}
}
});
return db;
}
```
## Updating the DOM with jQuery
Once you have your RxDB instance, you can query data reactively and use jQuery to manipulate the DOM:
```js
// Example: Displaying heroes using jQuery
$(document).ready(async function () {
const db = await initDatabase();
// Subscribing to all hero documents
db.hero
.find()
.$ // the observable
.subscribe((heroes) => {
// Clear the list
$('#heroList').empty();
// Append each hero to the DOM
heroes.forEach((hero) => {
$('#heroList').append(`
${hero.name} - Points: ${hero.points}
`);
});
});
// Example of adding a new hero
$('#addHeroBtn').on('click', async () => {
const heroName = $('#heroName').val();
const heroPoints = parseInt($('#heroPoints').val(), 10);
await db.hero.insert({
id: Date.now().toString(),
name: heroName,
points: heroPoints
});
});
});
```
With this approach, any time data in the `hero` collection changes - like when a new hero is added - your jQuery code re-renders the list of heroes automatically.
## Different RxStorage layers for RxDB
RxDB supports multiple storage backends ([RxStorage](../rx-storage.md) layers). Some popular ones:
- [LocalStorage.js RxStorage](../rx-storage-localstorage.md): Uses the browsers [localstorage](./localstorage.md). Fast and easy to set up.
- [IndexedDB RxStorage](../rx-storage-indexeddb.md): Direct IndexedDB usage, suitable for modern browsers.
- [OPFS RxStorage](../rx-storage-opfs.md): Uses the File System Access API for better performance in supported browsers.
- [Memory RxStorage](../rx-storage-memory.md): Stores data in memory, handy for tests or ephemeral data.
- [SQLite RxStorage](../rx-storage-sqlite.md): Uses SQLite (potentially via WebAssembly). In typical browser-based scenarios, localstorage or IndexedDB storage is usually more straightforward.
## Synchronizing Data with RxDB between Clients and Servers
### Offline-First Approach
RxDB's [offline-first](../offline-first.md) approach allows your jQuery application to store and query data locally. Users can continue interacting, even offline. When connectivity returns, RxDB syncs to the server.
### Conflict Resolution
Should multiple clients update the same document, RxDB offers [conflict handling strategies](../transactions-conflicts-revisions.md). You decide how to resolve conflicts - like keeping the latest edit or merging changes - ensuring data integrity across distributed systems.
### Bidirectional Synchronization
With RxDB, data changes flow both ways: from client to server and from server to client. This real-time synchronization ensures that all users or tabs see consistent, up-to-date data.
## Advanced RxDB Features and Techniques
### Indexing and Performance Optimization
Create indexes on frequently queried fields to speed up performance. For large data sets, indexing can drastically improve query times, keeping your jQuery UI snappy.
### Encryption of Local Data
RxDB supports [encryption to secure data stored in the browser](../encryption.md). This is crucial if your application handles sensitive user information.
### Change Streams and Event Handling
Use change streams to listen for data modifications at the database or collection level. This can trigger [real-time](./realtime-database.md) [UI updates](./optimistic-ui.md), notifications, or custom logic whenever the data changes.
### JSON Key Compression
If your data model has large or repetitive field names, [JSON key compression](../key-compression.md) can minimize stored document size and potentially boost performance.
## Best Practices for Using RxDB in jQuery Applications
- Centralize Your Database: Initialize and configure RxDB in one place. Expose the instance where needed or store it globally to avoid re-creating it on every script.
- Leverage Observables: Instead of polling or manually refreshing data, rely on RxDB's reactivity. Subscribe to queries and let RxDB inform you when data changes.
- Handle Subscriptions: If you create subscriptions in a single-page context, ensure you don't re-subscribe endlessly or create memory leaks. Clean them up if you're navigating away or removing DOM elements.
- Offline Testing: Thoroughly test how your jQuery app behaves without a network connection. Simulate offline states in your browser's dev tools or with flight mode to ensure the user experience remains smooth.
- Performance Profiling: For large data sets or frequent data updates, add indexes and carefully measure query performance. Optimize only where needed.
## Follow Up
To explore more about RxDB and leverage its capabilities for browser database development, check out the following resources:
- [RxDB GitHub Repository](/code/): Visit the official GitHub repository of RxDB to access the source code, documentation, and community support.
- [RxDB Quickstart](../quickstart.md): Get started quickly with RxDB by following the provided quickstart guide, which offers step-by-step instructions for setting up and using RxDB in your projects.
- [RxDB Examples](https://github.com/pubkey/rxdb/tree/master/examples): Browse official examples to see RxDB in action and learn best practices you can apply to your own project - even if jQuery isn't explicitly featured, the patterns are similar.
---
## JSON-Based Databases - Why NoSQL and RxDB Simplify App Development
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
# JSON-Based Databases: Why NoSQL and RxDB Simplify App Development
Modern applications handle highly dynamic, often deeply nested data structures, commonly represented in **JSON**. Whether you're building a real-time dashboard or a fully offline mobile app, storing and querying data in a JSON-friendly way can reduce overhead and coding complexity. This is where **JSON-based databases** (often part of the **NoSQL** family) come into play, letting you store objects in the same format they're used in your code, eliminating the schema wrangling that can come with a strict relational design.
Below, we explore why JSON-based databases naturally align with **NoSQL** principles, how relational engines (like PostgreSQL or SQLite) handle JSON columns, the pitfalls of storing data in a single plain JSON text file, and the ways [RxDB](https://rxdb.info/) stands out as an offline-first JSON solution for JavaScript developers, complete with advanced features like JSON-Schema and JSON-key-compression.
## Why JSON-Based Databases Are Typically NoSQL
### Document-Oriented by Nature
When your data is stored as JSON, each record or document can hold nested arrays and sub-objects with no forced table schema. NoSQL solutions such as [MongoDB](../rx-storage-mongodb.md), [CouchDB](../replication-couchdb.md), [Firebase](../replication-firestore.md), and **RxDB** store and retrieve these documents in their βrawβ JSON form. This model integrates smoothly with how front-end applications already handle data, minimizing transformations and improving developer productivity.
### Flexible, Schema-Agnostic
Traditional SQL tables enforce rigid column definitions and demand explicit schema migrations when you add or rename a field. By contrast, NoSQL solutions accept more dynamic data structures, allowing changes on the fly. This means a front-end developer can add a new field to a JSON object for a new feature without the friction of redefining or migrating a database schema. While this is possible, it is often not recommended.
### Aligned With Evolving User Interfaces
As modern UIs frequently manipulate deeply nested or changing data, developers find it easier to store whole objects directly, saving time that might otherwise be spent performing complex joins or normalizing data. For instance, frameworks like [React](./react-database.md), [Vue](./vue-database.md), or [Angular](./angular-database.md) are inherently comfortable with nested JSON structures, which map more directly toNoSQLβs βdocumentβ approach than to relational tables.
## Is NoSQL βBetterβ Than SQL?
It depends on your application. **SQL** remains exceptional for complex aggregations, enforced relationships, and sophisticated transaction handling. But **NoSQL** is often more intuitive and easier to maintain for βdocument-firstβ applications that:
- Thrive on flexible or rapidly evolving data models.
- Rely on hierarchical or nested JSON objects.
- Avoid multi-table joins.
- Require easy horizontal scaling for large sets of documents.
Relational databases are still a top choice for many enterprise back-ends, especially when advanced analytics or strongly enforced referential integrity is needed. But if your application is predominantly storing and manipulating JSON documents (e.g., user profiles, real-time chat logs, embedded items), a JSON-based or document-oriented approach can greatly reduce friction during development.
## When to Prefer SQL Instead of JSON/NoSQL
NoSQL solutions such as JSON-based document stores provide a natural fit for flexible, nested data in UI-heavy applications. However, certain scenarios may benefit more from a **SQL** solution:
1. **Complex Relationships**: If your data demands intricate joins across multiple entities (e.g., many-to-many relationships that canβt easily be embedded in a single document), a well-structured relational schema can simplify queries.
2. **Strong Integrity and Constraints**: SQL excels at enforcing constraints such as foreign keys, unique constraints, and advanced triggers. If your system needs strict data validation and complex business logic within the database, SQL might prove more robust.
3. **High-End Analytical Queries**: Relational databases can handle sophisticated aggregations, groupings, and joins more efficiently. If your app frequently runs advanced SQL queries, a NoSQL approach may complicate or slow down analytics.
4. **Legacy Integration**: Many enterprise systems are built around existing relational schemas. A purely NoSQL approach might mean rewriting or bridging systems that are heavily reliant on SQL constraints and transformations.
5. **Transaction Handling**: While many NoSQL solutions have improved transaction support, it can still lag behind well-established SQL transaction models. If ACID properties and multi-operation atomicity are paramount, you might prefer a tried-and-true relational engine.
In short, if you prioritize advanced relational queries, robust constraints, or complex business rules at the database level, SQL remains a powerful, and possibly superior, choice. For user-centric, fast-evolving JSON data, though, NoSQL or JSON-based solutions often reduce the friction of frequent schema changes.
## Storing JSON in Traditional SQL Databases
### JSON Columns in PostgreSQL or MySQL
To accommodate the demand for flexible data, several SQL engines (notably **PostgreSQL** and MySQL) introduced support for **JSON** columns. PostgreSQL offers the `JSON` and `JSONB` types, enabling developers to store raw JSON in a column. You can also index specific paths within the JSON to speed lookups on nested fields:
```sql
CREATE TABLE products (
id SERIAL PRIMARY KEY,
name TEXT,
details JSONB
);
-- Insert a record with JSON data
INSERT INTO products (name, details) VALUES
('Laptop', '{"brand": "BrandX", "features": ["Touchscreen", "SSD"]}');
```
Although this approach merges the best of both worlds (SQL queries + flexible JSON fields), it can also create a βsplit personalityβ in your schema. You might store stable data in normal columns, while unpredictable or nested details live inside a JSONB field. Some projects flourish with this hybrid design, others find it a bit unwieldy.
## Storing JSON in SQLite
SQLite also allows storing JSON data, typically as text columns, but with some additional features since **SQLite 3.9** (2015) including the [JSON1 extension](https://www.sqlite.org/json1.html). This extension can parse JSON text, perform queries on JSON fields, and do partial updates. However, storing JSON in SQLite does require you to ensure youβve compiled SQLite with JSON1 support or to rely on a library that bundles it. While possible, you still won't get quite the same schema-agnostic ease as a full document store, but itβs a pragmatic solution for smaller or embedded needs on the server side or occasionally in the browser if you run SQLite via WebAssembly. RxDB uses this in its [SQLite storage](../rx-storage-sqlite.md).
## JSON vs. Database - Why a Plain JSON Text File is a Problem
Some developers consider storing everything in a single JSON file, typically read and written directly from disk or local storage. This approach, while seemingly simple, usually does not scale. Key issues include:
- **No Concurrency**: If multiple parts of the application try to write to the same JSON file, you risk overwriting changes.
- **No Indexes**: Finding or filtering items in large JSON text requires scanning everything. This is slow and quickly becomes unmanageable.
- **No Partial Updates**: You often reload the entire file, modify it in memory, then write it back, which is highly inefficient for large data sets.
- **Corruption Risk**: A single corrupted write or partial save might break the entire JSON file, losing all data.
- **High Memory Usage**: The entire file may need to be parsed into memory, even if you only need a fraction of the data.
Both relational and NoSQL databases solve these issues by handling concurrency, enabling partial reads/writes, establishing indexes, and ensuring transactional integrity so you donβt lose everything if the process is interrupted mid-write.
## RxDB: A JSON-Focused Database for JavaScript Apps
Many NoSQL databases operate on the server, whereas RxDB is built for client-side usage in browsers, mobile apps, or [Node.js](../nodejs-database.md). It specializes in JSON documents and embraces an [offline-first](../offline-first.md) philosophy.
### Key Characteristics
1. **Local JSON Storage**
RxDB stores each record as a JSON document, closely matching how front-end frameworks handle state. This eliminates complex transformations or manual JSON parsing before writing to a table.
2. **Reactive Queries**
Instead of complex SQL, RxDB uses JSON-based [query](../rx-query.md) definitions. You can subscribe to query results, letting your UI automatically refresh when data changes locally or from remote sync updates:
3. **Offline-First Sync**
Built-in replication plugins push/pull changes to or from a remote server. If your app is offline, updates get stored locally, then sync up seamlessly once a connection is available.
4. **Optional JSON-Schema**
Though itβs a document database, RxDB encourages you to define a JSON-based schema for clarity, indexing, and type validation. This helps maintain data consistency while still allowing a measure of flexibility for new fields.
### Advanced JSON Features in RxDB
- **JSON-Schema**: By specifying a JSON-Schema, you can define which fields exist, whether they are required, and their data types. This is invaluable for catching malformed documents early and imposing mild structure in a NoSQL setting.
- **JSON Key-Compression**: Large, verbose field names can bloat storage usage. RxDBβs optional [key-compression plugin](../key-compression.md) automatically shortens field names in your JSON documents internally, reducing disk space and bandwidth:
```ts
// Example: how key-compression can transform your documents
const uncompressed = {
"firstName": "Corrine",
"lastName": "Ziemann",
"shoppingCartItems": [
{ "productNumber": 29857, "amount": 1 },
{ "productNumber": 53409, "amount": 6 }
]
};
const compressed = {
"|e": "Corrine",
"|g": "Ziemann",
"|i": [
{ "|h": 29857, "|b": 1 },
{ "|h": 53409, "|b": 6 }
]
};
```
The user sees no difference in their code since RxDB automatically decompresses data on read, but the overhead is drastically reduced behind the scenes.
Yes, databases that store records as dynamic, nested JSON documents (like MongoDB, CouchDB, or **[RxDB](https://rxdb.info)**) are inherently categorized under the broader "NoSQL" (Not Only SQL) umbrella. This document-oriented approach fundamentally bypasses the strict columns, predetermined schema migrations, and rigid normalization rules enforced by traditional relational SQL databases, prioritizing development speed and seamless alignment with JavaScript application state.
MongoDB and Couchbase are robust commercial engines built deliberately around schema-less JSON storage. For open-source, Postgres and SQLite offer impressive hybrid solutions by embedding JSON/JSONB text columns natively. However, for client-side JavaScript applications operating entirely in the browser, **[RxDB](https://rxdb.info)** offers the most comprehensive open-source solution, acting as a fully reactive, Offline-First NoSQL database optimized exclusively for JSON workloads.
NoSQL is often faster for local JSON workloads because it stores entire document hierarchies contiguously. Unlike normalized SQL databases that must lock multiple tables and execute computationally expensive `JOIN` operations to reconstruct a nested object, a NoSQL database simply reads or writes the complete JSON Document in a single, atomic disk operation. This significantly reduces I/O wait times and maps flawlessly into local JavaScript application state.
Using JSON as a schema format (specifically the [JSON-Schema standard](https://json-schema.org/)) provides robust type-safety without sacrificing the flexibility of a document store. Because JSON-Schema is machine-readable and heavily utilized by the web ecosystem, tools like [RxDB](../rx-database.md) can parse the schema at runtime to automatically validate document writes, infer TypeScript definitions, and securely compress document keys directly before saving to disk.
## Follow Up
JSON-based databases naturally align with NoSQL because they accommodate evolving, nested data without rigid schemas. This makes them appealing for many UI-centric or offline-first applications where flexible documents and agile development cycles matter more than heavy relational queries or constraints.
SQL can still store JSON, whether in PostgreSQLβs JSONB columns, MySQLβs JSON fields, or SQLiteβs JSON1 extension. For some teams, a hybrid approach pairing SQL for relational data with JSON columns for more flexible fields works well. However, storing everything in a single monolithic JSON text file is rarely advisable for anything beyond trivial tasks because databases excel at concurrency, indexing, and partial writes.
Tools like RxDB provide an even simpler [local-first](./local-first-future.md) take on JSON documents that is particularly well-suited for JavaScript projects. With offline [replication](../replication.md), reactive queries, optional JSON-Schema, and advanced optimizations such as key-compression, RxDB streamlines building dynamic, user-facing features while preserving the core benefits of a robust document database.
To explore more about RxDB and its capabilities for browser database development, check out the following resources:
- [RxDB GitHub Repository](/code/): Visit the official GitHub repository of RxDB to access the source code, documentation, and community support.
- [RxDB Quickstart](../quickstart.md): Get started quickly with RxDB by following the provided quickstart guide, which offers step-by-step instructions for setting up and using RxDB in your projects.
- [RxDB Examples](https://github.com/pubkey/rxdb/tree/master/examples): Browse official examples to see RxDB in action and learn best practices you can apply to your own project - even if jQuery isn't explicitly featured, the patterns are similar.
---
## RxDB - The JSON Database Built for JavaScript
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {CenteredImage} from '@site/src/components/centered-image';
# RxDB - JSON Database for JavaScript
Storing data as **JSON documents** in a **[NoSQL](./in-memory-nosql-database.md)** database is not just a trend; it's a practical choice. JSON data is highly compatible with various tools and is human-readable, making it an excellent fit for modern applications. JSON documents offer more flexibility compared to traditional SQL table rows, as they can contain nested data structures. This article introduces [RxDB](https://rxdb.info/), an open-source, flexible, performant, and battle-tested NoSQL JSON database specifically designed for **JavaScript** applications.
## Why Choose a JSON Database?
- **JavaScript Friendliness**: JavaScript, a prevalent language for web development, naturally uses JSON for data representation. Using a JSON database aligns seamlessly with JavaScript's native data format.
- **Compatibility**: JSON is widely supported across different programming languages and platforms. Storing data in JSON format ensures compatibility with a broad range of tools and systems. All modern programming ecosystems have packages to parse, validate and process JSON data.
- **Flexibility**: JSON documents can accommodate complex and nested data structures, allowing developers to store data in a more intuitive and hierarchical manner compared to SQL table rows. Nested data can be just stored in-document instead of having related tables.
- **Human-Readable**: JSON is easy to read and understand, simplifying debugging and data inspection tasks.
## Storage and Access Options for JSON Documents
When incorporating JSON documents into your application, you have several storage and access options to consider:
- **Local In-App Database with In-Memory Storage**: Ideal for lightweight applications or temporary data storage, this option keeps data in memory, ensuring fast read and write operations. However, data is not persistet beyond the current application session, making it suitable for temporary data storage. With RxDB, the [memory RxStorage](../rx-storage-memory.md) can be utilized to create an in-memory database.
- **Local In-App Database with Persistent Storage**: Suitable for applications requiring data retention across sessions. Data is stored on the user's device or inside of the Node.js application, offering persistence between application sessions. It balances speed and data retention, making it versatile for various applications. With RxDB, a whole range of persistent storages is available. As example, for browser there is the [IndexedDB storage](../rx-storage-indexeddb.md). For server side applications, the [Node.js Filesystem storage](../rx-storage-filesystem-node.md) can be used. There are [many more storages](../rx-storage.md) for React-Native, Flutter, Capacitors.js and others.
- **Server Database Connected to the Application**: For applications requiring data synchronization and accessibility from multiple processes, a server-based database is the preferred choice. Data is stored on a **remote server**, facilitating data sharing, synchronization, and accessibility across multiple processes. It's suitable for scenarios requiring centralized data management and enhanced security and backup capabilities on the server. RxDB supports the [FoundationDB](../rx-storage-foundationdb.md) and [MongoDB](../rx-storage-mongodb.md) as a remote database server.
## Compression Storage for JSON Documents
Compression storage for JSON documents is made effortless with RxDB's [key-compression plugin](../key-compression.md). This feature enables the efficient storage of compressed document data, reducing storage requirements while maintaining data integrity. Queries on compressed documents remain seamless, ensuring that your application benefits from both space-saving advantages and optimal query performance, making RxDB a compelling choice for managing JSON data efficiently. The compression happens inside of the [RxDatabase](../rx-database.md) and does not affect the API usage. The only limitation is that encrypted fields themself cannot be used inside a query.
## Schema Validation and Data Migration on Schema Changes
Storing JSON documents inside of a database in an application, can cause a problem when the format of the data changes. Instead of having a single server where the data must be migrated, many client devices are out there that have to run a migration.
When your application's schema evolves, RxDB provides [migration strategies](../migration-schema.md) to facilitate the transition, ensuring data consistency throughout schema updates.
**JSONSchema Validation Plugins**: RxDB supports multiple [JSONSchema validation plugins](../schema-validation.md), guaranteeing that only valid data is stored in the database. RxDB uses the JsonSchema standardization that you might know from other technologies like OpenAPI (aka Swagger).
```javascript
// RxDB Schema example
const mySchema = {
version: 0,
primaryKey: 'id', // <- define the primary key for your documents
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
name: {
type: 'string',
maxLength: 100
},
done: {
type: 'boolean'
},
timestamp: {
type: 'string',
format: 'date-time'
}
},
required: ['id', 'name', 'done', 'timestamp']
}
```
## Store JSON with RxDB in Browser Applications
RxDB offers versatile storage solutions for browser-based applications:
- **Multiple Storage Plugins**: RxDB supports various storage backends, including [IndexedDB](../rx-storage-indexeddb.md), [localstorage](../rx-storage-localstorage.md) and [In-Memory](../rx-storage-memory.md), catering to a range of browser environments.
- **Observable Queries**: With RxDB, you can create observable [queries](../rx-query.md) that work seamlessly across multiple browser tabs, providing real-time updates and synchronization.
## RxDB JSON Database Performance
Certainly! Let's delve deeper into the performance aspects of RxDB when it comes to working with JSON data.
1. **Efficient Querying:** RxDB is engineered for rapid and efficient querying of JSON data. It employs a well-optimized indexing system that allows for lightning-fast retrieval of specific data points within your JSON documents. Whether you're fetching individual values or complex nested structures, RxDB's query performance is designed to keep your application responsive, even when dealing with large datasets.
2. **Scalability:** As your application grows and your [JSON dataset](./json-based-database.md) expands, RxDB scales gracefully. Its performance remains consistent, enabling you to handle increasingly larger volumes of data without compromising on speed or responsiveness. This scalability is essential for applications that need to accommodate growing user bases and evolving data needs.
3. **Reduced Latency:** RxDB's streamlined data access mechanisms significantly reduce latency when working with JSON data. Whether you're reading from the database, making updates, or synchronizing data between clients and servers, RxDB's optimized operations help minimize the delays often associated with data access. Observed queries are optimized with the [EventReduce algorithm](https://github.com/pubkey/event-reduce) to provide nearly-instand UI updates on data changes.
4. **RxStorage Layer**: Because RxDB allows you to swap out the storage layer. A storage with the most optimal performance can be chosen for each runtime while not touching other database code. Depending on the access patterns, you can pick exactly the storage that is best:
## RxDB in Node.js
Node.js developers can also benefit from RxDB's capabilities. By integrating RxDB into your Node.js applications, you can harness the power of a NoSQL JSON db to efficiently manage your data on the server-side. RxDB's flexibility, performance, and essential features are equally valuable in server-side development. [Read more about RxDB+Node.js](../nodejs-database.md).
## RxDB to store JSON documents in React Native
For mobile app developers working with React Native, RxDB offers a convenient solution for handling JSON data. Whether you're building Android or iOS applications, RxDB's compatibility with JavaScript and its ability to work with JSON documents make it a natural choice for data management within your React Native apps. [Read more about RxDB+React-Native](../react-native-database.md).
## Using SQLite as a JSON Database
In some cases, you might want to use SQLite as a backend storage solution for your JSON data. RxDB can be configured [to work with SQLite](../rx-storage-sqlite.md), providing the benefits of both a relational database system and JSON document storage. This hybrid approach can be advantageous when dealing with complex data relationships while retaining the flexibility of JSON data representation.
## Follow Up
To further explore RxDB and get started with using it in your frontend applications, consider the following resources:
- [RxDB Quickstart](../quickstart.md): A step-by-step guide to quickly set up RxDB in your project and start leveraging its features.
- [RxDB GitHub Repository](https://github.com/pubkey/rxdb): The official repository for RxDB, where you can find the code, examples, and community support.
By embracing [RxDB](https://rxdb.info/) as your **JSON database** solution, you can tap into the extensive capabilities of JSON data storage. This empowers your applications with offline accessibility, caching, enhanced performance, and effortless data synchronization. RxDB's focus on JavaScript and its robust feature set render it the perfect selection for frontend developers in pursuit of efficient and scalable data storage solutions.
---
## Local Database - What It Is and How to Use One in JavaScript
import {Faq, FaqItem} from '@site/src/components/faq';
import {Steps} from '@site/src/components/steps';
import {CenteredImage} from '@site/src/components/centered-image';
import {ComparisonTable} from '@site/src/components/comparison-table';
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
# Local Database
A **local database** stores data directly on the user's device instead of on a remote server. Your application reads and writes through the local database, so every query runs on the device without a network round trip, and the app keeps working when the device goes offline. [RxDB](https://rxdb.info/) is a local database for JavaScript that adds [queries](../rx-query.md), [reactivity](../reactivity.md), and [replication](../replication.md) on top of the raw storage APIs of the browser, mobile, and [Node.js](../nodejs-database.md).
This page explains what a local database is, which options exist in JavaScript, where the raw storage APIs fall short, and how to run a local database in production.
## What is a Local Database?
A local database is a database engine that runs inside the application process on the client device. There is no database server to connect to and no network hop between your code and your data. The engine opens a file or a browser storage API, keeps indexes over the stored records, and answers queries from the same machine the user is holding.
Two properties define a local database:
- **The data lives on the device**: records are written to disk on the client, in [IndexedDB](../rx-storage-indexeddb.md), [localStorage](../rx-storage-localstorage.md), [OPFS](../rx-storage-opfs.md), a [SQLite](../rx-storage-sqlite.md) file, or a plain file on the [filesystem](../rx-storage-filesystem-node.md).
- **The application owns the database**: it starts and stops with your app, it needs no separate process, and it needs no credentials or connection pool.
Because of that, a read is a function call, not a request. The network becomes optional.
When the device is online again, most local databases push the local changes to a backend and pull the remote ones, which is what makes the local copy useful across devices. This is the [offline-first](../offline-first.md) architecture: the local database, not the server, is the gateway for all persistent state changes in your application.
## Local Database vs Remote Database
A remote database runs on a server you operate or rent. Every read and write travels over the network, so latency, packet loss, and downtime are part of every single operation. A local database moves that work to the client.
| Property | Remote Database | Local Database |
| --- | --- | --- |
| Read latency | 50ms to 500ms per query, depending on the network | Under 1ms, no network involved |
| Works offline | β | β |
| Data size | Unlimited, bound by server disk | Bound by device quota, roughly up to 2 GB in browsers |
| Query load | Runs on your servers, scales with user count | Runs on the user's device, scales for free |
| Access control | Enforced in the database | Has to be enforced on the sync backend |
| Multi-user consistency | Strong, one source of truth | Eventual, needs [conflict resolution](../transactions-conflicts-revisions.md) |
| Aggregations over all users | β | β |
The two are not exclusive. Most production apps run both: a local database on the client for everything the user sees, and a remote database on the server as the durable source of truth that all clients replicate against.
## Types of Local Databases in JavaScript
JavaScript runtimes ship several storage APIs, and each has a different tradeoff between size, speed, and query support. RxDB runs on top of all of them through the [RxStorage](../rx-storage.md) layer, so the decision is a configuration change, not a rewrite.
- **[localStorage](./localstorage.md)**: a synchronous key-value store with a limit of about 5 MB per origin. It blocks the main thread and has no indexes, but it is fast for small datasets and available everywhere.
- **[IndexedDB](../rx-storage-indexeddb.md)**: the standard browser database. It is asynchronous, transactional, supports secondary indexes, and stores [much more data](./indexeddb-max-storage-limit.md) than localStorage. The raw API is [low level and slow](../slow-indexeddb.md) for bulk operations.
- **[OPFS](../rx-storage-opfs.md)**: the Origin Private File System gives you file handles inside the browser sandbox. It is the fastest browser persistence option, and it needs a [Web Worker](../rx-storage-worker.md) for the synchronous access handles.
- **[SQLite](../rx-storage-sqlite.md)**: the default local database on mobile and desktop. It is used in [React Native](../react-native-database.md), [Capacitor](../capacitor-database.md), and [Electron](../electron-database.md), and it also runs in the browser compiled to WebAssembly.
- **[Filesystem](../rx-storage-filesystem-node.md)**: in Node.js, Deno, and Bun a local database can simply write to disk in the same process. See the [Node.js database](../nodejs-database.md) page.
- **[Memory](../rx-storage-memory.md)**: a non-persistent [in-memory database](./in-memory-nosql-database.md) for tests, short sessions, and caching layers.
A deeper comparison of the browser options with benchmarks is in the [browser storage](./browser-storage.md) overview and in the [localStorage vs IndexedDB vs OPFS vs SQLite](./localstorage-indexeddb-cookies-opfs-sqlite-wasm.md) article.
## Where Local Databases Are Used
- **Offline functionality**: field service tools, note apps, and [offline-first CRMs](./offline-database.md) have to stay usable in a basement, on a plane, or in a truck. The user keeps writing, and the changes sync later.
- **Zero-latency interfaces**: when the data is already on the device, a click updates the UI in the same frame. There is no spinner, and no [optimistic UI](./optimistic-ui.md) hack is needed, because the write is real and local.
- **Realtime and collaboration**: chat apps, dashboards, and shared editors observe the local database and re-render when a change arrives, no matter whether it came from the user, from [another browser tab](../leader-election.md), or from the [replication](../replication.md).
- **Reduced backend cost**: every query answered on the client is a query your server never runs. This is one of the strongest arguments for a [local-first architecture](./local-first-future.md) at scale.
- **[Progressive Web Apps](./progressive-web-app-database.md)**: a service worker caches the code, and a local database caches the state. Together they make a web app behave like a native one.
- **Privacy**: data that is processed on the device does not have to leave it. Fields that must stay secret can be [encrypted](../encryption.md) at rest.
## Where the Raw Storage APIs Fall Short
IndexedDB, localStorage, and SQLite are storage engines. They store bytes and give them back. The trouble starts when you build an actual application on top of them.
### 1. Queries
localStorage has no query support at all, so you end up parsing JSON and filtering arrays by hand. IndexedDB has indexes and cursors, but no query language: a filter over two fields with a sort is dozens of lines of cursor code, and you have to pick the right index yourself. RxDB gives you [MongoDB-style (Mango) queries](../rx-query.md) with a [query planner](../query-optimizer.md) that selects the index for you.
### 2. Reactivity
The raw APIs are request and response. When a document changes, nothing tells your UI. Most apps work around this with manual refetching after every write, which misses changes from other tabs and from the sync process. A local database with [observable queries](../reactivity.md) emits a new result set whenever a matching document changes, and RxDB uses the [EventReduce algorithm](https://github.com/pubkey/event-reduce) to compute the new result on the CPU instead of re-running the query.
### 3. Schemas and Migrations
Every client device carries its own copy of the data, so a schema change has to run on every device, at unpredictable times, and possibly across several app versions at once. Doing this by hand is where local-first projects lose data. RxDB validates documents against a [JSON schema](../rx-schema.md) and runs versioned [migrations](../migration-schema.md) on startup.
### 4. Synchronization and Conflicts
Two users edit the same document while both are offline. When they reconnect, someone has to decide what the document looks like now. A transaction cannot help here, because it is not possible to hold a lock across maybe-offline client devices. You need [revisions, checkpoints, and a conflict handler](../transactions-conflicts-revisions.md). RxDB ships this as the [Sync Engine](../replication.md), with plugins for [HTTP](../replication-http.md), [WebSocket](../replication-websocket.md), [GraphQL](../replication-graphql.md), [CouchDB](../replication-couchdb.md), [Firestore](../replication-firestore.md), [NATS](../replication-nats.md), and [peer-to-peer WebRTC](../replication-webrtc.md).
### 5. Encryption
IndexedDB writes plain text to the user's disk. There is no flag to turn that off. Anyone with file access to the profile folder can read every record. The [encryption plugin](../encryption.md) encrypts the fields you flag before they hit the disk and decrypts them on read, which matters for tokens, health data, and anything else you would not want on a stolen laptop. The details are in the [IndexedDB encryption](./indexeddb/indexeddb-encryption.md) guide.
### 6. Multi-Tab Behavior
A user opens your app in three tabs. Each tab has its own JavaScript process and its own view of the data, and each one runs its own replication. RxDB elects a [leader tab](../leader-election.md) so the sync runs once, and broadcasts changes to the other tabs so all of them stay consistent.
## How to Use RxDB as Your Local Database
RxDB (Reactive Database) is a local-first, NoSQL database for JavaScript applications. It runs in the browser, Node.js, Electron, React Native, Capacitor, Deno, and Bun. The following setup gives you a persistent local database with typed documents, reactive queries, and a sync target.
### Install RxDB
```bash
npm install rxdb rxjs
```
### Create the Database
Pick an [RxStorage](../rx-storage.md) for your runtime. The localStorage-based storage is the simplest browser default, and swapping it for [IndexedDB](../rx-storage-indexeddb.md), [OPFS](../rx-storage-opfs.md), or [SQLite](../rx-storage-sqlite.md) later is a one-line change.
```ts
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageLocalstorage()
});
```
### Define a Schema
The schema is [JSON schema](../rx-schema.md). It defines the fields, the indexes, and the primary key, and RxDB uses it to validate every write.
```ts
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
// the primary key must have a maxLength
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
done: { type: 'boolean' },
timestamp: { type: 'string', format: 'date-time' }
},
required: ['id', 'name', 'done', 'timestamp']
}
}
});
```
### Write and Query Locally
Inserts and queries run on the device. There is no `await fetch()` in this code path, so the numbers are microseconds, not milliseconds.
```ts
await db.todos.insert({
id: 'todo1',
name: 'Use a local database',
done: false,
timestamp: new Date().toISOString()
});
const openTodos = await db.todos.find({
selector: { done: { $eq: false } }
}).exec();
// > [RxDocument]
```
### Observe the Data
Subscribe to a query and the callback fires again on every change, whether it came from this tab, another tab, or the replication.
```ts
db.todos.find({
selector: { done: { $eq: false } }
}).$.subscribe(openTodos => {
// re-render the list, the local database pushed the update
console.log('open todos: ' + openTodos.length);
});
```
### Sync With a Backend
The [replication](../replication.md) runs in the background. Your UI keeps reading from the local database while the sync catches up.
```ts
import { replicateHTTP } from 'rxdb/plugins/replication-http';
replicateHTTP({
collection: db.todos,
replicationIdentifier: 'todos-http-replication',
live: true,
pull: {
handler: async (checkpoint) => fetch(
'https://example.com/api/todos/pull?' +
new URLSearchParams({ checkpoint: JSON.stringify(checkpoint) })
).then(res => res.json())
},
push: {
handler: async (rows) => fetch('https://example.com/api/todos/push', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(rows)
}).then(res => res.json())
}
});
```
The same code runs in [React](./react-database.md), [Angular](./angular-database.md), [Vue](./vue-database.md), and Svelte. Only the binding between the observable and the component changes.
## Local Database Performance
The main performance win of a local database is that the network is gone. What remains is the difference between the storage engines, and that difference is large. The chart below shows the same operations run against different browser storages (lower is better).
You can reproduce these numbers with the [performance test suite](../rx-storage-performance.md). Three things matter most in practice:
- **Batch your writes**: one bulk write of 500 documents is much cheaper than 500 single writes, because each transaction has a fixed overhead.
- **Index what you sort and filter on**: an unindexed query has to scan the whole collection, and on a client device that scan happens on the same thread that paints the UI.
- **Keep the dataset bounded**: sync only the documents a user needs, and run the [cleanup plugin](../cleanup.md) so deleted documents do not pile up.
For large datasets, [key compression](../key-compression.md) saves up to 40% disk space, and moving the storage into a [Web Worker](../rx-storage-worker.md) keeps the main thread free.
## When a Local Database Is the Wrong Choice
A local database is not free. Be honest about the cases where a server-side database is simply the better tool:
- **The dataset does not fit on the device.** Browsers cap storage per origin, and syncing gigabytes to every client is not realistic. Local-first works when the per-user dataset is bounded, usually below 2 GB.
- **You need aggregations over all users.** Reports across the whole dataset belong on the server, because no client has all of the data.
- **The data must never be on the client.** Anything on a user's device can be extracted from it. Encryption raises the bar, but data the user must never see should not be replicated to them.
- **Strong consistency is a hard requirement.** Bank transfers and seat reservations need a single authority. Offline clients cannot provide one.
For everything else, the [downsides of offline-first](../downsides-of-offline-first.md) page lists the tradeoffs in detail.
## FAQ
A **local database** is a database that runs on the user's own device inside the application process, instead of on a remote server. It stores records in browser storage like [IndexedDB](../rx-storage-indexeddb.md) or in a file such as [SQLite](../rx-storage-sqlite.md), and it answers queries without any network access. Because there is no round trip, reads and writes complete in under a millisecond, and the application keeps working when the device is offline.
Instant data access without a network. Queries and writes are handled on the device, so the UI updates immediately and the app stays usable during connection drops. You also move the query load off your servers and onto the user's hardware, which reduces backend cost and bandwidth. An [offline-first](../offline-first.md) application requires a local database to function without a network connection.
A **local database** runs on the user's device and answers every query locally. A **cloud database** runs on remote servers, needs an active connection for each request, and is centralized. Local databases give you zero latency, offline capability, and cheap horizontal scaling because each client does its own work. Cloud databases give you unlimited storage, aggregations across all users, and strong consistency. Most production apps use both and connect them with [replication](../replication.md).
For small datasets, the [localStorage RxStorage](../rx-storage-localstorage.md) is the simplest option with the smallest bundle. For anything bigger, use an [IndexedDB](../rx-storage-indexeddb.md) or [OPFS](../rx-storage-opfs.md) based storage, because they store far more data and do not block the main thread. **[RxDB](../rx-database.md)** runs on all of them through the [RxStorage](../rx-storage.md) layer, so you can start with localStorage and switch later without changing your application code.
Yes. Working offline is the reason local databases exist. All reads and writes go to the device, so the app behaves the same with or without a connection. The changes made while offline are queued and sent to the backend by a background [replication](../replication.md) process once connectivity returns, and any [conflicts](../transactions-conflicts-revisions.md) are resolved by a conflict handler you define.
It depends on the runtime. `localStorage` is limited to about 5 MB per origin. IndexedDB and OPFS use a quota derived from free disk space, which in Chrome is a percentage of the disk and in Safari is stricter, as described in the [IndexedDB storage limit](./indexeddb-max-storage-limit.md) article. On mobile and desktop, [SQLite](../rx-storage-sqlite.md) is bound only by the device's disk. As a planning number, keep the per-user dataset below 2 GB.
No, not by default. IndexedDB, localStorage, and plain SQLite files store data as plain text on disk, and anyone with file access to the device can read them. The RxDB [encryption plugin](../encryption.md) encrypts the fields you mark in the schema before they are written and decrypts them on read. See the [IndexedDB encryption](./indexeddb/indexeddb-encryption.md) guide for how this works in the browser.
An **embedded database** (such as [SQLite](../rx-storage-sqlite.md) or [RxDB](../rx-database.md)) is linked into the application itself instead of running as a separate service. Use one for client-side applications such as mobile apps, [Electron](../electron-database.md) desktop binaries, or [Progressive Web Apps](./progressive-web-app-database.md) that need low-latency data access and offline behavior, and when you want to avoid operating a separate database cluster. See the [embedded database](./embedded-database.md) article for details.
For JavaScript and TypeScript applications, **[RxDB](../rx-database.md)** provides offline-first synchronization with automated [conflict resolution](../transactions-conflicts-revisions.md) against [CouchDB](../replication-couchdb.md), [GraphQL](../replication-graphql.md), [HTTP](../replication-http.md) endpoints, or peer-to-peer networks via [WebRTC](../replication-webrtc.md). Other options in the ecosystem are PouchDB, WatermelonDB, and cloud SDKs like Firebase Firestore and Supabase. A comparison of them is in the [alternatives](../alternatives.md) list.
For traditional server clusters, PostgreSQL or MongoDB are the standard. For [Node.js](../nodejs-database.md) tools, edge deployments, and standalone applications, an embedded engine like **[SQLite](../rx-storage-sqlite.md)** or **[RxDB's filesystem storage](../rx-storage-filesystem-node.md)** gives you low-latency access inside the same process, without an external database dependency.
A **document-oriented database** such as RxDB stores data as [JSON documents](./json-database.md), which map directly onto JavaScript objects and tolerate evolving data models. A **relational local database** such as [SQLite](../rx-storage-sqlite.md) organizes data into rows and columns with a fixed schema and is optimized for JOIN queries. For client-side applications, documents usually win because serialization to the UI and to the sync protocol is trivial. The reasoning is explained on the [why NoSQL](../why-nosql.md) page.
## Follow Up
- Build a working local database in a few minutes with the [Quickstart Tutorial](../quickstart.md).
- Read how a local database changes the architecture of an app in the [local-first](./local-first-future.md) article.
- Compare RxDB with [other local database solutions](../alternatives.md) to find the fit for your requirements.
- Check the code on [GitHub](/code/) and leave a star β when RxDB is useful for you.
- Ask questions in the [community chat](/chat/).
---
## Why Local-First Software Is the Future and its Limitations
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {QuoteBlock} from '@site/src/components/quoteblock';
import {VideoBox} from '@site/src/components/video-box';
import {Faq, FaqItem} from '@site/src/components/faq';
# Why Local-First Software Is the Future and what are its Limitations
Imagine a web app that behaves seamlessly even with zero internet access, provides sub-millisecond response times, and keeps most of the user's data on their device. This is the **local-first** or [offline-first](../offline-first.md) approach. Although it has been around for a while, local-first has recently become more practical because of **maturing browser storage APIs** and new frameworks that simplify **data synchronization**. By allowing data to live on the client and only syncing with a server or other peers when needed, local-first apps can deliver a user experience that is **fast, resilient**, and **privacy-friendly**.
However, local-first is no silver bullet. It introduces tricky distributed-data challenges like conflict resolution and schema migrations on client devices. In this article, we'll dive deep into what local-first means, why it's trending, its pros and cons, and how to implement it in real applications. We'll also discuss other tools, criticisms, backend considerations, and how local-first compares to traditional cloud-centric approaches.
## What is the Local-First Paradigm
In **local-first** software, the primary copy of your data lives on the **client** rather than a remote server. Rather than sending each read or write over the network, you store and manipulate data in a [local database](./local-database.md) on the userβs device. Sync then happens in the background, ensuring all devices eventually converge to a consistent state.
This approach is increasingly popular because it leads to **instant** app responses (no network delay for most operations), genuine **offline capability**, and more direct **data ownership** for users. Local-first apps also sidestep outages and if the server or internet goes down, users can keep working with their local data. When connectivity returns, everything syncs. This makes the user experience **more resilient** and gives them control of their data, which is especially appealing when privacy concerns or limited connectivity are key factors.
Local-First software: A set of principles for software that enables both collaboration and ownership for users. Local-first ideals include the ability to work offline and collaborate across multiple devices, while also improving the security, privacy, long-term preservation, and user control of data.
## Why Local-First is Gaining Traction
The push for local-first is driven by a few key new technological capabilities that previously restricted client devices from running heavy local-first computing:
- **Relaxed Browser Storage Limits**: In the past, true local-first web apps were not very feasible due to **storage limitations** in browsers. Early web storage options like cookies or [localStorage](./localstorage.md#understanding-the-limitations-of-local-storage) had tiny limits (~5-10MB) and were unsuitable for complex data. Even **IndexedDB**, the structured client storage introduced over a decade ago, had restrictive quotas on many browsers: For example, older Firefox versions would **prompt the user if more than 50MB** was being stored. Mobile browsers often capped IndexedDB to 5MB without user permission. Such limits made it impractical to cache large application datasets on the client. However, modern browsers have dramatically [increased these limits](./indexeddb-max-storage-limit.md). Today, IndexedDB can typically store **hundreds of megabytes to multiple gigabytes** of data, depending on device capacity. Chrome allows up to ~80% of free disk space per origin (tens of GB on a desktop), Firefox now supports on the order of gigabytes per site (10% of disk size), and even Safari (historically strict) permits around 1GB per origin on iOS. In short, the storage quotas of 5-50MB are a thing of the past and modern web apps can cache very large datasets locally without hitting a ceiling. This shift in storage capabilities has unlocked new possibilities for **local-first web apps** that simply weren't viable a few years ago.
- **New Storage APIs (OPFS)**: The new Browser API [Origin Private File System](../rx-storage-opfs.md) (OPFS), part of the File System Access API, enables near-native file I/O from within a browser. It allows web apps to manage file handles securely and perform fast, synchronous reads/writes in Web Workers. This is a huge deal for local-first computing because it makes it feasible to embed robust database engines directly in the browser, persisting data to real files on a virtual filesystem. With OPFS, you can avoid some of the performance overhead that comes with [IndexedDB-based workarounds](../slow-indexeddb.md), providing a near-native [speed experience](./localstorage-indexeddb-cookies-opfs-sqlite-wasm.md#big-bulk-writes) for file-structured data.
- **Bandwidth Has Grown, But Latency Is Capped**: Internet infrastructure has rapidly expanded to provide higher throughput making it possible to transfer large amounts of data more quickly. However, latency (i.e., round-trip delay) is constrained by the **speed of light** and other physical limitations in fiber, satellite links, and routing. We can always build out bigger "pipes" to stream or send bulk data, but we can't significantly reduce the base round-trip time for each request. This is a physical limit, not a technological one. Local-first strategies mitigate this fundamental latency limit by avoiding excessive client-server calls in interactive workflows, once data is on the client, it's instantly available for reads and writes without waiting on a network round-trip. Imagine, transferring **around 100,000** "average" JSON documents might only consume **about the same bandwidth as two frames of a 4K YouTube video** which can be transferred in milliseconds. This shows just how far raw data throughput has come. Yet each request still has a 100-200ms latency or more, which becomes noticeable in user interactions. Local-first mitigates this by minimizing round-trip calls during active use and using the available bandwidth to directly transfer most of the data on the first app start.
- **WebAssembly**: Another advancement is **WebAssembly (WASM)**, which allows developers to compile low-level languages (C, C++, Rust) for execution in the browser at near-native speed. This means database engines, search algorithms, [vector databases](./javascript-vector-database.md), and other performance-heavy tasks can run right on the client. However, a key limitation is that **WASM cannot directly access persistent storage APIs** in the browser. Instead, all data must be sent from WASM to JavaScript (or the main thread) and then go through something like IndexedDB or OPFS. This extra indirection [is slower](./localstorage-indexeddb-cookies-opfs-sqlite-wasm.md) compared to plain JavaScript->storage calls. Looking ahead, there might come up future APIs that allow WASM to interface with persistent storage directly, and if those land, local-first systems could see another major boost in [performance](../rx-storage-performance.md).
- **Improvements in Local-First Tooling**: A major factor fueling the rise of local-first architectures is the **dramatic leap in client-side tooling and performance**. For instance, consider a local-first **email client** that stores **one million messages**. In 2014, searching through that many documents, especially with something like early PouchDB, could take **minutes** in a browser. Today, with advanced offline databases like **RxDB**, you can use the [OPFS storage](../rx-storage-opfs.md) with [sharding](../rx-storage-sharding.md) across multiple [web workers](../rx-storage-worker.md) (one per CPU) and use [memory-mapped](../rx-storage-memory-mapped.md) techniques. The result is a **regex search** of one million of these email documents in around **120 milliseconds** - all in JavaScript, running inside a standard web browser, on a mobile phone.
Better yet, this performance ceiling is likely to keep rising. Newer browser features and **WebAssembly** optimizations could enable even faster indexing and query operations, closing the gap with native desktop clients. I even experimented with GPU-accelarated queries (using **WebGPU**) which, while still in experimental stage, might deliver client-side performance that outperforms servers which do not have a graphics card.
These transformations highlight why local-first has become truly practical: not only can you sync and work offline, but you can handle **serious data loads** with performance that would have been unthinkable just a few years ago.
## What you can expect from a Local First App
[Jevons' Paradox](https://en.wikipedia.org/wiki/Jevons_paradox) says that making a _resource cheaper or more efficient to use often leads to greater overall consumption_. Originally about coal, it applies to the local-first paradigm in a way where we require apps to have more features, simply because it is technically possible, the app users and developers start to expect them:
### User Experience Benefits
- **Performance & UX:** Running from local storage means **low latency** and instantaneous interactions. There's no round-trip delay for most operations. Local-first apps aim to provide [near-zero latency](./zero-latency-local-first.md) responses by querying a [local database](./local-database.md) instead of waiting for a server responseβ. This results in a snappy UX (often no need for loading spinners) because data reads/writes happen immediately on-device. Modern users expect real-time feedback, and local-first delivers that by default.
- **User Control & Privacy:** Storing data locally can limit how much sensitive information is sent off to remote servers. End users have greater control over their data, and the app can implement [client-side encryption](../encryption.md), thereby reducing the risk of mass data breaches. Its even possible to only replicated encrypted data with a server so that the backend does not know about the data at all and just acts as a backup/replication endpoint.
- **Offline Resilience:** Obviously, being able to work offline is a major benefit. Users can continue using the app with no internet (or flaky connectivity), and their changes sync up once online. This is increasingly important not just for remote areas, but for any app that needs to be available 24/7. Even though mobile networks have improved, connectivity can still drop; local-first ensures the app doesn't grind to a halt. The app _"stores data locally at the client so that it can still access it when the internet goes away."_
- **Realtime Apps**: Today's users expect data to stay in sync across browser tabs and devices without constant page reloads. In a typical cloud app, if you want real-time updates (say to show that a friend edited a document), you'd need to implement a [websocket or polling](./websockets-sse-polling-webrtc-webtransport.md) system for the server to push changes to clients, which is complex. Local-first architectures naturally lend themselves to realtime-by-default updates because the application state lives in a local database that can be observed for changes. Any edits (local or incoming from the server) immediately trigger [UI updates](./optimistic-ui.md). Similarly, background sync mechanisms ensure that new server-side data flows into the local store and into the user interface right away, no need to hit F5 to fetch the latest changes like on a traditional webpage.
### Developer Experience Benefits
- **Reduced Server Load**: Because local-first architectures typically **transfer large chunks of data once** (e.g., during an initial sync) and then sync only small diffs (delta changes) afterward, the server does not have to handle repeated requests for the same dataset. This bulk-first, diff-later approach drastically decreases the total number of round-trip requests to the backend. In scenarios where hundreds of simultaneous users each require continuous data access, an offline-ready client that only periodically sends or receives changes can scale more efficiently, freeing your servers to handle more users or other tasks. Instead of being bombarded with frequent small queries and updates, the server focuses on periodic sync operations, which can be more easily optimized or batched. It **Scales with Data, Not Load**. In fact for most type of apps, most of the data itself rarely changes. Imagine a CRM system. How often does the data of a customer really change compared to how often a user opens the customer-overview page which would load data from a server in traditional systems?
- **Less Need for Custom API Endpoints**: A local-first architecture often simplifies backend design. Instead of writing extensive REST routes for each client operation (create, read, update, delete, etc.), you can build a **single replication endpoint** or a small set of endpoints to handle data synchronization for each entity. The client manages local data, merges edits, and pushes/pulls changes with the server automatically. This not only **reduces boilerplate code** on the backend but also **frees developers** to focus on business logic and domain-specific concerns rather than spending time creating and maintaining dozens of narrowly scoped endpoints. As a result, the overall system can be easier to scale and maintain, delivering a **smoother developer experience**.
- **Simplified State Management in Frontend**: Because the local database holds the authoritative state, you might rely less on complex state management libraries (Redux, MobX, etc.). The DB becomes a single source of truth for your UI. In an offline-first app, your global state is already there in a single place stored inside of the local database, so you don't need as many in-memory state layers to synchronizeβ. The UI can directly bind to the database (using queries or reactive subscriptions). All sources of changes (user input, remote updates, other tabs) funnel through the database. This can significantly reduce the "glue code" to keep UI state in sync with server state because the local DB does that for you. With Local-First tools, "you might not need Redux" because the reactive DB fulfills that roleβ of state management already.
- **Observable Queries**: One of the big advantages of storing data locally is the ability to **subscribe** to data changes in real time, often called **observable queries**. When the data changes - either from the user's actions or from a remote sync - the local database automatically updates the subscribed query results, and the UI can redraw without a manual refresh. This reactive pattern can make apps feel much more live and responsive.
In the beginnings this was mostly done by watching a changes feed (like in PouchDB) and **re-running queries** whenever data changed. However, this early approach was **slow and didn't scale well**, because the entire query had to be recalculated each time. Later, RxDB introduced the [EventReduce Algorithm](https://github.com/pubkey/event-reduce), which merges incoming document changes into an existing query result by using a big [binary decision tree](https://github.com/pubkey/binary-decision-diagram). With this, updated query results can be "calculated" on the CPU instead of re-running them over the database. This makes query updates almost instantaneous and scales better as your data grows. Nowadays, many local databases have added similar features: for example, **dexie.js** introduced `liveQuery`, letting developers build real-time UIs without repeatedly scanning the entire dataset.
- **Better Multi-Tab and Multi-Device Consistency**: Because the source of truth is on the client, if the user has the app open in multiple tabs or even multiple devices, each has a full copy of the data. In browsers, many offline databases use a storage like IndexedDB that is shared across tabs of the same origin. This means all tabs see the same up-to-date local state. For example, if a user logs in or adds data in one tab, the other tab can automatically reflect that change via the shared local DB and eventsβ. This solves a common issue in web apps where one tab doesn't know that something changed in another tab. With local-first, **multi-tab just works** by default because there's "exactly one state of the data across all tabs". Similarly, on multiple devices, once sync runs, each device eventually converges to the same state.
> If your users have to press F5 all the time, your app is broken!
- **Potential for P2P and Decentralization**: While most current local-first apps still use a central backend for syncing, the paradigm opens the door to [peer-to-peer data syncing](../replication-webrtc.md). Because each device has the full data, devices could sync directly via LAN or other P2P channels for collaboration, reducing reliance on central servers. There are experimental frameworks that allow truly decentralized sync. This is a more advanced benefit, but it aligns with the ethos of giving users more control and reducing dependence on any one cloud provider.
These advantages show why developers are excited about local-first. You get happier users thanks to a fast, offline-capable app, and you can differentiate your product by working in scenarios where others fail (e.g. poor connectivity). Companies like Google, Amazon, etc., invest heavily in reducing latency and adding offline modes for a reason: it improves retention and usability. Local-first design takes that to the extreme by default.
## Challenges and Limitations of Local-First
However, this approach is not without significant challenges. It's important to understand the drawbacks and trade-offs before deciding to go all-in on local-first. Let's examine the flip side.
You fully understood a technology when you know when not to use it
Critics of local-first approaches often point out these challenges. Here's a comprehensive list of cons, criticisms, and obstacles associated with local-first development and proposed solutions on how to solve these obstacles:
- **Data Synchronization**: Data synchronization is arguably the hardest part of local-first development, because when every user's device can be offline for extended periods, data inevitably diverges. Ensuring those changes propagate and reconcile with minimal user headaches is a major challenge in distributed systems. Two main approaches have emerged:
- **Use a bundled frontend+backend solution** where the backend is tightly coupled to the client SDK and knows exactly how to handle sync. A common example is Firestore (part of the Firebase ecosystem) where Google's servers and client libraries collectively manage storage, change detection, conflict resolution, and syncing. The upside is you have a turnkey solution, developers can focus on features rather than writing sync logic. The downside is lock-in, because the sync protocol is proprietary and tailored to that vendor. In many organizations, this is a non-starter: existing company infrastructure can't be uprooted or replaced just for a single offline-capable app. This lock-in issue can arise even if the backend is not strictly a third-party vendor but simply another technology like PostgreSQL, because it still forces you to consolidate all your data into a single system that you might not use otherwise.
- **Custom Replication with Your Own Endpoints**: Alternatively, tools like RxDB allow you to implement your own replication endpoints on top of your existing infrastructure. This is the approach relies on a lightweight, git-like [Sync Engine](../replication.md). The server remains relatively "dumb," focusing on storing revisions, tracking changes, and marking conflicts, while the client library does the actual [conflict resolution](../transactions-conflicts-revisions.md). During sync, if the server detects a conflict (e.g., two offline edits to the same document), it notifies the client, which then decides how to merge them, whether via last-write-wins, a custom merge function, or a [CRDT](../crdt.md). Setting up custom endpoints does require more development effort, but you avoid vendor lock-in and can integrate seamlessly with your existing database(s). Your system simply needs to support incrementally fetching changes (pull) and accepting local modifications (push), which can be layered on top of nearly any data store or architecture.
Tools which support "any backend" are of course harder to monetize because they cannot sell SaaS services or a Cloud Subscription which is why most tools use a fixed backend instead of an open Sync Engine.
- **Conflict Resolution**: When multiple offline edits happen on the same data, you inevitably get **merge conflicts**. For example, if two users (or the same user on two devices) both edit the same document offline, when both sync, whose changes win? Local-first systems need a conflict resolution strategy. Some systems use **last-write-wins** (firestore) or deterministic revision hashing to pick a "winner" (as in CouchDB/PouchDB)β. This is simple but may drop one user's changes. Other approaches keep both versions and merge them either via an implement ["merge-function"](../transactions-conflicts-revisions.md#custom-conflict-handler) or require a **manual merge** step (e.g., like git conflicts or showing the user a diff UI). More advanced solutions involve **CRDTs (Conflict-free Replicated Data Types)** which mathematically merge changes (used for rich text collaboration, for instance). Libraries like Automerge or Yjs implement CRDTs to "magically solve conflicts". But in practice, using CRDTs is also complex and has its own trade-offs and sometimes not even possible like when you need additional data from another instance for a "correct" merge. No matter which route, handling conflicts adds complexity to your app logic or infrastructure. In cloud-based (online-first) apps, you avoid this because everyone is always editing the single up-to-date copy on the server. Local-first shifts that burden to the client side.
Here is an example on how a client-side merge functions works in RxDB:
```ts
import { deepEqual } from 'rxdb/plugins/utils';
export const myConflictHandler = {
/**
* isEqual() is used to detect if two documents are
* equal. This is used internally to detect conflicts.
*/
isEqual(a, b) {
/**
* isEqual() is used to detect conflicts or to detect if a
* document has to be pushed to the remote.
* If the documents are deep equal,
* we have no conflict.
* Because deepEqual is CPU expensive,
* on your custom conflict handler you might only
* check some properties, like the updatedAt time or revision-strings
* for better performance.
*/
return deepEqual(a, b);
},
/**
* resolve() a conflict. This can be async so
* you could even show an UI element to let your user
* resolve the conflict manually.
*/
async resolve(i) {
/**
* In this example we drop the local state and use the server-state.
* This basically implements a "first-on-server-wins" strategy.
*
* In your custom conflict handler you could want to merge properties
* of the i.realMasterState, i.assumedMasterState and i.newDocumentState
* or return i.newDocumentState to have a "last-write-wins" strategy.
*/
return i.realMasterState;
}
};
```
- **Eventual Consistency (No Single Source of Truth):** A local-first system is **eventually consistent** by nature. There is no single authoritative copy of the data at all times. Instead you have one per device (and maybe one on the server), and they sync to become consistent eventually. This means at any given moment, two users might not see the same data if one hasn't synced recently. Users could even make decisions based on stale data. In many applications this is acceptable (the data will catch up), but for some scenarios it's problematic. For instance, an offline-first banking app that lets you initiate a money transfer offline could be dangerous if the account balance was out-of-date or if the transfer needs immediate consistency. Essentially, **not all apps can tolerate eventual consistency**. If your use case demands strong consistency (e.g., inventory systems where overselling is a big issue, or real-time collaborative editing where every keystroke must be seen by others instantly), a purely local-first approach might need augmentation or may not fit.
- **Initial Data Load and Data Size Limits:** Local-first requires pulling data **down to the client**. If your dataset is huge (gigabytes), it's simply not feasible to download everything to every client. For example, syncing every tweet on Twitter to every user's phone is impossible. Local-first works best when the data set per user is reasonably sized (up to 2 Gigabytes). In practice, you often **limit the data** to just that user's own data or a subset relevant to them. Even then, on first use the app might need to download a significant chunk of data to initialize the local database. There is a **upper bound on dataset size** beyond which the initial sync or storage needs become impractical. You cannot assume unlimited local storage. If your data is too large, local-first will either fail or you'll need to only sync partial data (and then handle what happens if the needed data isn't present locally). In short, **local-first is unsuitable for massive datasets** or data that cannot be partitioned per user.
- **Storage Persistence (Browser Limitations):** Storing data in the browser (via IndexedDB or similar) is not as durable as on a server disk. Browsers may **evict data** to save space (especially on mobile devices). For instance, Safari notoriously wipes out IndexedDB data if the user hasn't used the site in ~7 days. Other browsers have their own eviction policies for offline data. Also, users can at any time clear their browser storage (intentionally or via something like "Clear site data"). This means the local data **cannot be 100% trusted to stay forever**. A well-behaved local-first app needs to be able to recover if local data is lost, usually by pulling from the server againβ. Essentially, the server still often serves as a backup. But if your app had any purely local data (not intended to sync), that's at risk. **Mobile apps** (with SQLite or filesystem storage) are a bit more stable than web browsers, but even there, uninstalls or certain OS actions can remove local data. This is a challenge: How to cache data offline for speed while ensuring if it's wiped, the user doesn't lose everything important. Cloud-only apps by contrast keep data in the cloud so it's typically safe unless the server fails (and servers are easier to backup reliably).
- **Complex Client-Side Logic & Increased App Size**: A local-first app tends to be more complex on the client side. You're essentially putting what used to be server responsibilities (storage, query engine, sync logic) into the frontend. This can increase the size of your frontend bundle (including a database library, possibly CRDT or sync code, etc.). It also increases memory and CPU usage on the client, as the browser/phone is doing more work. Low-end devices or older phones might struggle if the app is not optimized. Developers need to consider performance tuning for the local database (indexing, query efficiency) just like they would on a server. So while the user gains benefits, the app developer has to manage this complexity.
- **Performance Constraints in JavaScript:** Even though devices are fast, a local JS database is generally **slower than a server DB** on robust hardware. There are many layers (JS -> IndexedDB -> possibly SQLite under the hood) that add overheadβ. For example, inserting a record might go through the DB library, the storage engine, the browser's implementation, down to disk. For most UI uses this is fine (you don't need 10k writes/sec in a to-do app, you need maybe a few writes per second at most). But if your app does heavy data processing, the browser might become a bottleneck.
The key question is _"Is it fast enough?"_. Often the answer is yes for typical usage, but developers must be mindful of not doing something on the client that truly requires big iron servers. For instance, full-text indexing of a million documents might be too slow in a client-side DB. **Unpredictable performance** is also a factor: Different users have different devices. A query that takes 50ms on a high-end desktop might take 500ms on a low-end phone in battery saving mode. So performance tuning and testing across devices is needed, and some heavy tasks might still belong on the server sideβ. For example if you build a [local vector database](./javascript-vector-database.md) you might want to create the embeddings on the server and sync them instead of creating them on the client.
- **Client Database Migrations:** As your app evolves, you'll change data models or add new fields. In a cloud-first app, you'd typically run a migration on the server database. In a local-first app, you have not only the server DB (if any) but also every client's local database to consider. Upgrading the schema means you need to write migration logic that runs on each client, perhaps the next time they launch the app after an update. Clients may be offline or not upgrade the app immediately, so you could have different versions of the schema in the wild. This complicates data handling (the sync protocol might need to handle multiple schema versions until everyone is updated). Providing a smooth migration path for local data is doable (many libraries provide [migration facilities](../migration-schema.md)), but it requires careful testing. In a worst case, a failed migration on a client could brick the app for that user or force a full resync. This is a **much bigger headache** than just migrating a centralized DB at midnight while your service is in maintenance mode. π
- **Security and Access Control:** In cloud-based apps, enforcing data security (who can see what) is done on the server. The client only gets the data it's authorized to get. In a local-first scenario, you often need to **partition data per user** on the backend as well, to ensure users only sync down their own data (or data they have permission for). One simple strategy is to give each user their own database or dataset on the server and only replicate that. For example, CouchDB allows creating one database per user and replication can be scoped to that DB which makes permission handling easy. But if you ever need to query across users (say an admin view or aggregate analytics), having data split into many small DBs becomes a pain. The alternative is a single backend database with a **fine-grained access control**, and the client asks to sync only certain documents/fields. That usually means writing a custom sync server or using something like GraphQL with resolvers that respect permissions. In short, **implementing auth and permissions in sync** adds complexity. Also, any data stored on the client is theoretically vulnerable to extraction (if someone compromises the device or uses dev tools). You can [encrypt local databases](../encryption.md) to prevent extraction after the server "revokes" the decryption password to mitigate the data extraction risk.
- **Relational Data and Complex Queries:** Most client-side/offline databases are [NoSQL/document oriented](../why-nosql.md) for flexibility in syncing and easy conflict handling. They may not support complex join queries or ACID transactions across multiple tables like a full SQL database would. This is partly because replicating a full relational model is much harder (maintaining referential integrity, etc., when data is partial on a client) or not even logically possible. For example if you have two offline clients running a complex `UPDATE X WHERE Y FROM Z INNER JOIN Alice INNER JOIN Bob` query and then they go online, you have no easy way of handling these conflicts.
If your app has heavy relational data requirements or relies on complex server-side queries (aggregations, multi-join reports), you might find the local database either cannot do it or is too slow to do it client-side. The lack of robust relational querying is something to plan for and you might need to adjust your data model to be more document-oriented or use client-side libraries to run [joins in memory](../why-nosql.md#relational-queries-in-nosql). Most tools use NoSQL because it makes replication easy and implementing true relational sync would require extremely sophisticated solutions and even needing an atomic clock for full consistency across nodes (like google spanner). So, **if your app truly needs SQL power on the client**, local-first might complicate things.
> In Local-First, most tools use NoSQL because it makes replication and conflict handling easy.
That's a long list of challenges! In summary, local-first approaches introduce distributed data issues on the client side that web developers usually didn't have to deal with. Despite these challenges, the local-first movement is steadily growing because the **benefits to user experience and data control are very compelling** and modern tools are emerging to mitigate a lot of these difficulties. All of these are solved or solvable for your specific use-case, just keep them in mind before you start architecting your local-first app.
## Local-First vs. Traditional Online-First Approaches
So now that you know the pros and cons about Local-First. Lets directly compare it to your previous "online-first" stack:
### Connectivity and Offline Usage
- **Local-first**: Apps like WhatsApp store messages locally on your device, letting you read past messages and even compose new ones without a stable network connection. Once online, the messages sync with the server and other participants.
- **Online-first**: Purely cloud-based apps typically stop functioning (beyond simple caching) when the network or server goes down. They rely on a constant connection to fetch or store user data.
### Latency and Performance
- **Local-first**: Because data operations happen on the device, you see near-instant interactions (e.g., typing and reading chats feels very responsive, as the messages are on your phone). Syncing occurs in the background without delaying the user.
- **Online-first**: Most interactions involve a round-trip to the server, adding network latency and potentially requiring loading states or fallback UIs. If the network is slow or unreliable, users can experience delays when sending or receiving updates.
### Complexity and Conflict Resolution
- **Local-first**: Much of the complexity like storing data or handling conflicts, shifts to the client. WhatsApp, for instance, caches all messages locally and queues unsent messages offline. Once reconnected, it must reconcile with the server and other devices, especially if the same account is used on multiple platforms.
- **Online-first**: A single central server manages data and concurrency, so clients remain thinner. However, the app becomes unusable if connectivity is lost, and any server downtime directly impacts all users.
### Data Ownership and Storage Limits
- **Local-first**: Users hold a complete copy of data on their own device, retaining control and quick access. This can raise challenges when data sets are very large; phone storage might be insufficient, or backups may be harder to guarantee.
- **Online-first**: Storing all data in the cloud scales more easily, and providers often manage backups automatically. On the downside, users depend on the service and must trust it to protect their data.
### When to Choose Which
- **Local-first**: Particularly helpful for scenarios where offline operation and immediate responsiveness matter, such as chat apps (like WhatsApp), field tools in low-connectivity environments, or any use case where users can't rely on being online 24/7.
- **Online-first**: Well-suited for systems that demand real-time central control or massive data aggregation with minimal offline needs, such as large-scale analytics platforms or services where guaranteed immediate global consistency is essential.
- **Hybrid**: In reality, most modern apps often blend these approaches, giving users partial offline capabilities (caching, queued updates) alongside a robust central service. This hybrid method offers the benefits of local speed and resilience, while still leveraging cloud infrastructure for collaboration and global reach. But a truth local-first app is way more than just a cache.
---
{/*
## Local-First in Practice with RxDB
To concretely understand how local-first development works, let's walk through an example using RxDB, a database for building local-first [realtime app](./realtime-database.md) in JavaScript. RxDB runs inside your app, storing data in IndexedDB (or SQLite, etc.) and supports real-time sync with a backend.
:::note
Because you read this on the RxDB website, in the following a local-first setup with RxDB is shown. If you only care about Local-First in general, you can [skip](#partial-sync) this part.
:::
### Setting up a Local Database
With RxDB, you first create a database and define a schema for your collections. For example, suppose we're building a simple to-do list app that we want to work offline. We can define a "todos" collection with fields like `id`, `title`, `done`, `timestamp`, etc. In code, it looks like:
#### Imports
```ts
import { createRxDatabase, addRxPlugin } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
import { wrappedValidateAjvStorage } from 'rxdb/plugins/validate-ajv';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
// Enable dev mode (for helpful warnings in development)
addRxPlugin(RxDBDevModePlugin);
```
#### Create a Database
Here we use the [localstorage](../rx-storage-localstorage.md) based storage for RxDB which stores data inside of localstorage in a **browser**. There is a wide range of other [storages](../rx-storage.md) for example in **[React Native](../react-native-database.md)** you would use the [SQLite storage](../rx-storage-sqlite.md) instead.
```ts
const db = await createRxDatabase({
name: 'myappdb',
storage: wrappedValidateAjvStorage({
storage: getRxStorageLocalstorage()
})
});
```
#### Add a Collection
```ts
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
title: { type: 'string' },
done: { type: 'boolean' },
timestamp: { type: 'string' }
},
required: ['id', 'title', 'done', 'timestamp']
}
}
});
```
Here, we've created a local database and a todos collection with a JSON schema.
### Run local CRUD operations
Now, our frontend app can use this local collection just like a normal database: insert todos, query them, etc., **without any network calls**. For example:
#### Insert a new todo (this happens locally, instantly)
```ts
await db.todos.insert({
id: 'todo1',
title: 'Learn RxDB local-first',
done: false,
timestamp: new Date().toISOString()
});
```
#### Query todos that are not done yet
```ts
const remaining = await db.todos.find({
selector: { done: { $eq: false } }
}).exec();
console.log(`Remaining todos: ${remaining.length}`);
```
#### Update a document
```ts
const firstTodo = remaining[0];
await firstTodo.patch({ done: true }); // mark as done
```
#### Remove a document
```ts
await firstTodo.remove();
```
All these operations are interacting with the **IndexedDB in the browser**. They will succeed and modify the local persistent state even if the app is completely offline. From the user's perspective, the app just works and adding or checking off a todo is instantaneous. The local database writes are [very fast](../rx-storage-performance.md) (usually on the order of milliseconds) and don't depend on any server.
### Reactive UI Updates
One powerful feature of RxDB is that **queries are observable**. You can subscribe to a query to get real-time updates whenever the underlying data changes (including changes coming from other tabs or from sync). For instance, with RxJS-based subscriptions:
```ts
// Set up a real-time subscription to all todos that are not done
db.todos.find({ selector: { done: false } }).$.subscribe(todoList => {
console.log('Currently have ' + todoList.length + ' todos left');
// Here you would update your UI to display the latest todos
});
```
The `.$` on a query gives an RxJS observable. This subscription will trigger every time the result set changes. If, for instance, in another part of the app (or another browser tab) a todo is marked done or added, this callback will fire with the updated list. This is incredibly useful for building UIs that automatically reflect the current state of the local DB. RxDB allows you to subscribe to changes even if they happen in "another part of your application, another browser tab, or during database replication/synchronization"β. This is incredibly useful for building UIs that update automatically without manual refreshes or polling.
Learn also: Using **Signals** instead of **RxJS** for Reactivity
While RxJS observables are a well-established approach in the RxDB ecosystem, RxDB 14 introduced an alternative reactivity API [based on signals](../reactivity.md). Signals are a simpler reactive primitive, commonly seen in frameworks like _Vue (reactive refs)_ or _SolidJS (signals)_ or _Angular (signals)_. Signals are more intuitive alternative to RxJS and do not require developers to learn about all these RxJS operators. Here's a quick look at how you might use signals with RxDB with react:
#### Create a Query
```ts
const todosQuery = db.todos.find({ selector: { done: false } });
```
#### Get a Signal from the Query
```ts
const todosSignal = todosQuery.$$;
```
#### Use the Signal in a React Component
```tsx
function TodosComponent() {
const todoList = todosSignal();
return (
{todoList.map(todo => (
{todo.title}
))}
);
}
```
([Learn more about using Signals](../reactivity.md))
At this stage, we've achieved a functional offline app: the user can do all CRUD operations on todos, even offline, and the data is saved locally (persistently in [IndexedDB](../rx-storage-indexeddb.md)). But right now, if they open the app on another device or in another browser, they wouldn't see the same data because we haven't implemented sync yet. The next step is enabling synchronization with a backend, so that multiple clients and a server can share data.
:::note
You can find a full implementation of this example at the [Quickstart Repository](https://github.com/pubkey/rxdb-quickstart).
:::
### Syncing with a Backend
RxDB provides plugins for syncing with various backends: you can sync to [CouchDB](../replication-couchdb.md), use a [GraphQL endpoint](../replication-graphql.md), use your [firebase backend](../replication-firestore.md), or even do [P2P sync via WebRTC](../replication-webrtc.md) and more. But most people do not use these plugins. Instead they use the replication-primtives and build their own [compatible HTTP Endpoints](../replication-http.md) on their existing infrastructure.
For our example, lets assume you already have a backend server with the **three endpoints** for synchronizing "to-do" data. One endpoint (GET `/api/todos/pull?checkpoint=X&limit=Y`) returns an array of documents changed since a particular checkpoint value. The other endpoint (POST `/api/todos/push`) accepts an array of changed documents and writes them to the server, then returns any that are detected as being conflicts. Also we have a [Server-Send-Events](./websockets-sse-polling-webrtc-webtransport.md#what-are-server-sent-events) (GET `/api/todos/pull-stream`) endpoint that pings the client whenever something on the server changes.
Using RxDB's HTTP replication functionality, you can sync the to-do application with these routes:
#### Import the Replication Plugin
```ts
import { replicateRxCollection } from 'rxdb/plugins/replication-http';
```
#### Start the Replication
```ts
const replicationState = await replicateRxCollection({
collection: db.todos,
replicationIdentifier: 'my-todos-replication',
live: true,
push: {
async handler(changedDocs) {
const response = await fetch('/api/todos/push', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(changedDocs)
});
const { errorDocuments } = await response.json();
return errorDocuments;
}
},
pull: {
async handler(lastCheckpoint, batchSize) {
const url = '/api/todos/pull?checkpoint=' +
encodeURIComponent(JSON.stringify(lastCheckpoint)) +
'&limit=' + batchSize;
const response = await fetch(url);
const { documents, checkpoint } = await response.json();
return {
documents,
checkpoint
};
},
stream$: async function () {
const eventSource = new EventSource('/api/todos/pull-stream');
return new Observable(subscriber => {
eventSource.onmessage = (msg) => {
const data = JSON.parse(msg.data);
subscriber.next({
documents: data.documents,
checkpoint: data.checkpoint
});
};
return () => eventSource.close();
});
}
},
});
```
With just this configuration, RxDB will begin to **pull** any new or changed documents from the Server and apply them to the local store, and **push** any local changes up to the server. Because `live: true` and our `pullStream$`, it will keep doing this continuously (it's not a one-time sync). Under the hood, it uses an iterating checkpoint so it doesn't fetch everything every time since it will fetch in batches of changes (you can set `batchSize`) and use the `pull.stream$` to get new updates. Conflict handling is also integrated: if a conflict is detected during replication, by default RxDB will use a `first-on-server-wins` strategy. But any other conflict handler can be used instead.
## Partial Sync
RxDB supports [partial sync](../partial-sync.md) patterns where you dynamically manage multiple replication states for different data scopes. This keeps local storage lean and reduces network overhead while still giving users the offline, instant-feedback experience that local-first apps are known for.
*/}
## Offline-First vs. Local-First
In the early days of offline-capable web apps (around 2014), the common phrase was **"Offline-First"**. Tools like **PouchDB** popularized the notion that developers should assume devices are often offline or have flaky connections, so apps must continue to work seamlessly without a network. The guiding principle was *"apps should treat being online as optional."* If a user has no internet access, the application's core features still function, saving or queuing data locally, and automatically synchronizing once connectivity is restored.
Over time, this focus on offline support evolved into the broader concept of **"Local-First Software,"** (see [Ink & Switch](https://martin.kleppmann.com/papers/local-first.pdf)) emphasizing not just offline operation but also the technical underpinnings of **storing data locally** in the client application. While offline-first is primarily about resilience to network loss, local-first highlights ownership, privacy, and performance benefits of keeping the primary data on the user's device. Most tools these days extended the original offline-first concepts, adding real-time reactivity, custom sync, and more nuances like conflict resolution or encryption.
However, the term **"local-first"** can be **confusing** to non-technical audiences because many people (especially in the US) associate "local first" with *community-oriented movements* that encourage buying from nearby businesses or supporting local initiatives. To reduce ambiguity, it may be clearer to use **"local first software"** or **"local first development"** in your documentation and marketing materials. When creating branding or logos around local-first software, **avoid using the "Google Maps Pin"** as a symbol. This icon typically implies geolocation or physical locality further mixing up the notion of "location-based services" with "on-device data storage."
## Do People Actually Use Local-First or Is It Just a Trend?
If we look at **npm download statistics**, we see that **PouchDB** - one of the oldest libraries for local-first apps - has about **53k** downloads each week, and **RxDB** - a newer library - has about **22k** weekly downloads. Other local-first tools often have even fewer downloads. In comparison, a popular library like **react-query**, which does not focus on local storage, is downloaded about **1.6 million** times a week. These numbers show that local-first libraries, while used, are not as common as some of the more traditional tools.
One reason is that **local-first** is still a new idea. Many developers are used to traditional "online-first" approaches, so switching to a local database and then syncing changes later can feel unfamiliar. Developers must learn different patterns, deal with offline synchronization, and handle possible conflicts. That extra work can be a barrier to adoption which might change in the future as tooling improves.
While most of RxDB is open source, there are also **premium plugins** that help sustain RxDB as a long-term project. Because people purchase these plugins, we gains insights into how developers are using local-first features:
- About **half** of these users mainly want **offline functionality** for cases such as farming equipment, mining, construction, or even a shrimp farm app.
- The **other half** focus on **faster, real-time UIs** for to-do or reading apps, a space launch planning tool, and various dashboard apps. This range of use cases highlights both the resilience offline mode can offer and the performance boost that local databases can provide when synced in the background.
## Why Local-First Is the Future
Early in the history of the web, users **expected** static pages. If you wanted to see new content, you **reloaded** the page. That was normal at the time, and nobody found it strange because everything worked that way.
Then, as more sites added **real-time** features - auto-updating feeds, live notifications, single-page apps - suddenly those older "reload-only" sites began to feel **slow** or **outdated**. Why wait for a manual refresh when real-time data was possible and readily available?
The same pattern is happening with **local-first** apps. Right now, most sites are still built around network availability. We see loading spinners whenever data is fetched, and we simply wait for the server response. As local-first experiences become **commonplace** - removing spinners, letting users keep working when offline, and syncing in the background - everything else will start to feel **frustratingly behind**. Users won't tolerate slow or blocked interactions if they've seen apps that respond instantly and remain usable offline. They'll expect that as the default and we'll likely see growing pressure on developers to eliminate those extra loading steps. For many users, the experience of **immediate local writes will become not just a perk, but an expectation!**
## FAQ
An offline-first database stores data locally on the user device. A cloud-first database stores data on a remote server. Field workers in remote areas often face poor network connectivity. An offline-first database allows users to read and write data without an internet connection. A cloud-first database requires continuous internet access to function. An offline-first approach synchronizes data automatically when a connection becomes available. A cloud-first approach blocks users from working during offline periods. You choose an offline-first database to ensure continuous productivity in remote locations.
Local-first data storage provides **Zero Latency** because data is read and written directly to the local device without waiting for network requests. It guarantees extreme **Reliability** since the application remains fully functional regardless of internet connectivity (offline support). Furthermore, it improves user **Privacy** by keeping sensitive data on the native client rather than a centralized server, and significantly reduces cloud infrastructure costs by offloading database compute to the user's hardware.
JavaScript applications, particularly Single Page Applications (SPAs) built with React, Vue, or Angular, are heavily state-driven. A local-first architecture aligns perfectly with this paradigm by using client-side databases like **[RxDB](../rx-database.md)** to instantly manage the application state locally. This eliminates loading spinners, provides instant visual feedback, and bridges the gap between web applications and native desktop/mobile app performance.
**Offline-First** means designing an application to function properly without an internet connection, often caching resources and queueing actions to sync later. **[Local-First](./local-first-future.md)** takes this further: it means the *primary* source of truth for the application is the local database on the device, rather than a remote cloud server. The cloud merely acts as an eventual [syncing mechanism](../replication.md) (or backup) in the background, rather than the primary endpoint for every user interaction.
Yes, applications like WorkFlowy, Capacities, Notion (to an extent), and Linear use local-first or heavily optimized offline-first architectures. They load the user's workspace into the local browser or desktop client memory/database immediately upon startup. Every interaction mutates the local state first to provide instant UI feedback, and then asynchronous replication protocols silently sync those changes to their backend servers in the background.
## See also
- Discuss [this topic on HackerNews](https://news.ycombinator.com/item?id=43289885)
- [Local-First Technologies](../alternatives.md): A list of databases and technologies (besides [RxDB](/)) that support offline-first or local-first use cases.
- [Discord](/chat/): Join our Discord server to talk with people and share ideas about this topic.
- [Ink & Switch](https://martin.kleppmann.com/papers/local-first.pdf): The "original" paper about Local-First from 2019 where the naming of local-first Software was first used and described.
- [Learn how to build a local-first Application with RxDB](../quickstart.md).
---
## LocalStorage vs. IndexedDB vs. Cookies vs. OPFS vs. WASM-SQLite
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_BROWSER, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {Faq, FaqItem} from '@site/src/components/faq';
{/*
GOALS:
- Compare latency of single bit writes
- Compare latency of bulk writes
- Compare the latency of single item reads
- Compare the latency of bulk reads
- Compare storage size limit
- Compare feature table
- Indexing
- Cross-tab events
- Browser Support
-
- Give a conclusion on what to use which
- Tell about how RxDB storages might improve stuff
*/}
# LocalStorage vs. IndexedDB vs. Cookies vs. OPFS vs. WASM-SQLite
So you are building that web application and you want to **store data inside of your users browser**. Maybe you just need to store some small flags or you even need a fully fledged database.
The types of web applications we build have changed significantly. In the early years of the web we served static html files. Then we served dynamically rendered html and later we build **single page applications** that run most logic on the client. And for the coming years you might want to build so called [local first apps](../offline-first.md) that handle big and complex data operations solely on the client and even work when offline, which gives you the opportunity to build **zero-latency** user interactions.
In the early days of the web, **cookies** were the only option for storing small key-value assignments.. But JavaScript and browsers have evolved significantly and better storage APIs have been added which pave the way for bigger and more complex data operations.
In this article, we will dive into the various technologies available for storing and querying data in a browser. We'll explore traditional methods like **Cookies**, **localStorage**, **WebSQL**, **IndexedDB** and newer solutions such as **OPFS** and **SQLite via WebAssembly**. We compare the features and limitations and through performance tests we aim to uncover how fast we can write and read data in a web application with the various methods.
:::note
You are reading this in the [RxDB](/) docs. RxDB is a JavaScript database that has different storage adapters which can utilize the different storage APIs.
**Since 2017** I spend most of my time working with these APIs, doing performance tests and building [hacks](../slow-indexeddb.md) and plugins to reach the limits of browser database operation speed.
:::
## The available Storage APIs in a modern Browser
First lets have a brief overview of the different APIs, their intentional use case and history:
### What are Cookies
Cookies were first introduced by [netscape in 1994](https://www.baekdal.com/thoughts/the-original-cookie-specification-from-1997-was-gdpr-compliant/).
Cookies store small pieces of key-value data that are mainly used for session management, personalization, and tracking. Cookies can have several security settings like a time-to-live or the `domain` attribute to share the cookies between several subdomains.
Cookies values are not only stored at the client but also sent with **every http request** to the server. This means we cannot store much data in a cookie but it is still interesting how good cookie access performance compared to the other methods. Especially because cookies are such an important base feature of the web, many performance optimizations have been done and even these days there is still progress being made like the [Shared Memory Versioning](https://blog.chromium.org/2024/06/introducing-shared-memory-versioning-to.html) by chromium or the asynchronous [CookieStore API](https://developer.mozilla.org/en-US/docs/Web/API/Cookie_Store_API).
### What is LocalStorage
The [localStorage API](./localstorage.md) was first proposed as part of the [WebStorage specification in 2009](https://www.w3.org/TR/2009/WD-webstorage-20090423/#the-localstorage-attribute).
LocalStorage provides a simple API to store key-value pairs inside of a web browser. It has the methods `setItem`, `getItem`, `removeItem` and `clear` which is all you need from a key-value store. LocalStorage is only suitable for storing small amounts of data that need to persist across sessions and it is [limited by a 5MB storage cap](./localstorage.md#understanding-the-limitations-of-local-storage). Storing complex data is only possible by transforming it into a string for example with `JSON.stringify()`.
The API is not asynchronous which means if fully blocks your JavaScript process while doing stuff. Therefore running heavy operations on it might block your UI from rendering.
> There is also the **SessionStorage** API. The key difference is that localStorage data persists indefinitely until explicitly cleared, while sessionStorage data is cleared when the browser tab or window is closed.
### What is IndexedDB
IndexedDB was first introduced as "Indexed Database API" [in 2015](https://www.w3.org/TR/IndexedDB/#sotd).
[IndexedDB](../rx-storage-indexeddb.md) is a low-level API for storing large amounts of structured JSON data. While the API is a bit hard to use, IndexedDB can utilize indexes and asynchronous operations. It lacks support for complex queries and only allows to iterate over the indexes which makes it more like a base layer for other libraries then a fully fledged database.
In 2018, IndexedDB version 2.0 [was introduced](https://hacks.mozilla.org/2016/10/whats-new-in-indexeddb-2-0/). This added some major improvements. Most noticeable the `getAll()` method which improves performance dramatically when fetching bulks of JSON documents.
IndexedDB [version 3.0](https://w3c.github.io/IndexedDB/) is in the workings which contains many improvements. Most important the addition of `Promise` based calls that makes modern JS features like `async/await` more useful.
### What is OPFS
The [Origin Private File System](../rx-storage-opfs.md) (OPFS) is a [relatively new](https://caniuse.com/mdn-api_filesystemfilehandle_createsyncaccesshandle) API that allows web applications to store large files directly in the browser. It is designed for data-intensive applications that want to write and read **binary data** in a simulated file system.
OPFS can be used in two modes:
- Either asynchronous on the [main thread](../rx-storage-opfs.md#using-opfs-in-the-main-thread-instead-of-a-worker)
- Or in a WebWorker with the faster, asynchronous access with the `createSyncAccessHandle()` method.
Because only binary data can be processed, OPFS is made to be a base filesystem for library developers. You will unlikely directly want to use the OPFS in your code when you build a "normal" application because it is too complex. That would only make sense for storing plain files like images, not to store and query [JSON data](./json-based-database.md) efficiently. I have build a [OPFS based storage](../rx-storage-opfs.md) for RxDB with proper indexing and querying and it took me several months.
### What is WASM SQLite
[WebAssembly](https://webassembly.org/) (Wasm) is a binary format that allows high-performance code execution on the web.
Wasm was added to major browsers over the course of 2017 which opened a wide range of opportunities on what to run inside of a browser. You can compile native libraries to WebAssembly and just run them on the client with just a few adjustments. WASM code can be shipped to browser apps and generally runs much faster compared to JavaScript, but still about [10% slower than native](https://www.usenix.org/conference/atc19/presentation/jangda).
Many people started to use compiled SQLite as a database inside of the browser which is why it makes sense to also compare this setup to the native APIs.
The compiled byte code of SQLite has a size of [about 938.9Β kB](https://sqlite.org/download.html) which must be downloaded and parsed by the users on the first page load. WASM cannot directly access any persistent storage API in the browser. Instead it requires data to flow from WASM to the main-thread and then can be put into one of the browser APIs. This is done with so called [VFS (virtual file system) adapters](https://www.sqlite.org/vfs.html) that handle data access from SQLite to anything else.
### What was WebSQL
WebSQL **was** a web API [introduced in 2009](https://www.w3.org/TR/webdatabase/) that allowed browsers to use SQL databases for client-side storage, based on SQLite. The idea was to give developers a way to store and query data using SQL on the client side, similar to server-side databases.
WebSQL has been **removed from browsers** in the current years for multiple good reasons:
- WebSQL was not standardized and having an API based on a single specific implementation in form of the SQLite source code is hard to ever make it to a standard.
- WebSQL required browsers to use a [specific version](https://developer.chrome.com/blog/deprecating-web-sql#reasons_for_deprecating_web_sql) of SQLite (version 3.6.19) which means whenever there would be any update or bugfix to SQLite, it would not be possible to add that to WebSQL without possible breaking the web.
- Major browsers like firefox never supported WebSQL.
Therefore in the following we will **just ignore WebSQL** even if it would be possible to run tests on in by setting specific browser flags or using old versions of chromium.
-------------
## Feature Comparison
Now that you know the basic concepts of the APIs, lets compare some specific features that have shown to be important for people using RxDB and browser based storages in general.
### Storing complex JSON Documents
When you store data in a web application, most often you want to store complex JSON documents and not only "normal" values like the `integers` and `strings` you store in a server side database.
- Only IndexedDB works with JSON objects natively.
- With SQLite WASM you can [store JSON](https://www.sqlite.org/json1.html) in a `text` column since version 3.38.0 (2022-02-22) and even run deep queries on it and use single attributes as indexes.
Every of the other APIs can only store strings or binary data. Of course you can transform any JSON object to a string with `JSON.stringify()` but not having the JSON support in the API can make things complex when running queries and running `JSON.stringify()` many times can cause performance problems.
### Multi-Tab Support
A big difference when building a Web App compared to [Electron](../electron-database.md) or [React-Native](../react-native-database.md), is that the user will open and close the app in **multiple browser tabs at the same time**. Therefore you have not only one JavaScript process running, but many of them can exist and might have to share state changes between each other to not show **outdated data** to the user.
> If your users' muscle memory puts the left hand on the **F5** key while using your website, you did something wrong!
Not all storage APIs support a way to automatically share write events between tabs.
Only localstorage has a way to automatically share write events between tabs by the API itself with the [storage-event](./localstorage.md#localstorage-vs-indexeddb) which can be used to observe changes.
```js
// localStorage can observe changes with the storage event.
// This feature is missing in IndexedDB and others
addEventListener("storage", (event) => {});
```
There was the [experimental IndexedDB observers API](https://stackoverflow.com/a/33270440) for chrome, but the proposal repository has been archived.
To workaround this problem, there are two solutions:
- The first option is to use the [BroadcastChannel API](https://github.com/pubkey/broadcast-channel) which can send messages across browser tabs. So whenever you do a write to the storage, you also send a notification to other tabs to inform them about these changes. This is the most common workaround which is also used by RxDB. Notice that there is also the [WebLocks API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Locks_API) which can be used to have mutexes across browser tabs.
- The other solution is to use the [SharedWorker](https://developer.mozilla.org/en-US/docs/Web/API/SharedWorker) and do all writes inside of the worker. All browser tabs can then subscribe to messages from that **single** SharedWorker and know about changes.
### Indexing Support
The big difference between a database and storing data in a plain file, is that a database is writing data in a format that allows running operations over indexes to facilitate fast performant queries. From our list of technologies only **IndexedDB** and **WASM SQLite** support for indexing out of the box. In theory you can build indexes on top of any storage like localstorage or OPFS but you likely should not want to do that by yourself.
In IndexedDB for example, we can fetch a bulk of documents by a given index range:
```ts
// find all products with a price between 10 and 50
const keyRange = IDBKeyRange.bound(10, 50);
const transaction = db.transaction('products', 'readonly');
const objectStore = transaction.objectStore('products');
const index = objectStore.index('priceIndex');
const request = index.getAll(keyRange);
const result = await new Promise((res, rej) => {
request.onsuccess = (event) => res(event.target.result);
request.onerror = (event) => rej(event);
});
```
Notice that IndexedDB has the limitation of [not having indexes on boolean values](https://github.com/w3c/IndexedDB/issues/76). You can only index strings and numbers. To workaround that you have to transform boolean to numbers and backwards when storing the data.
### WebWorker Support
When running heavy data operations, you might want to move the processing away from the JavaScript main thread. This ensures that our app keeps being responsive and fast while the processing can run in parallel in the background. In a browser you can either use the [WebWorker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API), [SharedWorker](https://developer.mozilla.org/en-US/docs/Web/API/SharedWorker) or the [ServiceWorker](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API) API to do that. In RxDB you can use the [WebWorker](../rx-storage-worker.md) or [SharedWorker](../rx-storage-shared-worker.md) plugins to move your storage inside of a worker.
The most common API for that use case is spawning a **WebWorker** and doing most work on that second JavaScript process. The worker is spawned from a separate JavaScript file (or base64 string) and communicates with the main thread by sending data with `postMessage()`.
Unfortunately **LocalStorage** and **Cookies** [cannot be used in WebWorker or SharedWorker](https://stackoverflow.com/questions/6179159/accessing-localstorage-from-a-webworker) because of the design and security constraints. WebWorkers run in a separate global context from the main browser thread and therefore cannot do stuff that might impact the main thread. They have no direct access to certain web APIs, like the DOM, localStorage, or cookies.
Everything else can be used from inside a WebWorker.
The fast version of OPFS with the `createSyncAccessHandle` method can **only** [be used in a WebWorker](../rx-storage-opfs.md#opfs-limitations), and **not on the main thread**. This is because all the operations of the returned `AccessHandle` are **not async** and therefore block the JavaScript process, so you do want to do that on the main thread and block everything.
-------------
## Storage Size Limits
- **Cookies** are limited to about `4 KB` of data in [RFC-6265](https://datatracker.ietf.org/doc/html/rfc6265#section-6.1). Because the stored cookies are send to the server with every HTTP request, this limitation is reasonable. You can test your browsers cookie limits [here](http://www.ruslog.com/tools/cookies.html). Notice that you should never fill up the full `4 KB` of your cookies because your web server will not accept too long headers and reject the requests with `HTTP ERROR 431 - Request header fields too large`. Once you have reached that point you can not even serve updated JavaScript to your user to clean up the cookies and you will have locked out that user until the cookies get cleaned up manually.
- **LocalStorage** has a storage size limitation that varies depending on the browser, but generally ranges from 4 MB to 10 MB per origin. You can test your localStorage size limit [here](https://arty.name/localstorage.html).
- Chrome/Chromium/Edge: 5 MB per domain
- Firefox: 10 MB per domain
- Safari: 4-5 MB per domain (varies slightly between versions)
- **IndexedDB** does not have a specific fixed size limitation like localStorage. The maximum storage size for IndexedDB depends on the browser implementation. The upper limit is typically based on the available disc space on the user's device. In chromium browsers it can use up to 80% of total disk space. You can get an estimation about the storage size limit by calling `await navigator.storage.estimate()`. Typically you can store gigabytes of data which can be tried out [here](https://demo.agektmr.com/storage/). Notice that we have a full article about [storage max size limits of IndexedDB](./indexeddb-max-storage-limit.md) that covers this topic.
- **OPFS** has the same storage size limitation as IndexedDB. Its limit depends on the available disc space. This can also be tested [here](https://demo.agektmr.com/storage/).
-------------
## Performance Comparison
Now that we've reviewed the features of each storage method, let's dive into performance comparisons, focusing on initialization times, read/write latencies, and bulk operations.
Notice that we only run simple tests and for your specific use case in your application the results might differ. Also we only compare performance in google chrome (version 128.0.6613.137). Firefox and Safari have similar **but not equal** performance patterns. You can run the test by yourself on your own machine from this [github repository](https://github.com/pubkey/localstorage-indexeddb-cookies-opfs-sqlite-wasm). For all tests we throttle the network to behave like the average german internet speed. (download: 135,900 kbit/s, upload: 28,400 kbit/s, latency: 125ms). Also all tests store an "average" JSON object that might be required to be stringified depending on the storage. We also only test the performance of storing documents by id because some of the technologies (cookies, OPFS and localstorage) do not support indexed range operations so it makes no sense to compare the performance of these.
### Initialization Time
Before you can store any data, many APIs require a setup process like creating databases, spawning WebAssembly processes or downloading additional stuff. To ensure your app starts fast, the initialization time is important.
The APIs of localStorage and Cookies do not have any setup process and can be directly used. IndexedDB requires to open a database and a store inside of it. WASM SQLite needs to download a WASM file and process it. OPFS needs to download and start a worker file and initialize the virtual file system directory.
Here are the time measurements from how long it takes until the first bit of data can be stored:
| Technology | Time in Milliseconds |
| ----------------------- | -------------------- |
| IndexedDB | 46 |
| OPFS Main Thread | 23 |
| OPFS WebWorker | 26.8 |
| WASM SQLite (memory) | 504 |
| WASM SQLite (IndexedDB) | 535 |
Here we can notice a few things:
- Opening a new IndexedDB database with a single store takes surprisingly long
- The latency overhead of sending data from the main thread to a WebWorker OPFS is about 4 milliseconds. Here we only send minimal data to init the OPFS file handler. It will be interesting if that latency increases when more data is processed.
- Downloading and parsing WASM SQLite and creating a single table takes about half a second. Using also the IndexedDB VFS to store data persistently adds additional 31 milliseconds. Reloading the page with enabled caching and already prepared tables is a bit faster with 420 milliseconds (memory).
### Latency of small Writes
Next lets test the latency of small writes. This is important when you do many small data changes that happen independent from each other. Like when you stream data from a websocket or persist pseudo randomly happening events like mouse movements.
| Technology | Time in Milliseconds |
| ----------------------- | -------------------- |
| Cookies | 0.058 |
| LocalStorage | 0.017 |
| IndexedDB | 0.17 |
| OPFS Main Thread | 1.46 |
| OPFS WebWorker | 1.54 |
| WASM SQLite (memory) | 0.17 |
| WASM SQLite (IndexedDB) | 3.17 |
Here we can notice a few things:
- LocalStorage has the lowest write latency with only 0.017 milliseconds per write.
- IndexedDB writes are about 10 times slower compared to localStorage.
- Sending the data to the WASM SQLite process and letting it persist via IndexedDB is slow with over 3 milliseconds per write.
The OPFS operations take about 1.5 milliseconds to write the JSON data into one document per file. We can see the sending the data to a webworker first is a bit slower which comes from the overhead of serializing and deserializing the data on both sides.
If we would not create on OPFS file per document but instead append everything to a single file, the performance pattern changes significantly. Then the faster file handle from the `createSyncAccessHandle()` only takes about 1 millisecond per write. But this would require to somehow remember at which position the each document is stored. Therefore in our tests we will continue using one file per document.
### Latency of small Reads
Now that we have stored some documents, lets measure how long it takes to read single documents by their `id`.
| Technology | Time in Milliseconds |
| ----------------------- | -------------------- |
| Cookies | 0.132 |
| LocalStorage | 0.0052 |
| IndexedDB | 0.1 |
| OPFS Main Thread | 1.28 |
| OPFS WebWorker | 1.41 |
| WASM SQLite (memory) | 0.45 |
| WASM SQLite (IndexedDB) | 2.93 |
Here we can notice a few things:
- LocalStorage reads are **really really fast** with only 0.0052 milliseconds per read.
- The other technologies perform reads in a similar speed to their write latency.
### Big Bulk Writes
As next step, lets do some big bulk operations with 200 documents at once.
| Technology | Time in Milliseconds |
| ----------------------- | -------------------- |
| Cookies | 20.6 |
| LocalStorage | 5.79 |
| IndexedDB | 13.41 |
| OPFS Main Thread | 280 |
| OPFS WebWorker | 104 |
| WASM SQLite (memory) | 19.1 |
| WASM SQLite (IndexedDB) | 37.12 |
Here we can notice a few things:
- Sending the data to a WebWorker and running it via the faster OPFS API is about twice as fast.
- WASM SQLite performs better on bulk operations compared to its single write latency. This is because sending the data to WASM and backwards is faster if it is done all at once instead of once per document.
### Big Bulk Reads
Now lets read 100 documents in a bulk request.
| Technology | Time in Milliseconds |
| ----------------------- | ------------------------------- |
| Cookies | 6.34 |
| LocalStorage | 0.39 |
| IndexedDB | 4.99 |
| OPFS Main Thread | 54.79 |
| OPFS WebWorker | 25.61 |
| WASM SQLite (memory) | 3.59 |
| WASM SQLite (IndexedDB) | 5.84 (35ms without cache) |
Here we can notice a few things:
- Reading many files in the OPFS webworker is about **twice as fast** compared to the slower main thread mode.
- WASM SQLite is surprisingly fast. Further inspection has shown that the WASM SQLite process keeps the documents in memory cached which improves the latency when we do reads directly after writes on the same data. When the browser tab is reloaded between the writes and the reads, finding the 100 documents takes about **35 milliseconds** instead.
## Performance Conclusions
- LocalStorage is really fast but remember that is has some downsides:
- It blocks the main JavaScript process and therefore should not be used for big bulk operations.
- Only Key-Value assignments are possible, you cannot use it efficiently when you need to do index based range queries on your data.
- OPFS is way faster when used in the WebWorker with the `createSyncAccessHandle()` method compare to using it directly in the main thread.
- SQLite WASM can be fast but you have to initially download the full binary and start it up which takes about half a second. This might not be relevant at all if your app is started up once and the used for a very long time. But for web-apps that are opened and closed in many browser tabs many times, this might be a problem.
-------------
## Possible Improvements
There is a wide range of possible improvements and performance hacks to speed up the operations.
- For IndexedDB I have made a list of [performance hacks here](../slow-indexeddb.md). For example you can do sharding between multiple database and webworkers or use a custom index strategy.
- OPFS is slow in writing one file per document. But you do not have to do that and instead you can store everything at a single file like a normal database would do. This improves performance dramatically like it was done with the RxDB [OPFS RxStorage](../rx-storage-opfs.md).
- You can mix up the technologies to optimize for multiple scenarios at once. For example in RxDB there is the [localstorage meta optimizer](../rx-storage-localstorage-meta-optimizer.md) which stores initial metadata in localstorage and "normal" documents inside of IndexedDB. This improves the initial startup time while still having the documents stored in a way to query them efficiently.
- There is the [memory-mapped](../rx-storage-memory-mapped.md) storage plugin in RxDB which maps data directly to memory. Using this in combination with a shared worker can improve pageloads and query time significantly.
- [Compressing](../key-compression.md) data before storing it might improve the performance for some of the storages.
- Splitting work up between [multiple WebWorkers](../rx-storage-worker.md) via [sharding](../rx-storage-sharding.md) can improve performance by utilizing the whole capacity of your users device.
Here you can see the [performance comparison](../rx-storage-performance.md) of various RxDB storage implementations which gives a better view of real world performance:
## Future Improvements
You are reading this in 2024, but the web does not stand still. There is a good chance that browser get enhanced to allow faster and better data operations.
- Currently there is no way to directly access a persistent storage from inside a WebAssembly process. If this changes in the future, running SQLite (or a similar database) in a browser might be the best option.
- Sending data between the main thread and a WebWorker is slow but might be improved in the future. There is a [good article](https://surma.dev/things/is-postmessage-slow/) about why `postMessage()` is slow.
- IndexedDB lately [got support](https://developer.chrome.com/blog/maximum-idb-performance-with-storage-buckets) for storage buckets (chrome only) which might improve performance.
## FAQ
The maximum storage limit for browser LocalStorage is generally [around 5 MiB](./localstorage.md) per origin (combination of protocol, domain, and port) across most modern web browsers. If your application needs to handle larger datasets, files, or complex objects, you should migrate to **[IndexedDB](../rx-storage-indexeddb.md)** or **[OPFS](../rx-storage-opfs.md)**, which offer significantly larger, often gigabyte-scale storage quotas.
Use **Cookies** exclusively for small, server-readable session identifiers or authentication tokens, as they are sent with every HTTP request. Use **[LocalStorage](./localstorage.md)** for small, synchronous, non-sensitive application state blocks (like UI themes or preferences) under 5 MiB. Use **[IndexedDB](../rx-storage-indexeddb.md)** for handling complex structured data, large document collections, binary blobs, and scenarios where asynchronous operations and indexing are mandatory.
**OPFS (Origin Private File System)** provides a sandboxed, highly performant filesystem API native to the browser, offering direct, in-place write access to local files. Compared to [IndexedDB](../rx-storage-indexeddb.md) (which is a generic NoSQL object store), OPFS is considerably faster for heavy I/O operations and handles raw bytes much better. [RxDB provides an OPFS storage adapter](../rx-storage-opfs.md) that leverages this extreme performance while maintaining a standard NoSQL query interface.
**Deno** inherently supports the standard `localStorage` JavaScript API natively out of the box, allowing you to persist data across execution runs seamlessly. However, **Bun** does *not* support the `localStorage` API natively as of its recent versions. For Bun, you must either polyfill the API, utilize the `bun:sqlite` module, or use a comprehensive local database like **[RxDB](../rx-database.md)** to manage state.
The **File System Access API** allows web applications to read and write directly to the user's local, native device filesystem (with their explicit permission). In contrast, **[LocalStorage](./localstorage.md)** and **[IndexedDB](../rx-storage-indexeddb.md)** are strictly managed by the browser and sandboxed within the application's origin, meaning users cannot easily access or modify those raw database files on their hard drive. This makes the File System Access API ideal for local-first document editors, but less optimal for high-speed, indexed database operations.
## Follow Up
- Share my [announcement tweet](https://x.com/rxdbjs/status/1846145062847062391) -->
- Reproduce the benchmarks at the [github repo](https://github.com/pubkey/localstorage-indexeddb-cookies-opfs-sqlite-wasm)
- Learn how to use RxDB with the [RxDB Quickstart](../quickstart.md)
- Check out the [RxDB github repo](https://github.com/pubkey/rxdb) and leave a star β
---
## Using localStorage in Modern Applications - A Comprehensive Guide
import {Faq, FaqItem} from '@site/src/components/faq';
# Using localStorage in Modern Applications: A Comprehensive Guide
When it comes to client-side storage in web applications, the localStorage API stands out as a simple and widely supported solution. It allows developers to store key-value pairs directly in a user's browser. In this article, we will explore the various aspects of the localStorage API, its advantages, limitations, and alternative storage options available for modern applications.
## What is the localStorage API?
The localStorage API is a built-in feature of web browsers that enables web developers to store small amounts of data persistently on a user's device. It operates on a simple key-value basis, allowing developers to save strings, numbers, and other simple data types. This data remains available even after the user closes the browser or navigates away from the page. The API provides a convenient way to maintain state and store user preferences without relying on server-side storage.
## Exploring local storage Methods: A Practical Example
Let's dive into some hands-on code examples to better understand how to leverage the power of localStorage. The API offers several methods for interaction, including setItem, getItem, removeItem, and clear. Consider the following code snippet:
```js
// Storing data using setItem
localStorage.setItem('username', 'john_doe');
// Retrieving data using getItem
const storedUsername = localStorage.getItem('username');
// Removing data using removeItem
localStorage.removeItem('username');
// Clearing all data
localStorage.clear();
```
## Storing Complex Data in JavaScript with JSON Serialization
While js localStorage excels at handling simple key-value pairs, it also supports more intricate data storage through JSON serialization. By utilizing JSON.stringify and JSON.parse, you can store and retrieve structured data like objects and arrays. Here's an example of storing a document:
```js
const user = {
name: 'Alice',
age: 30,
email: 'alice@example.com'
};
// Storing a user object
localStorage.setItem('user', JSON.stringify(user));
// Retrieving and parsing the user object
const storedUser = JSON.parse(localStorage.getItem('user'));
```
## Understanding the Limitations of local storage
Despite its convenience, localStorage does come with a set of limitations that developers should be aware of:
- **Non-Async Blocking API**: One significant drawback is that js localStorage operates as a non-async blocking API. This means that any operations performed on localStorage can potentially block the main thread, leading to slower application performance and a less responsive user experience.
- **Limited Data Structure**: Unlike more advanced databases, localStorage is limited to a simple key-value store. This restriction makes it unsuitable for storing complex data structures or managing relationships between data elements.
- **Stringification Overhead**: Storing [JSON data](./json-based-database.md) in localStorage requires stringifying the data before storage and parsing it when retrieved. This process introduces performance overhead, potentially slowing down operations by up to 10 times.
- **Lack of Indexing**: localStorage lacks indexing capabilities, making it challenging to perform efficient searches or iterate over data based on specific criteria. This limitation can hinder applications that rely on complex data retrieval.
- **Tab Blocking**: In a multi-tab environment, one tab's localStorage operations can impact the performance of other tabs by monopolizing CPU resources. You can reproduce this behavior by opening [this test file](https://pubkey.github.io/client-side-databases/database-comparison/index.html) in two browser windows and trigger localstorage inserts in one of them. You will observe that the indication spinner will stuck in both windows.
- **Storage Limit**: Browsers typically impose a storage limit of [around 5 MiB](https://developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_and_eviction_criteria#web_storage) for each origin's localStorage.
## Reasons to Still Use localStorage
### Is localStorage Slow?
Contrary to concerns about performance, the localStorage API in JavaScript is surprisingly fast when compared to alternative storage solutions like [IndexedDB or OPFS](./localstorage-indexeddb-cookies-opfs-sqlite-wasm.md). It excels in handling small key-value assignments efficiently. Due to its simplicity and direct integration with browsers, accessing and modifying localStorage data incur minimal overhead. For scenarios where quick and straightforward data storage is required, localStorage remains a viable option. For example RxDB uses localStorage in the [localStorage meta optimizer](../rx-storage-localstorage-meta-optimizer.md) to manage simple key values pairs while storing the "normal" documents inside of another storage like IndexedDB.
## When Not to Use localStorage
While localStorage offers convenience, it may not be suitable for every use case. Consider the following situations where alternatives might be more appropriate:
- **Data Must Be Queryable**: If your application relies heavily on querying data based on specific criteria, localStorage might not provide the necessary querying capabilities. Complex data retrieval might lead to inefficient code and slow performance.
- **Big JSON Documents**: Storing large JSON documents in localStorage can consume a significant amount of memory and degrade performance. It's essential to assess the size of the data you intend to store and consider more robust solutions for handling substantial datasets.
- **Many Read/Write Operations**: Excessive read and write operations on localStorage can lead to performance bottlenecks. Other storage solutions might offer better performance and scalability for applications that require frequent data manipulation.
- **Lack of Persistence**: If your application can function without persistent data across sessions, consider using in-memory data structures like `new Map()` or `new Set()`. These options offer speed and efficiency for transient data.
## What to use instead of the localStorage API in JavaScript
### localStorage vs IndexedDB
While **localStorage** serves as a reliable storage solution for simpler data needs, it's essential to explore alternatives like **[IndexedDB](../rx-storage-indexeddb.md)** when dealing with more complex requirements. **IndexedDB** is designed to store not only key-value pairs but also JSON documents. Unlike localStorage, which usually has a storage limit of around 5-10MB per domain, IndexedDB can handle significantly larger datasets. IndexDB with its support for indexing facilitates efficient querying, making range queries possible. However, it's worth noting that IndexedDB lacks observability, which is a feature unique to localStorage through the `storage` event. Also,
complex queries can pose a challenge with IndexedDB, and while its performance is acceptable, IndexedDB can be [too slow](../slow-indexeddb.md) for some use cases.
```js
// localStorage can observe changes with the storage event.
// This feature is missing in IndexedDB
addEventListener("storage", (event) => {});
```
For those looking to harness the full power of IndexedDB with added capabilities, using wrapper libraries like [RxDB](https://rxdb.info/) is recommended. These libraries augment IndexedDB with features such as complex queries and observability, enhancing its usability for modern applications by providing a real database instead of only a key-value store.
In summary when you compare IndexedDB vs localStorage, IndexedDB will win at any case where much data is handled while localStorage has better performance on small key-value datasets.
### File System API (OPFS)
Another intriguing option is the OPFS (File System API). This API provides direct access to an origin-based, sandboxed filesystem which is highly optimized for performance and offers in-place write access to its content.
OPFS offers impressive performance benefits. However, working with the OPFS API can be complex, and it's only accessible within a **WebWorker**. To simplify its usage and extend its capabilities, consider using a wrapper library like [RxDB's OPFS RxStorage](../rx-storage-opfs.md), which builds a comprehensive database on top of the OPFS API. This abstraction allows you to harness the power of the OPFS API without the intricacies of direct usage.
### localStorage vs Cookies
Cookies, once a primary method of client-side data storage, have fallen out of favor in modern web development due to their limitations. While they can store data, they are about **100 times slower** when compared to the localStorage API. Additionally, cookies are included in the HTTP header, which can impact network performance. As a result, cookies are not recommended for data storage purposes in contemporary web applications.
### localStorage vs WebSQL
WebSQL, despite offering a SQL-based interface for client-side data storage, is a **deprecated technology** and should be avoided. Its API has been phased out of modern browsers, and it lacks the robustness of alternatives like IndexedDB. Moreover, WebSQL tends to be around 10 times slower than IndexedDB, making it a suboptimal choice for applications that demand efficient data manipulation and retrieval.
### localStorage vs sessionStorage
In scenarios where data persistence beyond a session is unnecessary, developers often turn to sessionStorage. This storage mechanism retains data only for the duration of a tab or browser session. It survives page reloads and restores, providing a handy solution for temporary data needs. However, it's important to note that sessionStorage is limited in scope and may not suit all use cases.
### AsyncStorage for React Native
For [React Native](../react-native-database.md) developers, the [AsyncStorage API](https://reactnative.dev/docs/asyncstorage) is the go-to solution, mirroring the behavior of localStorage but with asynchronous support. Since not all JavaScript runtimes support localStorage, AsyncStorage offers a seamless alternative for data persistence in React Native applications.
### `node-localstorage` for Node.js
Because native localStorage is absent in the **[Node.js](../nodejs-database.md)** JavaScript runtime, you will get the error `ReferenceError: localStorage is not defined` in Node.js or node based runtimes like Next.js. The [node-localstorage npm package](https://github.com/lmaccherone/node-localstorage) bridges the gap. This package replicates the browser's localStorage API within the Node.js environment, ensuring consistent and compatible data storage capabilities.
## localStorage in browser extensions
While browser extensions for chrome and firefox support the localStorage API, it is not recommended to use it in that context to store extension-related data. The browser will clear the data in many scenarios like when the users clear their browsing history.
Instead the [Extension Storage API](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage#properties) should be used for browser extensions.
In contrast to localStorage, the storage API works `async` and all operations return a Promise. Also it provides automatic sync to replicate data between all instances of that browser that the user is logged into. The storage API is even able to storage JSON-ifiable objects instead of plain strings.
```ts
// Using the storage API in chrome
await chrome.storage.local.set({ foobar: {nr: 1} });
const result = await chrome.storage.local.get('foobar');
console.log(result.foobar); // {nr: 1}
```
## localStorage in Deno and Bun
The **Deno** JavaScript runtime has a working localStorage API so running `localStorage.setItem()` and the other methods, will just work and the locally stored data is persisted across multiple runs.
**Bun** does not support the localStorage JavaScript API. Trying to use `localStorage` will error with `ReferenceError: Can't find variable: localStorage`. To store data locally in Bun, you could use the `bun:sqlite` module instead or directly use a in-JavaScript database with Bun support like [RxDB](https://rxdb.info/).
## Conclusion: Choosing the Right Storage Solution
In the world of modern web development, **localStorage** serves as a valuable tool for lightweight data storage. Its simplicity and speed make it an excellent choice for small key-value assignments. However, as application complexity grows, developers must assess their storage needs carefully. For scenarios that demand advanced querying, complex data structures, or high-volume operations, alternatives like IndexedDB, wrapper libraries with additional features like [RxDB](../), or platform-specific APIs offer more robust solutions. By understanding the strengths and limitations of various storage options, developers can make informed decisions that pave the way for efficient and scalable applications.
## FAQ
Both `localStorage` and `sessionStorage` provide synchronous, key-value storage capabilities built natively into the web browser. The primary difference is their lifespan: data in **LocalStorage** persists indefinitely until explicitly cleared by the application or the user. Data in **SessionStorage**, however, is strictly bound to the specific browser tab that created it and is instantly deleted the moment you close the tab.
Yes, `localStorage` inherently adheres to the browser's strict Same-Origin Policy. All stored data is tightly sandboxed by the exact combination of the protocol, hostname, and port. For example, scripts loaded on `https://example.com` are physically unable to access data stored by `http://example.com` (different protocol) or `https://app.example.com` (different subdomain).
LocalStorage only officially supports storing string values. To store structured data like [JSON objects](./json-database.md) or arrays, you must serialize them to a string using `JSON.stringify()` before storage and parse them with `JSON.parse()` upon retrieval. You cannot store binary formats like Files or Blobs directly; they must either be converted to a Base64 string, which is inefficient and bulky, or natively stored using **[IndexedDB](../rx-storage-indexeddb.md)** or **[OPFS](../rx-storage-opfs.md)** instead.
No, data saved in LocalStorage is confined entirely to the local storage hardware of the specific device and browser profile that created it. It does not automatically synchronize to the cloud or other devices. To seamlessly sync locally stored data across multiple clients, you need a sync-ready database like **[RxDB](../rx-database.md)** that automatically replicates the document state to a central [backend server](../replication.md).
Yes, data in `localStorage` persists across entirely different browser sessions. Even if a user closes their tab, quits their browser application, or reboots their system, the data remains intact whenever they return. Furthermore, any changes made to `localStorage` in one tab are immediately available to all other active tabs operating under the identically formatted origin string.
Data in `localStorage` persists indefinitely. The API does not provide any native automatic expiration or Time-to-Live (TTL) mechanisms. If you need data to expire automatically after a specific time frame, you must manually save a timestamp alongside your data payload and implement your own JavaScript verification logic to delete the item when the operational threshold is passed.
No. Because LocalStorage strictly enforces the Same-Origin Policy, it physically cannot pass or share data across completely different domains or even subdomains out-of-the-box. The origin string (comprising the scheme, hostname, and port) must match exactly. Sharing real-time data or state across separate domains usually requires complex workarounds involving hidden `