Pre-built Database

Description — How to Handle Pre-Built Couchbase Lite Databases in Your App

Abstract — This content explains how to include a snapshot of a pre-built database in your package to shorten initial sync time and reduce bandwidth use.

Overview

Couchbase Lite's support for pre-built databases means you can pre-load your app with data instead of syncing it down from Sync Gateway during startup. This might benefit, for example, a mobile application developer striving to minimize consumer wait time (arising from data setup) on initial install and launch of the application.

Avoiding an initial bulk sync will help reduce startup time and network transfer costs, leading to a better consumer experience. It is typically more efficient to download bulk data using the HTTP/FTP stream employed during the application installation than to install a smaller application bundle and then have to use a replicator to pull-in the bulk data.

This prepackaging of the data typically applies to public/shared, non-user specific data that is mostly static. Even if the data is not static, you can still benefit from preloading it and only syncing the changed documents on startup.

The initial sync of any pre-built database will pull-in any content changes on the server that occurred after its incorporation into the app, quickly bringing the database up to date.

To use a Pre-built Database

1. Create a new Couchbase Lite database with the required data set - see Creating Pre-built database.
2. Incorporate the pre-built database with your app bundle as an asset/resource - see Bundle a Database with an Application. For Flutter apps it's recommended you zip the prebuilt database and then extract it to keep the size of your mobile app down.
3. Adjust the start-up logic of your app to check for the presence of the required database. If the database doesn't already exist, create one using the bundled pre-built database. Then initiate a sync to update the data - see Using Pre-built Database on App Launch.

Creating Pre-built Database

These steps should form part of your build and release process:

1. Create a fresh Couchbase Lite database (every time).

   Important

   Always create a new pre-built database for each app version. This ensures that there will be no checkpoint missmatch between the pre-built database and the replicator configuration that the app uses.

   Otherwise: The cached checkpoint in the packaged database might be invalidated on the first synchronization and the app will attempt to pull the data down again.

2. Pull the data from Capella App Services or Sync Gateway into the new Couchbase Lite database.

   Important

   Ensure the replication used to populate Couchbase Lite database uses the exact same remote URL and replication config parameters (channels and filters) as those your app will use when it is running.

   Otherwise: There will be a checkpoint mismatch and the app will attempt to pull the data down again.

   So don't, for instance, create a pre-built database against a staging Capella App Services or Sync Gateway server and try to use that within a production app that syncs against a production Sync Gateway.

   You can use the cblite tool (cblite cp) for this — see: cblite cp (export, import, push, pull) on GitHub.

3. Create the same indexes the app will use (wait for the replication to finish before doing this).

Bundle a Database with an Application

Copy the database into your app package. It's recommended you compress the database using zip to keep the size of your mobile app down.

Put it in an appropriate place (for example, an assets or resource folder).

Alternatively: Rather than bundling the database within the app, the app could pull the database down from a CDN server on launch, but this add to the time it takes for your app to bootstrap.

Database Encryption

Important

This feature is an Enterprise Edition feature.

If you are using en encrypted database, note that Database.copy does not change the encryption key. The encryption key specified in the config when opening the database is the encryption key used for both the original database and copied database.

If you copied an un-encrypted database and want to apply encryption to the copy, or if you want to change (or remove) the encryption key applied to the copy:

1. Provide the original encryption-key (if any) in the database copy's configuration using DatabaseConfiguration DatabaseConfiguration.encryptionKey.

2. Open the database copy.

3. Use Database.changeEncryptionKey on the database copy to set the required encryption key.

   tip

   To remove encryption on the copy, provide a null encryption-key.

Using Pre-built Database on App Launch

During the application start-up logic, check if database exists in the required location, and if not:

1. Locate the pre-packaged database (for example, in the assets or other resource folder).

2. Copy the pre-packaged database to the required location.

   Use the API's Database.copy method — see: Example 1. This ensures that a unique UUID is generated for each copy.

   IMPORTANT

   Do not copy the database using any other method.

   Otherwise: Each copy of the app will invalidate the other apps' checkpoints because a new UUID was not generated.

3. Open the database; you can now start querying the data and using it.

4. Start a pull replication, to sync any changes.

   The replicator uses the pre-built database's checkpoint as the timestamp to sync from; only documents changed since then are synced.

   note

   Start your normal application logic immediately, unless it is essential to have the absolute up-to-date data set to begin. That way the user is not kept hanging around watching a progress indicator. They can begin interacting with your app whilst any out-of-data data is being updated.

Example 1. Decompress and Copy Database using API

const databaseName = 'my-database';
const prebuiltDatabaseAssetFileName = 'asset/database/prebuiltDatabase.zip';
final documentsDirectory = await getApplicationDocumentsDirectory();
final databasesDirectory = '$documentsDirectory/databases';
final prebuiltDatabaseFileName =
    '$databasesDirectory/$databaseName-prebuilt.cblite2';

// Create the database if it doesn't already exist.
if (!Database.existsSync(databaseName, directory: databasesDirectory)) {
  final prebuiltDatabaseZip =
      await rootBundle.load(prebuiltDatabaseAssetFileName);

  if (prebuiltDatabaseZip.lengthInBytes > 0) {
    // Decompress the zip file into bytes and then convert them into a
    // Uint8List, which is required by the Archive framework.
    final archive =
        ZipDecoder().decodeBytes(prebuiltDatabaseZip.buffer.asUint8List());

    // Loop through directories and files in the zip file and create them.
    for (final file in archive) {
      final fileName = '$prebuiltDatabaseFileName/${file.name}';
      if (file.isFile) {
        final fileData = file.content as List<int>;
        File(fileName)
          ..createSync(recursive: true)
          ..writeAsBytesSync(fileData);
      } else {
        Directory(fileName).createSync(recursive: true);
      }
    }

    final config = DatabaseConfiguration(directory: databasesDirectory);
    await Database.copy(
      from: prebuiltDatabaseFileName,
      name: databaseName,
      config: config,
    );
  }
}