diff --git a/doc/guides/persistence.md b/doc/guides/persistence.md index 331cdb047..5aa7321cc 100644 --- a/doc/guides/persistence.md +++ b/doc/guides/persistence.md @@ -3,7 +3,18 @@ icon: database order: 98 --- -Ndk comes with several database offerings. The simplest is the `MemCacheManager` which is an in-memory cache. This is useful for testing and small applications. +NDK keeps two local stores side by side: + +- the **event cache** (`CacheManager`) — Nostr events, contact lists, metadata, NIP-05 records, relay sets, fetched ranges, etc. +- the **blob cache** (`BlobCacheManager`) — binary payloads from Blossom servers, content-addressed by SHA-256. + +Both are pluggable: pick the backend that fits your platform, or supply your own. + +> **Tip:** keep these databases dedicated to NDK and spin up a secondary database for your own app data. + +## Event cache + +The simplest backend is `MemCacheManager`, an in-memory cache useful for testing and small apps. Available databases: @@ -12,9 +23,7 @@ Available databases: - [`SembastCacheManager`](https://pub.dev/packages/sembast_cache_manager) - [`DriftCacheManager`](https://pub.dev/packages/ndk_drift) -If you want your own database, you need to implement the `CacheManager` interface. Contributions for more database implementations are welcome! - -Its recommended to use the database only for ndk and spin up a secondary db for your own app data. +If you want your own database, implement the `CacheManager` interface. Contributions for more backends are welcome. ```dart objectbox example import 'package:ndk/ndk.dart'; @@ -103,3 +112,100 @@ class NostrNoteModel extends NostrNote { ... } ``` + +## Blob cache + +Conceptually a *local Blossom server* without the network layer: the API mirrors a remote Blossom server's surface (`saveBlob`, `getBlob`, `hasBlob`, `listBlobs`, `removeBlob`) and reuses the same entities (`BlobDescriptor`, `BlobResponse`). + +If you don't configure `blobCache`, NDK falls back to an in-memory `IdbBlobCacheManager` (one per `Ndk`, lost on process exit). Pass your own factory for persistence. + +### Native: persistent cache + +Use `idb_io` (or [`idb_sqflite`](https://pub.dev/packages/idb_sqflite) for cross-process safety): + +```dart +import 'package:idb_shim/idb_io.dart'; +import 'package:ndk/ndk.dart'; + +final ndk = Ndk( + NdkConfig( + eventVerifier: Bip340EventVerifier(), + cache: MemCacheManager(), + blobCache: IdbBlobCacheManager( + factory: getIdbFactoryIo()!, + dbName: 'my_app_blob_cache', + ), + ), +); +``` + +### Web: persistent cache + +```dart +import 'package:idb_shim/idb_browser.dart'; +import 'package:ndk/ndk.dart'; + +final ndk = Ndk( + NdkConfig( + eventVerifier: Bip340EventVerifier(), + cache: MemCacheManager(), + blobCache: IdbBlobCacheManager( + factory: getIdbFactory()!, + dbName: 'my_app_blob_cache', + ), + ), +); +``` + +### Opting out + +```dart +final ndk = Ndk( + NdkConfig( + eventVerifier: Bip340EventVerifier(), + cache: MemCacheManager(), + blobCache: const NoopBlobCacheManager(), + ), +); +``` + +### How Blossom uses the cache + +Once configured, the cache is consulted automatically by [Blossom](/usecases/blossom.md): + +| Operation | Cache behaviour | +|---|---| +| `getBlob(sha256)` | check cache → on miss, fetch from server → save to cache | +| `downloadBlobToFile(...)` | check cache → on hit write bytes to file ; on miss stream from server (no auto-cache, would defeat streaming) | +| `uploadBlob(data)` | save to cache **before** the upload (local-first — the cache reflects what the user has, regardless of server outcome) | +| `uploadBlobFromFile(...)` | no auto-cache (streaming) | +| `deleteBlob(sha256)` | invalidates the cached entry | + +`getBlob` and `uploadBlob` accept `cacheWrite: false` to skip the save step for one-off operations: + +```dart +await ndk.blossom.getBlob( + sha256: hash, + serverUrls: [...], + cacheWrite: false, +); +``` + +### Direct cache access + +The cache is exposed via `ndk.config.blobCache`: + +```dart +final cache = ndk.config.blobCache!; + +await cache.saveBlob(data: bytes, mimeType: 'image/png'); +final all = await cache.listBlobs(); +final size = await cache.getTotalSize(); +await cache.removeBlob(sha); +``` + +### Implementing your own backend + +Implement `BlobCacheManager` directly and pass it as `blobCache`. + +> **Heads up:** the cache has no eviction or quota — it grows indefinitely. Apps caching large media should plan their own cleanup. diff --git a/doc/usecases/blossom.md b/doc/usecases/blossom.md index c8b582306..0d2d7720a 100644 --- a/doc/usecases/blossom.md +++ b/doc/usecases/blossom.md @@ -24,13 +24,13 @@ The auth events get automatically signed and are valid for: upload a blob, if serverMediaOptimisation is set to `true` the `/media` endpoint is used. -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="46-58" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="87-95" title="" ::: #### getBlob Download the blob and use fallback if the blob is not found or the server is offline. -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="97-105" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="289-296" title="" ::: #### checkBlob @@ -38,29 +38,37 @@ Download the blob and use fallback if the blob is not found or the server is off if you have a video player that uses a url you can use check to get a valid url first. Example can be found in NDK demo app !!! -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="148-159" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="430-436" title="" ::: #### getBlobStream Similar to `getBlob`, it streams the data, which is helpful for video files. -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="202-211" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="483-490" title="" ::: #### listBlobs -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="254-264" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="538-545" title="" ::: #### deleteBlob -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="301-308" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="591-595" title="" ::: #### directDownload -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="341-344" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="631-635" title="" ::: #### report -:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="348-362" title="" ::: +:::code source="../../packages/ndk/lib/domain_layer/usecases/files/blossom.dart" language="dart" range="654-661" title="" ::: + +### Local cache + +`getBlob`, `downloadBlobToFile`, `uploadBlob` and `deleteBlob` integrate transparently with the local `BlobCacheManager`. By default everything you fetch or upload is kept in a per-`Ndk` in-memory store; configure persistence (or opt out entirely) via `NdkConfig.blobCache`. + +Both `getBlob` and `uploadBlob` accept `cacheWrite: false` for one-off operations that should not pollute the local store. + +[!ref](/guides/persistence.md) ### methods - BlossomUserServerList diff --git a/packages/ndk/lib/data_layer/repositories/blob_cache/idb_blob_cache_manager.dart b/packages/ndk/lib/data_layer/repositories/blob_cache/idb_blob_cache_manager.dart new file mode 100644 index 000000000..5c58bbdae --- /dev/null +++ b/packages/ndk/lib/data_layer/repositories/blob_cache/idb_blob_cache_manager.dart @@ -0,0 +1,221 @@ +import 'dart:typed_data'; + +import 'package:crypto/crypto.dart' as crypto; +import 'package:idb_shim/idb.dart'; + +import '../../../domain_layer/entities/blossom_blobs.dart'; +import '../../../domain_layer/repositories/blob_cache_manager.dart'; + +const String _kDataStore = 'blob_data'; +const String _kMetadataStore = 'blob_metadata'; +const int _kSchemaVersion = 1; + +/// IndexedDB-backed [BlobCacheManager], usable on web (real IndexedDB), +/// native (idb_io / idb_sqflite) and in tests (`newMemoryIdbFactory()`). +/// +/// The concrete [IdbFactory] is injected so this class stays +/// platform-agnostic; callers pick the right one for their platform. +class IdbBlobCacheManager implements BlobCacheManager { + final IdbFactory _factory; + final String _dbName; + Future? _dbFuture; + + IdbBlobCacheManager({ + required IdbFactory factory, + String dbName = 'ndk_blob_cache', + }) : _factory = factory, + _dbName = dbName; + + Future _db() { + return _dbFuture ??= _factory.open( + _dbName, + version: _kSchemaVersion, + onUpgradeNeeded: (VersionChangeEvent e) { + final db = e.database; + if (!db.objectStoreNames.contains(_kDataStore)) { + db.createObjectStore(_kDataStore); + } + if (!db.objectStoreNames.contains(_kMetadataStore)) { + db.createObjectStore(_kMetadataStore); + } + }, + ); + } + + @override + Future saveBlob({ + required Uint8List data, + String? sha256, + String? mimeType, + String? sourceUrl, + BlobNip94? nip94, + }) async { + final hash = sha256 ?? crypto.sha256.convert(data).toString(); + final descriptor = BlobDescriptor( + url: sourceUrl ?? '', + sha256: hash, + size: data.length, + type: mimeType, + uploaded: DateTime.now(), + nip94: nip94, + ); + + final db = await _db(); + final txn = db.transaction( + [_kDataStore, _kMetadataStore], + idbModeReadWrite, + ); + try { + await txn.objectStore(_kDataStore).put(data, hash); + await txn.objectStore(_kMetadataStore).put(descriptor.toJson(), hash); + } finally { + await txn.completed; + } + return descriptor; + } + + @override + Future getBlob(String sha256) async { + final db = await _db(); + final txn = db.transaction( + [_kDataStore, _kMetadataStore], + idbModeReadOnly, + ); + Object? bytes; + Object? meta; + try { + bytes = await txn.objectStore(_kDataStore).getObject(sha256); + meta = await txn.objectStore(_kMetadataStore).getObject(sha256); + } finally { + await txn.completed; + } + if (bytes == null) return null; + final data = _asUint8List(bytes); + final descriptor = meta is Map + ? BlobDescriptor.fromJson(Map.from(meta)) + : null; + return BlobResponse( + data: data, + mimeType: descriptor?.type, + contentLength: data.length, + ); + } + + @override + Future hasBlob(String sha256) async { + final db = await _db(); + final txn = db.transaction(_kMetadataStore, idbModeReadOnly); + Object? key; + try { + key = await txn.objectStore(_kMetadataStore).getKey(sha256); + } finally { + await txn.completed; + } + return key != null; + } + + @override + Future getBlobDescriptor(String sha256) async { + final db = await _db(); + final txn = db.transaction(_kMetadataStore, idbModeReadOnly); + Object? meta; + try { + meta = await txn.objectStore(_kMetadataStore).getObject(sha256); + } finally { + await txn.completed; + } + if (meta is! Map) return null; + return BlobDescriptor.fromJson(Map.from(meta)); + } + + @override + Future> listBlobs() async { + final db = await _db(); + final txn = db.transaction(_kMetadataStore, idbModeReadOnly); + List values; + try { + values = await txn.objectStore(_kMetadataStore).getAll(); + } finally { + await txn.completed; + } + final descriptors = []; + for (final v in values) { + if (v is Map) { + descriptors.add(BlobDescriptor.fromJson(Map.from(v))); + } + } + descriptors.sort((a, b) => b.uploaded.compareTo(a.uploaded)); + return descriptors; + } + + @override + Future removeBlob(String sha256) async { + final db = await _db(); + final txn = db.transaction( + [_kDataStore, _kMetadataStore], + idbModeReadWrite, + ); + try { + await txn.objectStore(_kDataStore).delete(sha256); + await txn.objectStore(_kMetadataStore).delete(sha256); + } finally { + await txn.completed; + } + } + + @override + Future removeAllBlobs() async { + final db = await _db(); + final txn = db.transaction( + [_kDataStore, _kMetadataStore], + idbModeReadWrite, + ); + try { + await txn.objectStore(_kDataStore).clear(); + await txn.objectStore(_kMetadataStore).clear(); + } finally { + await txn.completed; + } + } + + @override + Future getTotalSize() async { + // Sum the `size` field from descriptors instead of loading payloads. + final db = await _db(); + final txn = db.transaction(_kMetadataStore, idbModeReadOnly); + List values; + try { + values = await txn.objectStore(_kMetadataStore).getAll(); + } finally { + await txn.completed; + } + var total = 0; + for (final v in values) { + if (v is Map) { + final size = v['size']; + if (size is int) total += size; + } + } + return total; + } + + @override + Future close() async { + final f = _dbFuture; + if (f != null) { + _dbFuture = null; + final db = await f; + db.close(); + } + } + + /// Some [IdbFactory] implementations return `List` instead of a + /// proper [Uint8List]. Normalise so callers always get a [Uint8List]. + Uint8List _asUint8List(Object value) { + if (value is Uint8List) return value; + if (value is List) return Uint8List.fromList(value); + throw StateError( + 'Unexpected blob payload type: ${value.runtimeType}', + ); + } +} diff --git a/packages/ndk/lib/data_layer/repositories/blob_cache/noop_blob_cache_manager.dart b/packages/ndk/lib/data_layer/repositories/blob_cache/noop_blob_cache_manager.dart new file mode 100644 index 000000000..7000f6aac --- /dev/null +++ b/packages/ndk/lib/data_layer/repositories/blob_cache/noop_blob_cache_manager.dart @@ -0,0 +1,62 @@ +import 'dart:typed_data'; + +import 'package:crypto/crypto.dart' as crypto; + +import '../../../domain_layer/entities/blossom_blobs.dart'; +import '../../../domain_layer/repositories/blob_cache_manager.dart'; + +/// A [BlobCacheManager] that never stores anything. +/// +/// Useful when caching must be opted out entirely: +/// - tests asserting per-server behaviour +/// - memory- or storage-constrained environments +/// - debugging / always-fetch flows +/// +/// `saveBlob` returns a descriptor for caller convenience but discards +/// the bytes; every subsequent read returns the empty / absent state. +class NoopBlobCacheManager implements BlobCacheManager { + const NoopBlobCacheManager(); + + @override + Future saveBlob({ + required Uint8List data, + String? sha256, + String? mimeType, + String? sourceUrl, + BlobNip94? nip94, + }) async { + final hash = sha256 ?? crypto.sha256.convert(data).toString(); + return BlobDescriptor( + url: sourceUrl ?? '', + sha256: hash, + size: data.length, + type: mimeType, + uploaded: DateTime.now(), + nip94: nip94, + ); + } + + @override + Future getBlob(String sha256) async => null; + + @override + Future hasBlob(String sha256) async => false; + + @override + Future getBlobDescriptor(String sha256) async => null; + + @override + Future> listBlobs() async => const []; + + @override + Future removeBlob(String sha256) async {} + + @override + Future removeAllBlobs() async {} + + @override + Future getTotalSize() async => 0; + + @override + Future close() async {} +} diff --git a/packages/ndk/lib/domain_layer/entities/blossom_blobs.dart b/packages/ndk/lib/domain_layer/entities/blossom_blobs.dart index 9722f54a5..63020938d 100644 --- a/packages/ndk/lib/domain_layer/entities/blossom_blobs.dart +++ b/packages/ndk/lib/domain_layer/entities/blossom_blobs.dart @@ -1,13 +1,9 @@ import 'dart:typed_data'; -int _parseSize(dynamic size) { - if (size is int) { - return size; - } else if (size is String) { - return int.tryParse(size) ?? 0; - } else { - return 0; - } +int? _parseSize(dynamic size) { + if (size is int) return size; + if (size is String) return int.tryParse(size); + return null; } /// Descriptor of a blob - when getting a blob from a server @@ -52,6 +48,17 @@ class BlobDescriptor { nip94: json['nip94'] != null ? BlobNip94.fromJson(json['nip94']) : null, ); } + + /// Inverse of [BlobDescriptor.fromJson]. `uploaded` is encoded as a + /// Unix timestamp in seconds to match the server-side BUD-02 format. + Map toJson() => { + 'url': url, + 'sha256': sha256, + 'size': size, + 'type': type, + 'uploaded': uploaded.millisecondsSinceEpoch ~/ 1000, + 'nip94': nip94?.toJson(), + }; } /// Result of a blob upload @@ -118,7 +125,7 @@ class BlobNip94 { final int? size; /// size of file in pixels as String in the form <width>x<height> - final String? dimenssions; + final String? dimensions; /// URI to torrent magnet final String? magnet; @@ -162,7 +169,7 @@ class BlobNip94 { this.alt, this.fallback, this.service, - this.dimenssions, + this.dimensions, }); /// converts json response to BlobNip94, \ @@ -176,7 +183,7 @@ class BlobNip94 { sha256: json['x'] ?? '', // parse int from string size: _parseSize(json['size']), - dimenssions: json['dim'].toString(), + dimensions: json['dim']?.toString(), magnet: json['magnet'], torrentInfoHash: json['i'], blurhash: json['blurhash'], @@ -188,4 +195,31 @@ class BlobNip94 { service: json['service'], ); } + + /// Inverse of [BlobNip94.fromJson]. Uses BUD-08 / NIP-94 short tag + /// keys (`m`, `x`, `i`, `dim`, ...). `thumbnail`, `image` and + /// `fallback` are encoded as a single string (their first element) + /// since [BlobNip94.fromJson] only consumes one. + Map toJson() => { + 'content': content, + 'url': url, + 'm': mimeType, + 'x': sha256, + 'size': size, + 'dim': dimensions, + 'magnet': magnet, + 'i': torrentInfoHash, + 'blurhash': blurhash, + 'thumb': (thumbnail != null && thumbnail!.isNotEmpty) + ? thumbnail!.first + : null, + 'image': + (image != null && image!.isNotEmpty) ? image!.first : null, + 'summary': summary, + 'alt': alt, + 'fallback': (fallback != null && fallback!.isNotEmpty) + ? fallback!.first + : null, + 'service': service, + }; } diff --git a/packages/ndk/lib/domain_layer/repositories/blob_cache_manager.dart b/packages/ndk/lib/domain_layer/repositories/blob_cache_manager.dart new file mode 100644 index 000000000..b5f033a81 --- /dev/null +++ b/packages/ndk/lib/domain_layer/repositories/blob_cache_manager.dart @@ -0,0 +1,58 @@ +import 'dart:typed_data'; + +import '../entities/blossom_blobs.dart'; + +/// Local store for binary blobs, content-addressed by SHA-256. +/// +/// Conceptually a *local Blossom server* (without the network layer): the +/// API mirrors the operations a remote Blossom server exposes +/// (save/get/has/list/remove) so cached and remote blobs share the same +/// vocabulary and entities ([BlobDescriptor], [BlobResponse]). +/// +/// Differences with a real Blossom server: +/// - no HTTP, no port, no kind 24242 authorization (BUD-11) +/// - single local store: no `serverUrls`, no `pubkey` filter on list +/// - mirror, media optimization, reports and payments are not in scope +/// - [BlobDescriptor.url] of cached entries is empty, or carries the +/// original source URL when the caller passes one +abstract class BlobCacheManager { + /// Persist a blob keyed by its SHA-256. + /// + /// If [sha256] is omitted it is computed from [data]. Pass it when + /// already known (e.g. right after a Blossom GET) to avoid hashing + /// twice. The caller is trusted not to lie about a provided hash. + /// + /// Re-saving an existing blob updates the descriptor metadata and + /// keeps the existing bytes. + Future saveBlob({ + required Uint8List data, + String? sha256, + String? mimeType, + String? sourceUrl, + BlobNip94? nip94, + }); + + /// Read a blob by its SHA-256. Returns `null` when absent. + Future getBlob(String sha256); + + /// True when a blob with this SHA-256 is in the cache. + Future hasBlob(String sha256); + + /// Read the descriptor only (no bytes). Returns `null` when absent. + Future getBlobDescriptor(String sha256); + + /// Enumerate every blob currently stored, newest first. + Future> listBlobs(); + + /// Delete a blob by its SHA-256. No-op if absent. + Future removeBlob(String sha256); + + /// Delete every blob. + Future removeAllBlobs(); + + /// Total size in bytes of all stored blob payloads. + Future getTotalSize(); + + /// Release any underlying resources (db handles, etc.). + Future close(); +} diff --git a/packages/ndk/lib/domain_layer/usecases/files/blossom.dart b/packages/ndk/lib/domain_layer/usecases/files/blossom.dart index 083d94687..5a11f5cd8 100644 --- a/packages/ndk/lib/domain_layer/usecases/files/blossom.dart +++ b/packages/ndk/lib/domain_layer/usecases/files/blossom.dart @@ -2,6 +2,7 @@ import 'dart:typed_data'; import 'package:crypto/crypto.dart'; import '../../../config/blossom_config.dart'; +import '../../../data_layer/io/file_io.dart'; import '../../../data_layer/repositories/signers/bip340_event_signer.dart'; import '../../../shared/nips/nip01/bip340.dart'; import '../../entities/blob_upload_progress.dart'; @@ -9,6 +10,7 @@ import '../../entities/blossom_blobs.dart'; import '../../entities/blossom_strategies.dart'; import '../../entities/nip_01_event.dart'; import '../../entities/nip_01_utils.dart'; +import '../../repositories/blob_cache_manager.dart'; import '../../repositories/blossom.dart'; import '../../repositories/event_signer.dart'; import '../accounts/accounts.dart'; @@ -32,14 +34,20 @@ class Blossom { final BlossomUserServerList _userServerList; final BlossomRepository _blossomImpl; final Accounts _accounts; + final BlobCacheManager _blobCache; + final FileIO _fileIO; Blossom({ required BlossomUserServerList blossomUserServerList, required BlossomRepository blossomRepository, required Accounts accounts, + required BlobCacheManager blobCache, + required FileIO fileIO, }) : _accounts = accounts, _userServerList = blossomUserServerList, - _blossomImpl = blossomRepository; + _blossomImpl = blossomRepository, + _blobCache = blobCache, + _fileIO = fileIO; /// Gets the signer to use for blossom operations /// Priority: customSigner > logged in account signer > temporary signer @@ -65,6 +73,17 @@ class Blossom { /// if no signer is available, a temporary signer is created \ /// [strategy] is the upload strategy, default is mirrorAfterSuccess \ /// [serverMediaOptimisation] is whether the server should optimise the media [BUD-05], IMPORTANT: the server hash will be different \ + /// \ + /// When [cacheWrite] is true (the default) the local bytes are + /// written to the [BlobCacheManager] **before** the upload starts — + /// local-first: the cache reflects what the user has, regardless of + /// what any remote server eventually accepts. As a result a subsequent + /// [getBlob] for the same sha256 is served locally even if every + /// server rejected the upload. Set [cacheWrite] to `false` for + /// fire-and-forget uploads (sensitive data, very large media...) that + /// should not stay in the local store. Note that with + /// [serverMediaOptimisation] the server's returned hash differs from + /// the local one; the cache only knows about the original bytes. Future> uploadBlob({ required Uint8List data, List? serverUrls, @@ -72,6 +91,7 @@ class Blossom { UploadStrategy strategy = UploadStrategy.mirrorAfterSuccess, bool serverMediaOptimisation = false, EventSigner? customSigner, + bool cacheWrite = true, }) async { final now = DateTime.now().millisecondsSinceEpoch ~/ 1000; @@ -102,6 +122,14 @@ class Blossom { throw Exception("User has no server list"); } + if (cacheWrite) { + await _blobCache.saveBlob( + data: data, + sha256: dataSha256.toString(), + mimeType: contentType, + ); + } + final stream = _blossomImpl.uploadBlob( dataStreamFactory: () => Stream.value(data), contentLength: data.length, @@ -113,7 +141,6 @@ class Blossom { ); final done = await stream.last; - return done.completedUploads; } @@ -252,14 +279,24 @@ class Blossom { /// Gets a blob by trying servers sequentially until success (fallback) \ /// if [serverUrls] is null, the userServerList is fetched from nostr. \ - /// if the pukey has no UserServerList (kind: 10063), throws an error + /// if the pukey has no UserServerList (kind: 10063), throws an error \ + /// \ + /// The local [BlobCacheManager] is consulted first; on a cache miss the + /// download is performed and (when [cacheWrite] is true) the bytes are + /// written back to the cache. Set [cacheWrite] to `false` for one-off + /// fetches that should not pollute the local store. To opt out of + /// caching entirely use a `NoopBlobCacheManager` in [NdkConfig]. Future getBlob({ required String sha256, bool useAuth = false, List? serverUrls, String? pubkeyToFetchUserServerList, EventSigner? customSigner, + bool cacheWrite = true, }) async { + final cached = await _blobCache.getBlob(sha256); + if (cached != null) return cached; + Nip01Event? myAuthorization; Nip01Event? signedAuthorization; @@ -296,11 +333,21 @@ class Blossom { throw Exception("User has no server list"); } - return _blossomImpl.getBlob( + final response = await _blossomImpl.getBlob( sha256: sha256, authorization: signedAuthorization, serverUrls: serverUrls, ); + + if (cacheWrite) { + await _blobCache.saveBlob( + data: response.data, + sha256: sha256, + mimeType: response.mimeType, + ); + } + + return response; } /// Downloads a blob directly to a file path (without loading into memory) @@ -308,7 +355,14 @@ class Blossom { /// For web: triggers browser download dialog to save the file /// /// if [serverUrls] is null, the userServerList is fetched from nostr. \ - /// if the pubkey has no UserServerList (kind: 10063), throws an error + /// if the pubkey has no UserServerList (kind: 10063), throws an error \ + /// \ + /// On a cache hit the cached bytes are written to [outputPath] and no + /// network request is made. On a cache miss the file is streamed to + /// disk by the underlying repository and is **not** added to the cache + /// (would require reading the whole file back into memory). Use + /// [getBlob] when in-memory caching is desired. To opt out of caching + /// entirely use a `NoopBlobCacheManager` in [NdkConfig]. Future downloadBlobToFile({ required String sha256, required String outputPath, @@ -317,6 +371,12 @@ class Blossom { String? pubkeyToFetchUserServerList, EventSigner? customSigner, }) async { + final cached = await _blobCache.getBlob(sha256); + if (cached != null) { + await _fileIO.writeFile(outputPath, cached.data); + return; + } + Nip01Event? myAuthorization; Nip01Event? signedAuthorization; @@ -524,12 +584,17 @@ class Blossom { /// if [serverUrls] is null, the userServerList is fetched from nostr. \ /// if the pukey has no UserServerList (kind: 10063), throws an error \ /// the current signer is used to sign the request, or [customSigner] if provided \ - /// if no signer is available, a temporary signer is created + /// if no signer is available, a temporary signer is created \ + /// \ + /// The local [BlobCacheManager] entry for this sha256 is also removed + /// so subsequent reads do not serve stale bytes. Future> deleteBlob({ required String sha256, List? serverUrls, EventSigner? customSigner, }) async { + await _blobCache.removeBlob(sha256); + final now = DateTime.now().millisecondsSinceEpoch ~/ 1000; final signer = _getSigner(customSigner); diff --git a/packages/ndk/lib/ndk.dart b/packages/ndk/lib/ndk.dart index ecb1c0f9f..df32203fd 100644 --- a/packages/ndk/lib/ndk.dart +++ b/packages/ndk/lib/ndk.dart @@ -45,6 +45,9 @@ export 'domain_layer/entities/blossom_blobs.dart'; export 'domain_layer/entities/blossom_strategies.dart'; export 'domain_layer/entities/blob_upload_progress.dart'; export 'domain_layer/entities/file_hash_progress.dart'; +export 'domain_layer/repositories/blob_cache_manager.dart'; +export 'data_layer/repositories/blob_cache/idb_blob_cache_manager.dart'; +export 'data_layer/repositories/blob_cache/noop_blob_cache_manager.dart'; export ''; export 'domain_layer/entities/account.dart'; diff --git a/packages/ndk/lib/presentation_layer/init.dart b/packages/ndk/lib/presentation_layer/init.dart index 1fc452f3f..64350b306 100644 --- a/packages/ndk/lib/presentation_layer/init.dart +++ b/packages/ndk/lib/presentation_layer/init.dart @@ -1,5 +1,7 @@ import 'package:http/http.dart' as http; +import 'package:idb_shim/idb_client_memory.dart'; +import '../data_layer/repositories/blob_cache/idb_blob_cache_manager.dart'; import '../data_layer/repositories/cashu_seed_secret_generator/dart_cashu_key_derivation.dart'; import '../data_layer/repositories/wallets/wallets_repo_stub.dart'; import '../shared/net/user_agent.dart'; @@ -15,6 +17,7 @@ import '../domain_layer/entities/jit_engine_relay_connectivity_data.dart'; import '../domain_layer/entities/wallet/providers/cashu/cashu_wallet_provider.dart'; import '../domain_layer/entities/wallet/providers/nwc/nwc_wallet_provider.dart'; import '../domain_layer/entities/wallet/providers/lnurl/lnurl_wallet_provider.dart'; +import '../domain_layer/repositories/blob_cache_manager.dart'; import '../domain_layer/repositories/blossom.dart'; import '../domain_layer/repositories/cashu_repo.dart'; import '../domain_layer/repositories/lnurl_transport.dart'; @@ -73,6 +76,7 @@ class Initialization { /// use cases late RelayManager relayManager; + late BlobCacheManager blobCache; late CacheWrite cacheWrite; late CacheRead cacheRead; late Requests requests; @@ -158,11 +162,20 @@ class Initialization { final Nip05Repository nip05repository = Nip05HttpRepositoryImpl(httpDS: _httpRequestDS); + final fileIO = createFileIO(); + final BlossomRepository blossomRepository = BlossomRepositoryImpl( client: _httpRequestDS, - fileIO: createFileIO(), + fileIO: fileIO, ); + // Fresh in-memory factory per Ndk instance, otherwise the + // process-wide `idbFactoryMemory` singleton would leak cached + // blobs across instances (and across tests). + blobCache = _ndkConfig.blobCache ?? + IdbBlobCacheManager(factory: newIdbFactoryMemory()); + _ndkConfig.blobCache = blobCache; + final CashuRepo cashuRepo = CashuRepoImpl( client: _httpRequestDS, ); @@ -285,6 +298,8 @@ class Initialization { blossomRepository: blossomRepository, accounts: accounts, blossomUserServerList: blossomUserServerList, + blobCache: blobCache, + fileIO: fileIO, ); files = Files(blossom: blossom); diff --git a/packages/ndk/lib/presentation_layer/ndk_config.dart b/packages/ndk/lib/presentation_layer/ndk_config.dart index 56d027499..5aac9d502 100644 --- a/packages/ndk/lib/presentation_layer/ndk_config.dart +++ b/packages/ndk/lib/presentation_layer/ndk_config.dart @@ -6,6 +6,7 @@ import '../config/request_defaults.dart'; import '../domain_layer/entities/cashu/cashu_user_seedphrase.dart'; import '../domain_layer/entities/event_filter.dart'; import '../domain_layer/entities/nip_85.dart'; +import '../domain_layer/repositories/blob_cache_manager.dart'; import '../domain_layer/repositories/cache_manager.dart'; import '../domain_layer/repositories/event_verifier.dart'; import '../domain_layer/repositories/wallets_repo.dart'; @@ -22,6 +23,16 @@ class NdkConfig { /// The cache manager (DB) used to store and retrieve Nostr data. E.g MemCacheManager() CacheManager cache; + /// Local store for binary blobs, content-addressed by SHA-256. + /// + /// Conceptually a local Blossom server: cached blobs are exposed via + /// the same vocabulary as remote ones (BlobDescriptor, BlobResponse). + /// When `null`, [Initialization] falls back to an in-memory + /// [IdbBlobCacheManager] (lost on process exit). Provide your own + /// instance — typically `IdbBlobCacheManager(factory: getIdbFactoryIo())` + /// on native or `getIdbFactory()` on web — for persistent storage. + BlobCacheManager? blobCache; + /// The wallets repository used to manage wallet data. E.g MemWalletsRepo() WalletsRepo? walletsRepo; @@ -98,6 +109,7 @@ class NdkConfig { NdkConfig({ required this.eventVerifier, required this.cache, + this.blobCache, this.walletsRepo, this.engine = NdkEngine.RELAY_SETS, this.ignoreRelays = const [], diff --git a/packages/ndk/pubspec.yaml b/packages/ndk/pubspec.yaml index 4b1bdaeab..01b399b4e 100644 --- a/packages/ndk/pubspec.yaml +++ b/packages/ndk/pubspec.yaml @@ -49,6 +49,7 @@ dependencies: web: ^1.1.1 sembast: ">=3.8.6 <4.0.0" sembast_web: ^2.4.4+1 + idb_shim: ^2.9.1 path: ^1.9.1 dev_dependencies: diff --git a/packages/ndk/test/data_layer/blob_cache/idb_blob_cache_manager_test.dart b/packages/ndk/test/data_layer/blob_cache/idb_blob_cache_manager_test.dart new file mode 100644 index 000000000..118087a0d --- /dev/null +++ b/packages/ndk/test/data_layer/blob_cache/idb_blob_cache_manager_test.dart @@ -0,0 +1,168 @@ +import 'dart:typed_data'; + +import 'package:crypto/crypto.dart' as crypto; +import 'package:idb_shim/idb_client_memory.dart'; +import 'package:ndk/data_layer/repositories/blob_cache/idb_blob_cache_manager.dart'; +import 'package:ndk/domain_layer/entities/blossom_blobs.dart'; +import 'package:ndk/domain_layer/repositories/blob_cache_manager.dart'; +import 'package:test/test.dart'; + +void main() { + late BlobCacheManager cache; + + setUp(() { + // Fresh in-memory factory per test → isolated db. + cache = IdbBlobCacheManager(factory: newIdbFactoryMemory()); + }); + + tearDown(() async { + await cache.close(); + }); + + Uint8List bytes(String s) => Uint8List.fromList(s.codeUnits); + String hashOf(Uint8List data) => crypto.sha256.convert(data).toString(); + + group('saveBlob', () { + test('computes sha256 from data when not provided', () async { + final data = bytes('hello blob'); + final descriptor = await cache.saveBlob(data: data); + expect(descriptor.sha256, hashOf(data)); + expect(descriptor.size, data.length); + expect(descriptor.url, ''); + expect(descriptor.type, isNull); + }); + + test('uses provided sha256 without recomputing', () async { + final data = bytes('hello blob'); + // Pass an arbitrary "sha256" — the cache trusts the caller. + final descriptor = + await cache.saveBlob(data: data, sha256: 'trusted-key'); + expect(descriptor.sha256, 'trusted-key'); + }); + + test('round-trips mime type, source url and nip94', () async { + final data = bytes('payload'); + final nip94 = BlobNip94( + content: '', + url: 'https://cdn.example.com/x', + mimeType: 'image/png', + sha256: 'aaaa', + size: data.length, + dimensions: '100x100', + ); + await cache.saveBlob( + data: data, + mimeType: 'image/png', + sourceUrl: 'https://cdn.example.com/x', + nip94: nip94, + ); + final descriptor = await cache.getBlobDescriptor(hashOf(data)); + expect(descriptor, isNotNull); + expect(descriptor!.type, 'image/png'); + expect(descriptor.url, 'https://cdn.example.com/x'); + expect(descriptor.nip94, isNotNull); + expect(descriptor.nip94!.dimensions, '100x100'); + }); + + test('overwrites an existing blob with the same sha256', () async { + final hash = 'reused-key'; + await cache.saveBlob(data: bytes('first'), sha256: hash); + await cache.saveBlob( + data: bytes('second'), + sha256: hash, + mimeType: 'text/plain', + ); + final response = await cache.getBlob(hash); + expect(response, isNotNull); + expect(response!.data, bytes('second')); + expect(response.mimeType, 'text/plain'); + }); + }); + + group('getBlob / hasBlob / getBlobDescriptor', () { + test('returns null / false when absent', () async { + expect(await cache.getBlob('missing'), isNull); + expect(await cache.hasBlob('missing'), isFalse); + expect(await cache.getBlobDescriptor('missing'), isNull); + }); + + test('returns the persisted bytes on hit', () async { + final data = bytes('contents'); + final hash = hashOf(data); + await cache.saveBlob(data: data, mimeType: 'text/plain'); + final response = await cache.getBlob(hash); + expect(response, isNotNull); + expect(response!.data, data); + expect(response.contentLength, data.length); + expect(response.mimeType, 'text/plain'); + expect(await cache.hasBlob(hash), isTrue); + }); + }); + + group('listBlobs', () { + test('returns an empty list when the cache is empty', () async { + expect(await cache.listBlobs(), isEmpty); + }); + + test('returns all stored descriptors, newest first', () async { + await cache.saveBlob(data: bytes('first'), sha256: 'a'); + // Wait > 1s so the unix-second timestamp differs (BlobDescriptor + // serialises `uploaded` to seconds, dropping sub-second precision). + await Future.delayed(const Duration(milliseconds: 1100)); + await cache.saveBlob(data: bytes('second'), sha256: 'b'); + final list = await cache.listBlobs(); + expect(list.map((d) => d.sha256).toList(), ['b', 'a']); + }); + }); + + group('removeBlob / removeAllBlobs', () { + test('removeBlob is a no-op when the blob is absent', () async { + await cache.removeBlob('missing'); + expect(await cache.listBlobs(), isEmpty); + }); + + test('removeBlob removes both bytes and descriptor', () async { + final data = bytes('to-be-removed'); + final hash = hashOf(data); + await cache.saveBlob(data: data); + await cache.removeBlob(hash); + expect(await cache.hasBlob(hash), isFalse); + expect(await cache.getBlob(hash), isNull); + expect(await cache.getBlobDescriptor(hash), isNull); + }); + + test('removeAllBlobs empties the store', () async { + await cache.saveBlob(data: bytes('a'), sha256: '1'); + await cache.saveBlob(data: bytes('b'), sha256: '2'); + await cache.removeAllBlobs(); + expect(await cache.listBlobs(), isEmpty); + expect(await cache.getTotalSize(), 0); + }); + }); + + group('getTotalSize', () { + test('returns 0 when the cache is empty', () async { + expect(await cache.getTotalSize(), 0); + }); + + test('sums the size of every stored blob', () async { + await cache.saveBlob(data: bytes('aaa'), sha256: '1'); + await cache.saveBlob(data: bytes('bbbb'), sha256: '2'); + expect(await cache.getTotalSize(), 7); + }); + }); + + test('separate IdbBlobCacheManager instances have isolated stores', () async { + // Each `newIdbFactoryMemory()` returns a fresh memory db. + final a = IdbBlobCacheManager(factory: newIdbFactoryMemory()); + final b = IdbBlobCacheManager(factory: newIdbFactoryMemory()); + addTearDown(() async { + await a.close(); + await b.close(); + }); + + await a.saveBlob(data: bytes('only-in-a'), sha256: 'x'); + expect(await a.hasBlob('x'), isTrue); + expect(await b.hasBlob('x'), isFalse); + }); +} diff --git a/packages/ndk/test/data_layer/blob_cache/noop_blob_cache_manager_test.dart b/packages/ndk/test/data_layer/blob_cache/noop_blob_cache_manager_test.dart new file mode 100644 index 000000000..3d84c8133 --- /dev/null +++ b/packages/ndk/test/data_layer/blob_cache/noop_blob_cache_manager_test.dart @@ -0,0 +1,55 @@ +import 'dart:typed_data'; + +import 'package:crypto/crypto.dart' as crypto; +import 'package:ndk/data_layer/repositories/blob_cache/noop_blob_cache_manager.dart'; +import 'package:test/test.dart'; + +void main() { + const cache = NoopBlobCacheManager(); + + Uint8List bytes(String s) => Uint8List.fromList(s.codeUnits); + + test('saveBlob computes the descriptor but discards the data', () async { + final data = bytes('payload'); + final descriptor = await cache.saveBlob( + data: data, + mimeType: 'text/plain', + sourceUrl: 'https://cdn.example.com/x', + ); + expect(descriptor.sha256, crypto.sha256.convert(data).toString()); + expect(descriptor.size, data.length); + expect(descriptor.type, 'text/plain'); + expect(descriptor.url, 'https://cdn.example.com/x'); + + // Despite saveBlob succeeding, nothing is stored. + expect(await cache.hasBlob(descriptor.sha256), isFalse); + expect(await cache.getBlob(descriptor.sha256), isNull); + expect(await cache.getBlobDescriptor(descriptor.sha256), isNull); + }); + + test('saveBlob honours an explicit sha256', () async { + final descriptor = + await cache.saveBlob(data: bytes('x'), sha256: 'forced-key'); + expect(descriptor.sha256, 'forced-key'); + }); + + test('every read returns the empty / absent state', () async { + expect(await cache.getBlob('any'), isNull); + expect(await cache.hasBlob('any'), isFalse); + expect(await cache.getBlobDescriptor('any'), isNull); + expect(await cache.listBlobs(), isEmpty); + expect(await cache.getTotalSize(), 0); + }); + + test('remove operations are no-ops and never throw', () async { + await cache.removeBlob('missing'); + await cache.removeAllBlobs(); + expect(await cache.listBlobs(), isEmpty); + }); + + test('close is a no-op', () async { + await cache.close(); + // Still usable after close. + expect(await cache.hasBlob('x'), isFalse); + }); +} diff --git a/packages/ndk/test/entities/blob_nip94_test.dart b/packages/ndk/test/entities/blob_nip94_test.dart index 570334d76..55bd282e4 100644 --- a/packages/ndk/test/entities/blob_nip94_test.dart +++ b/packages/ndk/test/entities/blob_nip94_test.dart @@ -85,8 +85,8 @@ void main() async { final result = BlobDescriptor.fromJson(malformedJson); final result2 = BlobDescriptor.fromJson(malformedJson2); - expect(result.nip94!.dimenssions, equals("1920x1080")); - expect(result2.nip94!.dimenssions, equals("100")); + expect(result.nip94!.dimensions, equals("1920x1080")); + expect(result2.nip94!.dimensions, equals("100")); }); test( @@ -107,12 +107,147 @@ void main() async { final obj = jsonDecode(json); final result = BlobNip94.fromJson(obj); - expect(result.dimenssions, equals("590x1280")); + expect(result.dimensions, equals("590x1280")); // replaced by second instance (BAD!!) // https://github.com/hzrd149/blossom/pull/60 expect(result.fallback!.first, equals("https://nostr.download/bbbb.mp4")); expect(result.thumbnail!.first, equals("https://nostr.download/thumb/aaaa.webp")); }); + + test('size is null when missing or unparseable, not 0', () { + final missing = BlobDescriptor.fromJson({ + "url": '', + "sha256": '', + "type": "notype", + }); + final garbage = BlobDescriptor.fromJson({ + "url": '', + "sha256": '', + "size": "not-a-number", + "type": "notype", + }); + expect(missing.size, isNull); + expect(garbage.size, isNull); + }); + + test('dimensions is null when dim is missing, not the string "null"', () { + final result = BlobNip94.fromJson({ + "url": '', + "x": '', + "m": '', + }); + expect(result.dimensions, isNull); + }); + }); + + group('toJson', () { + test('BlobDescriptor.toJson encodes uploaded as unix seconds', () { + final descriptor = BlobDescriptor( + url: 'https://cdn.example.com/abc', + sha256: 'abc', + size: 42, + type: 'image/png', + uploaded: DateTime.fromMillisecondsSinceEpoch(1700000000 * 1000), + ); + final json = descriptor.toJson(); + expect(json['url'], 'https://cdn.example.com/abc'); + expect(json['sha256'], 'abc'); + expect(json['size'], 42); + expect(json['type'], 'image/png'); + expect(json['uploaded'], 1700000000); + expect(json['nip94'], isNull); + }); + + test('BlobDescriptor roundtrips through toJson / fromJson', () { + final original = BlobDescriptor( + url: 'https://cdn.example.com/abc', + sha256: + '0000000000000000000000000000000000000000000000000000000000000000', + size: 1234, + type: 'video/mp4', + // truncate to second precision since toJson stores unix seconds + uploaded: DateTime.fromMillisecondsSinceEpoch(1700000000 * 1000), + ); + final restored = BlobDescriptor.fromJson(original.toJson()); + expect(restored.url, original.url); + expect(restored.sha256, original.sha256); + expect(restored.size, original.size); + expect(restored.type, original.type); + expect(restored.uploaded, original.uploaded); + expect(restored.nip94, isNull); + }); + + test('BlobNip94.toJson uses NIP-94 short tag keys', () { + final nip94 = BlobNip94( + content: 'a video', + url: 'https://cdn.example.com/v.mp4', + mimeType: 'video/mp4', + sha256: 'deadbeef', + size: 999, + dimensions: '1920x1080', + magnet: 'magnet:?xt=...', + torrentInfoHash: 'hash123', + blurhash: 'L00000', + thumbnail: ['https://cdn.example.com/thumb.png'], + image: ['https://cdn.example.com/preview.png'], + fallback: ['https://mirror.example.com/v.mp4'], + summary: 'a short clip', + alt: 'description', + service: 'nip-96', + ); + final json = nip94.toJson(); + expect(json['m'], 'video/mp4'); + expect(json['x'], 'deadbeef'); + expect(json['i'], 'hash123'); + expect(json['dim'], '1920x1080'); + expect(json['size'], 999); + expect(json['thumb'], 'https://cdn.example.com/thumb.png'); + expect(json['image'], 'https://cdn.example.com/preview.png'); + expect(json['fallback'], 'https://mirror.example.com/v.mp4'); + expect(json['service'], 'nip-96'); + }); + + test('BlobNip94.toJson emits null for empty/absent list fields', () { + final nip94 = BlobNip94( + content: '', + url: '', + mimeType: '', + sha256: '', + size: null, + ); + final json = nip94.toJson(); + expect(json['thumb'], isNull); + expect(json['image'], isNull); + expect(json['fallback'], isNull); + expect(json['dim'], isNull); + }); + + test('BlobDescriptor with nested nip94 roundtrips', () { + final original = BlobDescriptor( + url: 'https://cdn.example.com/v.mp4', + sha256: 'aaaa', + size: 6762754, + type: 'video/mp4', + uploaded: DateTime.fromMillisecondsSinceEpoch(1700000000 * 1000), + nip94: BlobNip94( + content: '', + url: 'https://cdn.example.com/v.mp4', + mimeType: 'video/mp4', + sha256: 'aaaa', + size: 6762754, + dimensions: '590x1280', + thumbnail: ['https://cdn.example.com/thumb.webp'], + ), + ); + final restored = BlobDescriptor.fromJson(original.toJson()); + expect(restored.nip94, isNotNull); + expect(restored.nip94!.mimeType, 'video/mp4'); + expect(restored.nip94!.sha256, 'aaaa'); + expect(restored.nip94!.size, 6762754); + expect(restored.nip94!.dimensions, '590x1280'); + expect(restored.nip94!.thumbnail!.first, + 'https://cdn.example.com/thumb.webp'); + }); }); } diff --git a/packages/ndk/test/usecases/files/blossom_blob_cache_test.dart b/packages/ndk/test/usecases/files/blossom_blob_cache_test.dart new file mode 100644 index 000000000..0377c9362 --- /dev/null +++ b/packages/ndk/test/usecases/files/blossom_blob_cache_test.dart @@ -0,0 +1,354 @@ +import 'dart:convert'; +import 'dart:io'; +import 'dart:typed_data'; + +import 'package:ndk/ndk.dart'; +import 'package:ndk/shared/nips/nip01/bip340.dart'; +import 'package:test/test.dart'; + +import '../../mocks/mock_blossom_server.dart'; +import '../../mocks/mock_event_verifier.dart'; + +const int port = 30100; + +void main() { + late MockBlossomServer server; + late Ndk ndk; + late Blossom blossom; + late BlobCacheManager blobCache; + + setUp(() async { + server = MockBlossomServer(port: port); + await server.start(); + + final key = Bip340.generatePrivateKey(); + ndk = Ndk( + NdkConfig( + eventVerifier: MockEventVerifier(), + cache: MemCacheManager(), + engine: NdkEngine.JIT, + ), + ); + ndk.accounts.loginPrivateKey( + pubkey: key.publicKey, + privkey: key.privateKey!, + ); + + blossom = ndk.blossom; + // Default fallback in Initialization populates this with a fresh + // in-memory IdbBlobCacheManager (via newIdbFactoryMemory()). + blobCache = ndk.config.blobCache!; + }); + + tearDown(() async { + await server.stop(); + await blobCache.close(); + await ndk.destroy(); + }); + + Uint8List bytesOf(String s) => Uint8List.fromList(utf8.encode(s)); + + // Test helper — uploads to the mock server without populating the + // local cache, so getBlob tests start from a known-empty state. + // Tests that exercise upload-caching behaviour call uploadBlob + // directly with their own cacheWrite value. + Future uploadAndGetSha(String payload) async { + final response = await blossom.uploadBlob( + data: bytesOf(payload), + serverUrls: ['http://localhost:$port'], + cacheWrite: false, + ); + return response.first.descriptor!.sha256; + } + + test('config exposes the default in-memory blob cache instance', () { + expect(ndk.config.blobCache, isNotNull); + expect(ndk.config.blobCache, same(blobCache)); + }); + + group('getBlob', () { + test('caches the bytes after a cache miss', () async { + final sha = await uploadAndGetSha('cache me'); + expect(await blobCache.hasBlob(sha), isFalse, + reason: 'cache should be empty before the first get'); + + await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + + expect(await blobCache.hasBlob(sha), isTrue, + reason: 'cache should be populated after a successful get'); + final descriptor = await blobCache.getBlobDescriptor(sha); + expect(descriptor!.size, 'cache me'.length); + }); + + test('serves from the cache without hitting the server', () async { + final sha = await uploadAndGetSha('cached payload'); + + // First get → server hit + cache save. + final first = await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(utf8.decode(first.data), 'cached payload'); + + // Stop the server: a second get must succeed from the cache. + await server.stop(); + final second = await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(utf8.decode(second.data), 'cached payload'); + }); + + test('after removeBlob the next get hits the server again', () async { + final sha = await uploadAndGetSha('payload'); + + await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + // Drop the cache entry then stop the server: the next get has + // nowhere to source the bytes from and must throw. + await blobCache.removeBlob(sha); + await server.stop(); + final attempt = blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(attempt, throwsException); + }); + + test('user-managed cache is consulted by Blossom', () async { + // The user pre-populates the cache with arbitrary bytes for an + // arbitrary sha — Blossom should serve them without ever + // contacting the server. + const sha = 'user-injected-key'; + final data = bytesOf('injected from outside'); + await blobCache.saveBlob( + data: data, + sha256: sha, + mimeType: 'text/plain', + ); + + // Server has no idea about this sha. + final response = await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(response.data, data); + expect(response.mimeType, 'text/plain'); + }); + }); + + group('downloadBlobToFile', () { + late Directory tmp; + + setUp(() async { + tmp = await Directory.systemTemp.createTemp('ndk_blob_cache_test_'); + }); + + tearDown(() async { + if (tmp.existsSync()) { + await tmp.delete(recursive: true); + } + }); + + test('writes the cached bytes to disk on cache hit', () async { + final sha = await uploadAndGetSha('to disk from cache'); + // Populate cache via getBlob first. + await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + + // Stop the server: download must come from the cache. + await server.stop(); + final out = '${tmp.path}/blob.bin'; + await blossom.downloadBlobToFile( + sha256: sha, + outputPath: out, + serverUrls: ['http://localhost:$port'], + ); + + final written = await File(out).readAsBytes(); + expect(utf8.decode(written), 'to disk from cache'); + }); + + test('after removeBlob the next downloadBlobToFile hits the server', + () async { + final sha = await uploadAndGetSha('refresh me'); + await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + + await blobCache.removeBlob(sha); + await server.stop(); + final out = '${tmp.path}/blob.bin'; + final attempt = blossom.downloadBlobToFile( + sha256: sha, + outputPath: out, + serverUrls: ['http://localhost:$port'], + ); + expect(attempt, throwsException); + }); + }); + + group('deleteBlob', () { + test('invalidates the cached entry', () async { + final sha = await uploadAndGetSha('delete me'); + await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(await blobCache.hasBlob(sha), isTrue); + + await blossom.deleteBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(await blobCache.hasBlob(sha), isFalse); + }); + }); + + group('cacheWrite', () { + test('uploadBlob auto-caches the bytes by default', () async { + final data = bytesOf('upload-and-cache'); + final result = await blossom.uploadBlob( + data: data, + serverUrls: ['http://localhost:$port'], + ); + final sha = result.first.descriptor!.sha256; + expect(await blobCache.hasBlob(sha), isTrue); + + // Stop the server: getBlob must succeed from the local cache, + // proving the upload populated it. + await server.stop(); + final fetched = await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + ); + expect(fetched.data, data); + }); + + test('uploadBlob with cacheWrite:false does not populate the cache', + () async { + final result = await blossom.uploadBlob( + data: bytesOf('upload-no-cache'), + serverUrls: ['http://localhost:$port'], + cacheWrite: false, + ); + final sha = result.first.descriptor!.sha256; + expect(await blobCache.hasBlob(sha), isFalse); + }); + + test('getBlob with cacheWrite:false fetches but does not cache', + () async { + final sha = await uploadAndGetSha('fetch-no-cache'); + expect(await blobCache.hasBlob(sha), isFalse); + + await blossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$port'], + cacheWrite: false, + ); + expect(await blobCache.hasBlob(sha), isFalse); + }); + + test('uploadBlob caches even when every server rejects', () async { + // Local-first: the cache reflects what the user has, regardless + // of remote outcome. The user can retry the upload later or + // serve the bytes from their own local store. + final data = bytesOf('all-fail'); + final result = await blossom.uploadBlob( + data: data, + serverUrls: ['http://dead.example.com'], + ); + expect(result.every((r) => !r.success), isTrue, + reason: 'sanity: every server should reject'); + + final sha = result.first.descriptor?.sha256 ?? + // Failed uploads don't carry a descriptor — recompute the + // local hash to look the entry up. + (await blossom.uploadBlob( + data: data, + serverUrls: ['http://localhost:$port'], + cacheWrite: false, + )) + .first + .descriptor! + .sha256; + expect(await blobCache.hasBlob(sha), isTrue); + final fetched = await blobCache.getBlob(sha); + expect(fetched!.data, data); + }); + }); + + group('NoopBlobCacheManager opt-out', () { + late MockBlossomServer noopServer; + late Ndk noopNdk; + late Blossom noopBlossom; + const noopPort = 30101; + + setUp(() async { + noopServer = MockBlossomServer(port: noopPort); + await noopServer.start(); + + final key = Bip340.generatePrivateKey(); + noopNdk = Ndk( + NdkConfig( + eventVerifier: MockEventVerifier(), + cache: MemCacheManager(), + blobCache: const NoopBlobCacheManager(), + engine: NdkEngine.JIT, + ), + ); + noopNdk.accounts.loginPrivateKey( + pubkey: key.publicKey, + privkey: key.privateKey!, + ); + noopBlossom = noopNdk.blossom; + }); + + tearDown(() async { + await noopServer.stop(); + await noopNdk.destroy(); + }); + + test('every getBlob hits the server, nothing is stored', () async { + final upload = await noopBlossom.uploadBlob( + data: bytesOf('never cached'), + serverUrls: ['http://localhost:$noopPort'], + ); + final sha = upload.first.descriptor!.sha256; + + // Two successive gets should both reach the server (Noop returns + // null on every read). + await noopBlossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$noopPort'], + ); + await noopBlossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$noopPort'], + ); + + // After stopping the server the third get must fail — no cache + // to fall back on. + await noopServer.stop(); + final attempt = noopBlossom.getBlob( + sha256: sha, + serverUrls: ['http://localhost:$noopPort'], + ); + expect(attempt, throwsException); + + // The exposed cache instance reports nothing. + final exposed = noopNdk.config.blobCache!; + expect(await exposed.hasBlob(sha), isFalse); + expect(await exposed.listBlobs(), isEmpty); + expect(await exposed.getTotalSize(), 0); + }); + }); +} diff --git a/packages/ndk/test/usecases/files/blossom_test.dart b/packages/ndk/test/usecases/files/blossom_test.dart index 4d15859a1..4687a6406 100644 --- a/packages/ndk/test/usecases/files/blossom_test.dart +++ b/packages/ndk/test/usecases/files/blossom_test.dart @@ -38,6 +38,11 @@ void main() { NdkConfig( eventVerifier: MockEventVerifier(), cache: MemCacheManager(), + // These tests assert Blossom protocol behaviour (server fallback, + // per-server availability...) — disable the local blob cache so + // it never masks server-side state. Cache integration is + // covered separately in blossom_blob_cache_test.dart. + blobCache: const NoopBlobCacheManager(), engine: NdkEngine.JIT, ), ); @@ -276,20 +281,23 @@ void main() { final sha256 = server1Result.descriptor!.sha256; - final deadServer = client.getBlob(sha256: sha256, serverUrls: [ - 'http://dead.example.com', - ]); + final deadServer = client.getBlob( + sha256: sha256, + serverUrls: ['http://dead.example.com'], + ); expect(deadServer, throwsException); - final server1 = await client.getBlob(sha256: sha256, serverUrls: [ - 'http://localhost:${server2.port}', - ]); + final server1 = await client.getBlob( + sha256: sha256, + serverUrls: ['http://localhost:${server2.port}'], + ); expect(utf8.decode(server1.data), equals('strategy test')); - final served2 = client.getBlob(sha256: sha256, serverUrls: [ - 'http://localhost:${server.port}', - ]); + final served2 = client.getBlob( + sha256: sha256, + serverUrls: ['http://localhost:${server.port}'], + ); expect(served2, throwsException); });