fsblobstore

Check-in Differences
Login

Check-in Differences

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From fsblobstore-0.0.1 To fsblobstore-0.0.2

2024-01-30
00:02
Update to tmpfile 0.0.2. Add a factory method to use the 'small file' support of tmpfile. check-in: ef3e8cd6e8 user: jan tags: trunk
2024-01-28
23:32
Release maintenance. check-in: cfdf2cf9dd user: jan tags: fsblobstore-0.0.2, trunk
23:30
Re-export TmpFile. check-in: 20a3f8cf75 user: jan tags: trunk
18:07
Mention doc generation. check-in: 4a250096a6 user: jan tags: trunk
17:53
Include examples in packaging. check-in: 141272c25c user: jan tags: fsblobstore-0.0.1, trunk
17:52
Ignore datastore directory when packaging. check-in: 4e2b410713 user: jan tags: trunk

Changes to Cargo.toml.

1
2
3

4
5
6
7
8
9
10
1
2

3
4
5
6
7
8
9
10


-
+







[package]
name = "fsblobstore"
version = "0.0.1"
version = "0.0.2"
edition = "2021"
license = "0BSD"
categories = [ "filesystem" ]
keywords = [ "blob", "datastore" ]
repository = "https://repos.qrnch.tech/pub/fsblobstore"
description = "A file-system backed blob storage abstraction."
rust-version = "1.56"

Changes to src/ch.rs.

1
2
3
4
5
6
7

8
9
10
11
12
13
14
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15







+







use std::{
  fmt,
  hash::{Hash, Hasher},
  ops::Deref,
  str::FromStr
};

/// A hash of a content blob.
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)]
#[repr(transparent)]
pub struct ContentHash(Vec<u8>);

impl ContentHash {
  pub fn into_inner(self) -> Vec<u8> {
    self.0

Changes to src/lib.rs.

1
2












3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24

25
26
27
28
29
30
31
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

32
33
34
35
36
37
38
39
40
41
42
43
44


+
+
+
+
+
+
+
+
+
+
+
+

















-
+




+







//! A abstraction over a filesystem blob storage where each blob is
//! named/key'd by its own hash.
//!
//! # Features
//! | Feature     | Function
//! |-------------|----------
//! | `enumerate` | Enable method for enumerating all keys in storage.
//! | `get-fname` | Enable method for acquiring the path of a blob.
//!
//! The use of the `enumerate` and `get-fname` features are discouraged since
//! they may encourage breaking the intended usage pattern for `FsBlobStore`
//! instances.

#![cfg_attr(docsrs, feature(doc_cfg))]

mod ch;
mod err;

use std::{
  fs,
  path::{Path, PathBuf}
};

#[cfg(feature = "enumerate")]
use {
  std::{path::Component, thread},
  walkdir::WalkDir
};

use idbag::IdBag;

use tmpfile::{TmpFile, TmpProc};
use tmpfile::TmpProc;

use sha2::{Digest, Sha256};

pub use ch::ContentHash;
pub use tmpfile::TmpFile;

pub use err::Error;


/// Internal type used by the [`TmpFile`] to hash and move blobs into their
/// final location.
struct Hasher {
180
181
182
183
184
185
186

187
188
189
190
191
192
193
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207







+







  /// Enumerating the `FsBlobStore` is potentially slow.  Its use should be
  /// limited to infrequent integrity checks.
  ///
  /// This method will launch a background thread which lives as long as it
  /// performs its work.  It is inadvisable to allow end users to trigger this
  /// method to be run.
  #[cfg(feature = "enumerate")]
  #[cfg_attr(docsrs, doc(cfg(feature = "enumerate")))]
  pub fn enumerate(
    &self
  ) -> (recstrm::Receiver<ContentHash, ()>, thread::JoinHandle<()>) {
    let (tx, rx) = recstrm::channel::<ContentHash, ()>(32, None);
    let basedir = self.basedir.clone();
    let jh = thread::spawn(move || {
      // Send hashes in batches
261
262
263
264
265
266
267

268
269
270
271
272
273
274
275
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290







+








  ///
  /// # Caveat
  /// The use of this method is strongly discouraged.  Use
  /// `FsBlobStore::have()` to check if a blob exists in the datastore,
  /// `FsBlobStore::reader()` to read a blob, and `FsBlobStore::rm()` to remove
  /// a blob.
  #[cfg(feature = "get-fname")]
  #[cfg_attr(docsrs, doc(cfg(feature = "get-fname")))]
  pub fn get_fname(&self, hash: &[u8]) -> Result<PathBuf, std::io::Error> {
    let fname = self.abspathname(hash);
    fs::metadata(&fname)?;
    Ok(fname)
  }
}

// vim: set ft=rust et sw=2 ts=2 sts=2 cinoptions=2 tw=79 :

Changes to www/changelog.md.

1
2
3
4
5

6
7
8
9
10
11
12










13
14
15
16
17
18
1
2
3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28




-
+







+
+
+
+
+
+
+
+
+
+






# Change Log

## [Unreleased]

[Details](/vdiff?from=fsblobstore-0.0.1&to=trunk)
[Details](/vdiff?from=fsblobstore-0.0.2&to=trunk)

### Added

### Changed

### Removed

---

## [0.0.2] - 2024-01-29

[Details](/vdiff?from=fsblobstore-0.0.1&to=fsblobstore-0.0.2)

### Added

- Re-export `TmpFile`.

---

## [0.0.1] - 2024-01-28

Initial release.

Changes to www/index.md.

1
2
3
4
5
6
7
8
9
10
11
12











13
14
15
16
17
18
19
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30












+
+
+
+
+
+
+
+
+
+
+







# fsblobstore

An abstraction over a file system-backed blob storage.

The `FsBlobStore` object can be used to insert blobs to a datastore, which
will return a key (the content's hash).  The key can then be used to request a
reader for the blob in the storage.

_fsblobstore_ is meant to be used as a backend storage; a separate database
(typically wrapped around the fsblobstore) should be used to index the blobs
(using their keys).


## Feature labels in documentation

The crate's documentation uses automatically generated feature labels, which
currently requires nightly featuers.  To build the documentation locally use:

```
$ RUSTFLAGS="--cfg docsrs" RUSTDOCFLAGS="--cfg docsrs" \
cargo +nightly doc --all-features
```


## Change log

The details of changes can always be found in the timeline, but for a
high-level view of changes between released versions there's a manually
maintained [Change Log](./changelog.md).