Ergonomics around use in a stateful application (like Tauri)

[Gawdl3y]

Hi there! I'm working on adding a database layer to my Tauri app, Resolute, and this crate showed up at basically the perfect time. I'm having some trouble with making use of it, however, mostly due to the lifetime parameter on the Database struct. Ideally, I want to have a database wrapper that entirely abstracts away the use of native_db, like so:

pub struct ResoluteDatabase {
	db: Database,
}

impl ResoluteDatabase {
	pub fn open(db_path: impl AsRef<Path>) -> Result<Self> {
		let builder = DatabaseBuilder::new();
		builder.define::<ResoluteMod>()?;

		let db = builder.create(&db_path)?;

		info!("Database initialized at {}", db_path.as_ref().display());
		Ok(Self { db })
	}

	pub fn get_mods(&self) -> Result<Vec<ResoluteMod>> {
		let read = self.db.r_transaction()?;
		let mods = read.scan().primary()?.all().collect();
		Ok(mods)
	}

	pub fn get_mod(&self, id: String) -> Result<Option<ResoluteMod>> {
		let read = self.db.r_transaction()?;
		let rmod = read.get().primary(id)?;
		Ok(rmod)
	}

	pub fn add_mod(&self, rmod: ResoluteMod) -> Result<()> {
		let rw = self.db.rw_transaction()?;
		rw.insert(rmod)?;
		rw.commit()?;
		Ok(())
	}

	// ...
}

And the usage of this would look something like:

let db = resolute::db::ResoluteDatabase::open("resolute.db");
db.add_mod(rmod);
let mods = db.get_mods();
// etc

But since native_db's Database struct has a lifetime parameter that starts at DatabaseBuilder.init, I'm unable to do that - I, at the very least, have to have the DatabaseBuilder passed in to the ResoluteDatabase wrapper, more like this:

pub struct ResoluteDatabase<'a> {
	db: Database<'a>,
}

impl<'a> ResoluteDatabase<'a> {
	pub fn open(builder: &'a mut DatabaseBuilder, db_path: impl AsRef<Path>) -> Result<Self> {
		builder.define::<ResoluteMod>()?;
		let db = builder.create(&db_path)?;
		info!("Database initialized at {}", db_path.as_ref().display());
		Ok(Self { db })
	}

	pub fn get_mods(&self) -> Result<Vec<ResoluteMod>> {
		let read = self.db.r_transaction()?;
		let mods = read.scan().primary()?.all().collect();
		Ok(mods)
	}

	pub fn get_mod(&self, id: String) -> Result<Option<ResoluteMod>> {
		let read = self.db.r_transaction()?;
		let rmod = read.get().primary(id)?;
		Ok(rmod)
	}

	pub fn add_mod(&self, rmod: ResoluteMod) -> Result<()> {
		let rw = self.db.rw_transaction()?;
		rw.insert(rmod)?;
		rw.commit()?;
		Ok(())
	}

	// ...
}

As a consequence of that lifetime parameter being required upwards and the builder now needing to be constructed outside the wrapper, the Tauri app not only has to make the builder, but also has to keep it at a static lifetime in order to manage the database wrapper as state since it (understandably) takes ownership of any given state (and the builder needs to not be dropped for that to happen):

static mut DB_BUILDER: Lazy<DatabaseBuilder> = Lazy::new(DatabaseBuilder::new);

fn main() -> anyhow::Result<()> {
	// Set up a shared HTTP client
	let http_client = reqwest::Client::builder()
		.connect_timeout(Duration::from_secs(10))
		.use_rustls_tls()
		.build()
		.context("Unable to build HTTP client")?;

	// Set up and run the Tauri app
	tauri::Builder::default()
		.invoke_handler(tauri::generate_handler![
			// ...
		])
		.manage(http_client)
		.manage(ResoluteState::default())
		.setup(|app| {
			// ...

			// Open the database
			let resolver = app.path_resolver();
			let db_path = resolver
				.app_data_dir()
				.expect("unable to get data dir")
				.join("resolute.db");
			let db = unsafe {
				resolute::db::ResoluteDatabase::open(
					&mut DB_BUILDER,
					db_path.to_str().expect("unable to convert db path to str"),
				)
				.expect("unable to open database")
			};

			// Set up the shared mod manager
			let http_client = app.state::<reqwest::Client>();
			let manager = ModManager::new(db, "", &http_client);
			app.manage(Mutex::new(manager));

			Ok(())
		})
		.run(tauri::generate_context!())
		.with_context(|| "Unable to initialize Tauri application")?;

	Ok(())
}

I'm still a relative beginner to writing Rust applications, and I haven't really done anything with lifetimes in-depth, so I'm hoping I'm just missing something or doing something wrong, because this is fairly cumbersome as-is. Not only am I having to use unsafe code (mutable statics) just to initialize the database, but if I ever want to close/reinitialize it, I'd have to reuse the same builder (which may or may not work?).

Basically, I'm looking for some advice here, since the way I'm using it right now is very kludgey/unergonomic.

[vincent-herlemont]

@Gawdl3y It's interesting, I've never used the database in the context of Tauri, so
I wouldn't have a ready-made answer. Your architecture seems correct, and it's true that DatabaseBuilder and Database are linked.

For a brief explanation: Redb is used in the backend and its definition, redb::TableDefinition, has a name: &'a str field which is a reference. This forces us to store it in an external variable outside the Database struct.

However 👉 in a multi-threading context, we can use std::thread::scope which allows us to use non-'static. Maybe that can help you, but I don't know Tauri very well so I can not give a good answer.

Unlike non-scoped threads, scoped threads can borrow non-'static data, as the scope guarantees all threads will be joined at the end of the scope.

Example of use with native_db: tests/simple_multithreads.rs#L27-L56

With these explanations, I'm not sure if I've been of help, but I would be curious to know if there's a more elegant solution with Tauri if you find one.

I'll try to look into it on my side when I have some time.

[vincent-herlemont]

It seems there is issue about thread::scope and Tauri tauri-apps/tauri#6396 (comment).

[vincent-herlemont]

Or maybe you can define DatabaseBuilder directly inside the setup handler.

[Gawdl3y]

Defining the DatabaseBuilder inside setup doesn't work out since the builder would be dropped upon going out of scope, but the Database is still hanging around.

	// Open the database
	let resolver = app.path_resolver();
	let db_path = resolver
		.app_data_dir()
		.expect("unable to get data dir")
		.join("resolute.db");
	let mut builder = DatabaseBuilder::new();
	let db = resolute::db::ResoluteDatabase::open(
		&mut builder,
		db_path.to_str().expect("unable to convert db path to str"),
	)
	.expect("Unable to open database");

Compiler diagnostic:

error[E0597]: `builder` does not live long enough
   --> crates\tauri-app\src\main.rs:121:5
    |
119 |               let mut builder = DatabaseBuilder::new();
    |                   ----------- binding `builder` declared here
120 |               let db = resolute::db::ResoluteDatabase::open(
    |  ______________________-
121 | |                 &mut builder,
    | |                 ^^^^^^^^^^^^ borrowed value does not live long enough
122 | |                 db_path.to_str().expect("unable to convert db path to str"),
123 | |             )
    | |_____________- argument requires that `builder` is borrowed for `'static`
...
132 |           })
    |           - `builder` dropped here while still borrowed

[Gawdl3y]

Is reusing a DatabaseBuilder valid/safe? If so, I can potentially just move the static DB_BUILDER declaration to the file with ResoluteDatabase in it and use it directly in open(path) rather than passing it to it from the main Tauri app file.

[vincent-herlemont]

@Gawdl3y Maybe try another way like it:

pub struct ResoluteDatabase<'a> {
    db: Database<'a>,
}

impl<'a> ResoluteDatabase<'a> {
    pub fn open(db: Database<'a>) -> Result<ResoluteDatabase> {
        Ok(Self { db })
    }

    pub fn get_mods(&self) -> Result<Vec<ResoluteMod>> {
        let read = self.db.r_transaction()?;
        let mods = read.scan().primary()?.all().collect();
        Ok(mods)
    }

    // ...
}

And in the setup method:

.setup(|app| {
   ...
    let mut builder = DatabaseBuilder::new();
    builder.define::<ResoluteMod>()?;
    let db = builder.create(db_path.as_path())?;
    let db = ResoluteDatabase::open(db)?;
...

[Gawdl3y]

This doesn't work for the same reason as before - builder gets dropped at the end of the setup closure.

error[E0597]: `builder` does not live long enough
   --> crates\tauri-app\src\main.rs:125:36
    |
123 |             let mut builder = DatabaseBuilder::new();
    |                 ----------- binding `builder` declared here
124 |             builder.define::<ResoluteMod>()?;
125 |             let db = ResoluteDatabase::open(builder.create(db_path)?);
    |                      -----------------------^^^^^^^------------------
    |                      |                      |
    |                      |                      borrowed value does not live long enough
    |                      argument requires that `builder` is borrowed for `'static`
...
138 |         })
    |         - `builder` dropped here while still borrowed

It seems like it's essentially mandatory to have builder live for 'static in any kind of stateful use of Database, which is less than ideal. In my case, it's currently not a huge issue since I can just use a Lazy and declare it as a static, since I don't have any plans for closing/reopening the DB at runtime, but it'd be a much larger barrier of entry for anyone that does intend on doing those things.

It looks like the creator of redb intends on removing as many lifetimes as possible from the public API for 2.0, based on a checklist item in cberner/redb#677 (and some of it has happened in cberner/redb#732). Hopefully that comes to fruition, as the lifetimes are making it somewhat tough to work with.

[vincent-herlemont]

As I've explained here, the 'static is mandatory because Tauri does not use thread scoped. In reality, in a different context, 'static is not needed.

However, it's true that in the case of Tauri, it's necessary. I've created a small example project to explore possibilities. Feel free to get inspiration (I used Lazy in the builder like you did) from it and critique it 👉 native_db_tauri_vanilla :).

An important thing to note is that I saw you used unsafe in your first example, and it would be advisable to avoid using it.

NOTE: Regarding the removal of the reference in TableDefinition.TableDefinition implements the Copy trait, which implies copying the reference and not cloning the value, I suppose for performance reasons it might be necessary to keep a reference in TableDefinition.

[Gawdl3y]

@vincent-herlemont Just want to say thanks for your advice and assistance, and thank you for making such a nice library! I think that once redb removes those lifetimes where they can, my only real pain point will probably be resolved. 🤞

[vincent-herlemont]

@Gawdl3y Some improvements here: tauri-apps/tauri#8489 (reply in thread).

Feel free to use a new version of the example native_db_tauri_vanilla.

[Qbicz]

Excellent research @Gawdl3y, your examples helped me get around this problem in my project. Thanks!