Immutable noLock VFS
source link: https://bugzilla.mozilla.org/show_bug.cgi?id=1801495
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Details
VFS on top of the -none ones to implement immutable behavior. More details to follow.
I presume this is low enough level in whatever use case it's for that mozStorage won't be the right place for canonical docs, but it would be great to make sure any docs here can chain upwards to wherever describes:
1: I'm assuming that the high level intent here is to use SQLite databases like pre-computed look-up tables which themselves are not mutated by Firefox at runtime, so they are individually immutable, but that there is a pipeline that produces new immutable successor databases.
Hi, sorry I didn't find more time to explain this yesterday. The use-case is opening third party (other browsers) files to import data, we already introduced the noLock VFS for that (unix-none and win32-none) some time ago to be able to open a file when it's in use, we made those only supported in read-only mode, because if you use a noLock VFS to write, you are 99% of the times corrupting your data.
The remaining problem, that we just discovered, is this: opening a WAL database in readonly mode doesn't work, unless the -wal and -shm exist. This is actually documented at https://sqlite.org/wal.html:
It is not possible to open read-only WAL databases. The opening process must have write privileges for "-shm" wal-index shared memory file associated with the database, if that file exists, or else write access on the directory containing the database file if the "-shm" file does not exist. Beginning with version 3.22.0 (2018-01-22), a read-only WAL-mode database file can be opened if the -shm and -wal files already exists or those files can be created or the database is immutable.
These files are sometimes in places where we can't write to, other times we must copy the db to a different position, and it's much easier to just copy the main db. Ideally if it's in a position where we can create files, it should automatically create them and allow to open (the documentation says "or those files can be created") though it should really not be necessary to create a journal for a read-only access. Plus we really don't want to create files in other apps folders.
So the ideal solution is immutable, but it can only be passed with fileuris (as immutable=1 option), there's no corresponding SQLITE_OPEN_* flag. We may ask for it, but as you know we're a bit blocked on Sqlite upgrades, and it's not a given.
Since we basically only need this in the noLock case, I thought I could just write a shim VFS on top of the -none ones that simulates a read-only device. This solves the problem indeed, as expected.
This is only supposed to be used in read only mode, indeed there's an assert in place for that, and the only way to opt-in to this is to use the ignoreLockingMode option of the connection (that enforces read-only mode).
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK