End to End Encryption breaks CouchDB <-> Storage Peers

[KalimaSan]

I successfully created a bridge between a couchdb and storage type peer of a very simple Obsidian vault. However, when I tried to then use end to end encryption, it resulted in errors. All my other vaults have end to end encryption and I wasn't able to get a bridge setup for those at all. This is why I tried a simple vault without encryption. I believe encryption is somehow not supported? If not, would it be added at some point?

My config.json and error log are below to review. Thanks for this project, it's the last piece that makes Obsidian able to be self-hosted with a headless sync to a file system that is backed up via traditional techniques.

config.json:

{
    "peers": [
        {
            "type": "couchdb",
            "name": "livesync-bridge-couchdb",
            "database": "mysync",
            "username": "deprecated",
            "password": "deprecated",
            "url": "http://192.168.1.1:1234",
            "passphrase": "deprecated",
            "customChunkSize": 100,
            "minimumChunkSize": 20,
            "obfuscatePassphrase": "",
            "baseDir": ""
        },
        {
            "type": "storage",
            "name": "livesync-bridge-storage",
            "baseDir": "/livesync-bridge/mysync",
            "scanOfflineChanges": true
        }
    ]
}

error.log

2/23/2024, 10:50:37 PM	1	WATCH: SKIP obsydian_livesync_version: Not a document
2/23/2024, 10:50:37 PM	1	Couldn't decode! You should wrong the passphrases
Error: Encrypted data corrupted!
    at decrypt (file:///livesync-bridge/repo/livesync-bridge/lib/src/e2ee_v2.ts:232:19)
    at DirectFileManipulator.decryptDocumentPath (file:///livesync-bridge/repo/livesync-bridge/lib/src/DirectFileManipulator.ts:156:26)
    at DirectFileManipulator.processJSONL (file:///livesync-bridge/repo/livesync-bridge/lib/src/DirectFileManipulator.ts:463:45)
    at DirectFileManipulator.beginWatch (file:///livesync-bridge/repo/livesync-bridge/lib/src/DirectFileManipulator.ts:510:36)
    at eventLoopTick (ext:core/01_core.js:153:7)
2/23/2024, 10:50:37 PM	1	Error:Encrypted data corrupted!
2/23/2024, 10:50:37 PM	1	WATCH: SOMETHING WENT WRONG ON EACH PROCESS
Error: Encrypted data corrupted!
    at decrypt (file:///livesync-bridge/repo/livesync-bridge/lib/src/e2ee_v2.ts:232:19)
    at DirectFileManipulator.decryptDocumentPath (file:///livesync-bridge/repo/livesync-bridge/lib/src/DirectFileManipulator.ts:156:26)
    at DirectFileManipulator.processJSONL (file:///livesync-bridge/repo/livesync-bridge/lib/src/DirectFileManipulator.ts:463:45)
    at DirectFileManipulator.beginWatch (file:///livesync-bridge/repo/livesync-bridge/lib/src/DirectFileManipulator.ts:510:36)
    at eventLoopTick (ext:core/01_core.js:153:7)
2/23/2024, 10:50:37 PM	1	Error:Encrypted data corrupted!
2/23/2024, 10:50:37 PM	1	{"seq":"4-g1AAAACLeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5kTkXKMBumpKYampqgK4Yh_Y8FiDJ0ACk_kNNYYSYYmRknpJkiK4nCwBQhip-","id":"Test1.md","changes":[{"rev":"1-00b247a18d6e284b85b416b20dca256e"}],"doc":{"_id":"Test1.md","_rev":"1-00b247a18d6e284b85b416b20dca256e","children":["h:+1sv5w1hjhek03"],"path":"Test1.md","ctime":1708714883053,"mtime":1708714883053,"size":20,"type":"plain"}}
2/23/2024, 10:50:37 PM	1	WATCH: SKIP _design/replicate: Not a document
2/23/2024, 10:51:16 PM	1	Couldn't decode! You should wrong the passphrases
Error: Encrypted data corrupted!

[vrtmrz]

Thank you for the reporting!
In some cases, decrypting the path of the document had been failed.
Now in today's version, it has been fixed.
Would you mind if I ask you to check the behaviour, please?

[KalimaSan]

I pulled the latest revision and retested. My vault is encrypted still. When I configure with a blank passphrase the log looks as if it successfully processed it. Of course the contents were the encrypted contents (unusable). When I configured it with the correct passphrase I got errors. Both the blank passphrase and configured passphrase logs are attached. So appreciate the help, this makes Obsidian so much more useful!

output.log

[ImDevinC]

I also cannot get decrypting to work when I'm using E2EE from obsidian-livesync. I know the credentials are correct, and if I only use livesync-bridge to sync between two PC devices, encryption works correctly. However, once I try to setup the obsidian-livesync plug-in inside of Obsidian, I get the decryption error.

Here's my current setup:

1. I have couchdb running in a kubernetes cluster on my homelab, but accessible to the internet through a domain name
2. On my desktop and laptop, I have setup livesync-bridge, running in Docker, to sync my files using an encryption password, but not path obfuscation. This works correctly.
3. On my same desktop, I created a new vault in a new location and setup the obsidian-livesync to sync with couchdb and I get the logs below.

Note

I only setup the Obsidian app with a new vault on my desktop because it was easy to replicate this issue. I originally noticed it on my Android device and have tested on multiple devices that livesync-bridge -> couchdb -> livesync-obsidian does not decrypt properly

Note

If I instead just do livesync-obsidan -> couchdb -> livesync-obsidian (essentially, removing livesync-bridge) everything also works fine. This only seems to be an issue with livesync-bridge which I'm trying to use because I use vim a lot more than the default Obsidian app.

9/24/2024, 9:02:40 PM->Cache initialized 300 / 250000000000
9/24/2024, 9:02:40 PM->loading plugin
9/24/2024, 9:02:40 PM->Self-hosted LiveSync v0.23.23 0.23.23
9/24/2024, 9:02:40 PM->xxhash for plugin initialised
9/24/2024, 9:02:40 PM->Self-hosted LiveSync has undergone a major upgrade. Please open the setting dialog, and check the information pane.
9/24/2024, 9:02:40 PM->Waiting for ready...
9/24/2024, 9:02:40 PM->Opening Database...
9/24/2024, 9:02:40 PM->Database is now ready.
9/24/2024, 9:02:42 PM->Hidden files and plugin synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.
9/24/2024, 9:02:42 PM->redflag3.md or flag_fetch.md has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.
9/24/2024, 9:02:46 PM->Hidden files and plugin synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.
9/24/2024, 9:02:46 PM->Suspending reflection: Database and storage changes will not be reflected in each other until completely finished the fetching.
9/24/2024, 9:02:46 PM->Database closed for reset Database.
9/24/2024, 9:02:46 PM->Opening Database...
9/24/2024, 9:02:46 PM->Database is now ready.
9/24/2024, 9:02:46 PM->Local Database Reset
9/24/2024, 9:02:47 PM->Database closed (by close)
9/24/2024, 9:02:47 PM->Waiting for ready...
9/24/2024, 9:02:47 PM->Opening Database...
9/24/2024, 9:02:47 PM->Database is now ready.
9/24/2024, 9:02:47 PM->Mark this device as 'resolved'.
9/24/2024, 9:02:48 PM->OneShot Sync begin... (pullOnly)
9/24/2024, 9:02:48 PM->Looking for the point last synchronized point.
9/24/2024, 9:02:48 PM->Replication activated
9/24/2024, 9:02:48 PM->Decryption failed.
9/24/2024, 9:02:48 PM->Replication error
9/24/2024, 9:02:48 PM->Replication stopped.
9/24/2024, 9:02:48 PM->Replication error
9/24/2024, 9:02:48 PM->Error:Encrypted data corrupted!
9/24/2024, 9:02:49 PM->OneShot Sync begin... (pullOnly)
9/24/2024, 9:02:49 PM->Looking for the point last synchronized point.
9/24/2024, 9:02:49 PM->Replication activated
9/24/2024, 9:02:49 PM->Decryption failed.
9/24/2024, 9:02:49 PM->Replication error
9/24/2024, 9:02:49 PM->Replication stopped.
9/24/2024, 9:02:49 PM->Replication error
9/24/2024, 9:02:49 PM->Error:Encrypted data corrupted!
9/24/2024, 9:02:49 PM->Database and storage reflection has been resumed!
9/24/2024, 9:02:49 PM->Initializing
9/24/2024, 9:02:49 PM->Initialize and checking database files
9/24/2024, 9:02:49 PM->Checking deleted files
9/24/2024, 9:02:49 PM->Synchronising...
9/24/2024, 9:02:49 PM->UPDATE DATABASE
9/24/2024, 9:02:49 PM->UPDATE STORAGE: Nothing to do
9/24/2024, 9:02:49 PM->SYNC DATABASE AND STORAGE: Nothing to do
9/24/2024, 9:02:49 PM->UPDATE DATABASE All done: DONE:1, FAILED:0
9/24/2024, 9:02:49 PM->Initialized, NOW TRACKING!
9/24/2024, 9:02:49 PM->Initialize done!
9/24/2024, 9:02:50 PM->Hidden files and plugin synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.