Failure to resolve conflicting hidden files

[izosozi]

Abstract

Multiple attempts to resolve a batch of conflicts in hidden files fail to resolve them, including fetching from the remote DB.

Reproducing procedure

1. Enabled Hidden File Sync on both devices A and B
2. Made a change to a hidden file on device A
3. Waited for conflict to appear on device B
4. Pressed LiveSync Settings > Hatch > Resolve All (Resolve all conflicts with the newer version)

Expected:

The conflicts should be resolved in the background. No conflicts should appear in the "Pick a file to resolve conflict" menu

Actual:

The same conflicts as before appear in the conflict picker.

5. Enabled LiveSync Settings > Patches > Compatibility (Conflict Behaviour) > Apply Latest Change if Conflicting
6. Reloaded Obsidian

Expected:

The conflicts should be resolved in the background. No conflicts should appear in the "Pick a file to resolve conflict" menu

Actual:

The same conflicts as before appear in the conflict picker.

7. Disabled LiveSync Settings > Patches > Compatibility (Conflict Behaviour) > Resolve All
8. Reloaded Obsidian
9. Enabled LiveSync Settings > Sync Settings > Conflict resolution > (BETA) Always overwrite with a newer file
10. Reloaded Obsidian

Expected:

The conflicts should be resolved in the background. No conflicts should appear in the "Pick a file to resolve conflict" menu

Actual:

The same conflicts as before appear in the conflict picker.

11. Pressed LiveSync Settings > Maintenance > Rebuilding Operations (Local) > Fetch from remote w/o restarting
12. Selected "Fetch" when prompted about hidden files
13. Waited for process to complete and reloaded Obsidian

Expected:

There should be no conflicts since the local version is freshly fetched from the remote db. No conflicts should appear in the "Pick a file to resolve conflict" menu

Actual:

The same conflicts as before appear in the conflict picker.

14. Disabled LiveSync Settings > Sync Settings > Conflict resolution > (BETA) Always overwrite with a newer file
15. Enabled LiveSync Settings > Patches > Compatibility (Conflict Behaviour) > Apply Latest Change if Conflicting
16. Ran command "Resolve all conflicted files" in Command Palette

Expected:

All conflicts should be resolved in the background

Actual:

The same conflicts as before appear in the conflict picker, but they do get resolved automatically when clicked on one by one

UPDATE: This issue occurred again, and this final solution failed to resolve any of the conflicts.

Report materials

Report from the LiveSync

Report from hatch

# ---- Obsidian info ----
# Navigator: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.7.7 Chrome/128.0.6613.186 Electron/32.2.5 Safari/537.36
# FileSystem: insensitive
# ---- remote config ----
cluster:
  n: "1"
cors:
  credentials: "true"
  origins: app://obsidian.md,capacitor://localhost,http://localhost
chttpd:
  bind_address: 0.0.0.0
  enable_cors: "true"
  max_http_request_size: "4294967296"
  port: "5984"
  require_valid_user: "true"
admins: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
vendor:
  name: The Apache Software Foundation
feature_flags:
  partitioned||*: "true"
chttpd_auth:
  hash_algorithms: sha256, sha
  require_valid_user: "true"
  secret: 6449e3015df52bb23673b5763bbbc71c
indexers:
  couch_mrview: "true"
prometheus:
  additional_port: "false"
  bind_address: 127.0.0.1
  port: "17986"
httpd:
  WWW-Authenticate: Basic realm="couchdb"
  bind_address: 127.0.0.1
  enable_cors: "true"
  port: "5986"
smoosh:
  state_dir: ./data
couch_httpd_auth:
  authentication_db: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
  secret: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
  authentication_redirect: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
couchdb_engines:
  couch: couch_bt_engine
couchdb:
  database_dir: ./data
  max_document_size: "50000000"
  uuid: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
  view_index_dir: ./data

# ---- Plug-in config ---
version:0.24.3
remoteType: ""
useCustomRequestHandler: false
couchDB_URI: self-hosted(HTTP)
couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
liveSync: false
syncOnSave: false
syncOnStart: false
savingDelay: 200
lessInformationInLog: false
gcDelay: 0
versionUpFlash: ""
minimumChunkSize: 20
longLineThreshold: 250
showVerboseLog: true
suspendFileWatching: false
trashInsteadDelete: true
periodicReplication: false
periodicReplicationInterval: 10
syncOnFileOpen: false
encrypt: false
passphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
usePathObfuscation: false
doNotDeleteFolder: false
resolveConflictsByNewerFile: false
batchSave: false
batchSaveMinimumDelay: 3
batchSaveMaximumDelay: 30
deviceAndVaultName: ""
usePluginSettings: false
showOwnPlugins: false
showStatusOnEditor: false
showStatusOnStatusbar: false
showOnlyIconsOnEditor: false
usePluginSync: false
autoSweepPlugins: false
autoSweepPluginsPeriodic: false
notifyPluginOrSettingUpdated: false
checkIntegrityOnSave: false
batch_size: 25
batches_limit: 25
useHistory: true
disableRequestURI: true
skipOlderFilesOnSync: true
checkConflictOnlyOnOpen: false
showMergeDialogOnlyOnActive: false
syncInternalFiles: false
syncInternalFilesBeforeReplication: false
syncInternalFilesIgnorePatterns: node_modules\/, \.git, plugins\/obsidian-livesync\/, \.stversions, \.stfolder, \.trash, plugins\/remember-cursor-position\/cursor-positions\.json$, \.DS_Store, plugins\/obsidian-raindrop-highlights\/data\.json$, hotkeys\.json, workspace\.json, workspace-mobile\.json, plugins\/smart-connections\/data\.json$
syncInternalFilesInterval: 60
additionalSuffixOfDatabaseName: f207f9baa45a8afc
ignoreVersionCheck: false
lastReadUpdates: 0
deleteMetadataOfDeletedFiles: false
syncIgnoreRegEx: \.DS_Store|[]|^raindrop\/
syncOnlyRegEx: ""
customChunkSize: 0
readChunksOnline: true
watchInternalFileChanges: true
automaticallyDeleteMetadataOfDeletedFiles: 1
disableMarkdownAutoMerge: false
writeDocumentsIfConflicted: false
useDynamicIterationCount: false
syncAfterMerge: false
configPassphraseStore: ""
encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
permitEmptyPassphrase: false
useIndexedDBAdapter: true
useTimeouts: false
writeLogToTheFile: true
doNotPaceReplication: false
hashCacheMaxCount: 300
hashCacheMaxAmount: 50
concurrencyOfReadChunksOnline: 40
minimumIntervalOfReadChunksOnline: 50
hashAlg: xxhash64
suspendParseReplicationResult: false
doNotSuspendOnFetching: false
useIgnoreFiles: false
ignoreFiles: .gitignore
syncOnEditorSave: false
pluginSyncExtendedSetting: {}
syncMaxSizeInMB: 50
settingSyncFile: livesync-config.md
writeCredentialsForSettingSync: false
notifyAllSettingSyncFile: false
isConfigured: true
settingVersion: 10
enableCompression: false
accessKey: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
bucket: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷(0 letters)
endpoint: Not configured or AWS
region: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷(0 letters)
secretKey: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
useEden: false
maxChunksInEden: 10
maxTotalLengthInEden: 1024
maxAgeInEden: 10
disableCheckingConfigMismatch: false
displayLanguage: ""
enableChunkSplitterV2: false
disableWorkerForGeneratingChunks: false
processSmallFilesInUIThread: false
notifyThresholdOfRemoteStorageSize: 5000
usePluginSyncV2: true
usePluginEtc: false
handleFilenameCaseSensitive: false
doNotUseFixedRevisionForChunks: false
showLongerLogInsideEditor: false
sendChunksBulk: false
sendChunksBulkMaxSize: 1
useSegmenter: false
useAdvancedMode: true
usePowerUserMode: true
useEdgeCaseMode: true
enableDebugTools: false
configPassphrase: ""
preset: ""
syncMode: ONEVENTS
dummy: 0

Plug-in log

Plug-in log

output.txt

Screenshots

Hidden files which failed to be resolved:

Other information, insights and intuition.

This issue may be a result of my own inaccurate expectations of what these settings do. If so, some more relabeling/description/documentation would be appreciated.

It is also possible that these conflicts were themselves being synchronized into the remote DB, which would explain why there were still conflicts after fetching.

Thank you for all that you do!

[izosozi]

This issue occurred again a few minutes later, and the exact same list of files had conflicts. This seems odd, since the only one that would have changed was the recent-files-obsidian/data.json file.

1. Opened each file and confirmed that there weren't any changing data in the file contents, other than the recent-files-obsidian/data.json.
2. Enabled Sync Settings > Conflict resolution > (BETA) Always overwrite with a newer file
3. Ran command "Resolve all conflicted files" in Command Palette
4. Clicked on files one at a time

The files which had no changes did get resolved immediately on clicking them, but the recent-files-obsidian/data.json file did not get resolved automatically. I assumed this would happen based on what I saw in the final step of "Reproducing procedure" in the original issue.

I then tried it with enabling LiveSync Settings > Patches > Compatibility (Conflict Behaviour) > Apply Latest Change if Conflicting, with both options on, and with both options off.

In all cases, the result was the same: the unchanged files disappeared when clicked, the conflicting file did not get resolved and presented a popup when clicked.

This leads me to believe that the files that are 'disappearing' when clicked are not being resolved automatically at all. They might not even be real conflicts, in which case clicking on them simply makes the ghost conflict disappear.

It also shows that both of the settings to enable automatic conflict handling have no effect on this issue.

[izosozi]

I also failed to resolve the issue with deletion of the local database before a fetch.

1. LiveSync Settings > Maintenance > Reset > Delete
2. LiveSync Settings > Maintenance > Rebuilding Operations (Local) > Fetch

Expected:
No conflicts should exist in any files, since the local database and local filesystems should exactly match the state of the remote.

Actual:
The hidden file conflicts remain the same as seen in the "Pick a file to resolve conflict" menu.

[izosozi]

I tried using the options in "LiveSync Settings > Sync Settings > Hidden Files > Enable Hidden files sync" to resolve the issue.

"Merge" and "Fetch" did not remove or change any of the conflicted hidden files as seen in the "Pick a file to resolve conflict" menu.

"Overwrite" resulted in an entirely new set of conflicting hidden files being shown (edit: added to the list of conflicting files), which is weird.

Now this new set of conflicting hidden files is reflected across all my devices (edit: in addition to the previous conflicts). None of these files have had any changes to their contents (just like the earlier conflicts), and they disappear immediately when clicking them one by one in the "Pick a file to resolve conflict" menu.

---

EDIT:

I thought the Overwrite action replaced the conflicts with new ones, but the old conflicts are still there, just at the bottom of the (very long) list of conflicts.

[izosozi]

I have resolved this by:

1. From a device with the up to date vault, run LiveSync > Maintenance > Total Overhaul > Rebuild
2. On all other devices LiveSync > Maintenance > Rebuilding Operations (Local) > Fetch

After doing this, no conflicts appear in the menu. This indicates to me that the conflicts are within the remote db state itself, and it might be the case that these can't be resolved by applying changes on local storage or the local db alone.