Direct upload issue (batch and FileManager.Import fail with Concurrent update)

[s-guillaume]

As reported in Jira [(https://jira.nuxeo.com/browse/NXP-22826)], I identified an issue on Nuxeo client. With a LTS2019HF72 and PostgreSQL 13. Using Nuxeo Drive with direct upload, selecting a local folder with 9 files for example and 4 of them start with the same prefix (Strategic Partnerships Follow Up 1.xlsx, Strategic Partnerships Follow Up 2.xlsx, Strategic Partnerships Follow Up 3.xlsx, Strategic Partnerships Follow Up.rar). You can test with any files having the same kind of prefix on the first 24 characters.

You can follow the attached PDF in Jira ticket to reproduce. In some cases, I can have 2, 3 or 5 documents created out of the 9 expected.

The problem is related to the batch upload that is unique per uploaded file. In Nuxeo Drive, a call to get a batch ID is done for each file, then a FileManager.Import is called with this batch id. In case of multiple files to upload, Nuxeo Drive will call, mostly in parallel, 9 times to get a batch Id and 9 times FileManager.Import. Some of this import will fail because of unicity on name (as shown in the Jira ticket logs)

In WebUI, you can upload the same sample files without any problem as WebUI will use the same batch upload for the 9 files and will call FileManager.Import only once

Could you let me know if something can be done to fix this issue. The problem is also that Nuxeo Drive History tab is showing that the direct upload has worked perfectly (10/10 with 1 folder and 9 documents) but that's not the case. More, when you try to click on "Export to CSV", it fails (check the Nuxeo Drive logs also in Jira)

[manonlumeau]

Hi Sébastien,

Thanks for your message.
Unfortunately as LTS 2019 is no longer supported, we tested against LTS 2021 and LTS 2023 and Drive version 5.3.2: we weren't able to reproduce with the combination of the file names mentioned in the ticket.

I hope it will help!

Thanks,

Manon Lumeau
PO Nuxeo Drive

[s-guillaume]

Hi Manon, sorry for my late feedback but I have not the same result as you with LTS2021. I have Nuxeo LTS2021HF45/WebUI + PostgreSQL 13 + Nuxeo Drive 5.3.1

On Nuxeo Drive client, click on Upload content
In the new popup, click on "New Transfer"
In the new popup, click on Add a folder and then select a folder with the provided files
Click on OK to launch the transfer

You will have a feedback with all files uploaded correctly but in fact, it's not the case, some documents are missing (1 or 2). An error is also shown as toggle mentionning an upload error on a file (which is not the right one ...)

In Nuxeo Drive client, on history tab, you cannot click on CSV export (error)

I updated the Jira https://jira.nuxeo.com/browse/NXP-22826