What do you want to see in a better drag & drop API for Windows?

[Ram-G]

Discussion: What do you want to see in a better drag & drop API for Windows?

In a similar vein to other discussion threads (Clipboard in #63), we at Windows are trying to understand how we can improve the drag & drop APIs in Windows as part of Project Reunion. As such, we would love your feedback on drag & drop in Windows -

• What have you liked about it, what have you disliked?
• If you are implementing clipboard functionality in your app, why are you not implementing drag & drop? This especially becomes a relevant discussion as we move to a world of devices which are "touch-first" as opposed to "mouse & keyboard-first".

We are open to your ideas, constructive criticisms, comments, questions, concerns in this thread. We really want to hear them to make it easy for you to build more powerful apps which delight your users!

Examples of bugs / feature requests relating to drag & drop you have raised on our partner XAML team's github which we are tracking internally:

https://github.com/microsoft/microsoft-ui-xaml/issues/2364
microsoft/microsoft-ui-xaml#2421

---

High-level overview of the drag & drop API surface on Windows

There are a few ways to implement drag & drop in a Windows app today:

1. If your app uses XAML/WinUI
   Detailed blog post: https://docs.microsoft.com/en-us/archive/msdn-magazine/2015/august/windows-10-modern-drag-and-drop-for-windows-universal-applications
   Samples: https://github.com/microsoft/Windows-universal-samples/tree/master/Samples/XamlDragAndDrop

2. If your app is a WPF: https://docs.microsoft.com/en-us/dotnet/framework/wpf/advanced/drag-and-drop-overview

3. (For all UWP apps) Via the UWP API: https://docs.microsoft.com/en-us/uwp/api/windows.applicationmodel.datatransfer.dragdrop.core?view=winrt-19041

4. (For all classic Win32 apps) Via the OLE drag & drop API: https://docs.microsoft.com/en-us/cpp/mfc/drag-and-drop-ole?view=vs-2019

We understand that this is obviously really confusing. The most modern APIs are 1) and 3), the UWP ones and we expect these to be the foundation for Project Reunion drag & drop. But we want to do better - we want to know what your pain points have been at using any of them, and what you'd like to make things better for your users.

---

Please respond to this thread to let us know your ideas, and we'll break them off into separate issues/discussions as necessary.

[mdtauk]

Irrespective of platform framework, these are some of the things I would expect from a Drag and Drop experience

• Custom visual for the thing whilst it is being dragged;
• Changing the visual to reflect when a drag operation will be successful or will fail;
• Enable whole window as drop target, as well as a UI element acting as a drop target;
• Metadata attached to the dragged content, so apps can provide multiple experiences depending on what content it is receiving; (images, text or formatted content etc)
• Enable drop of multiple items; (batch processing?)
• Apps can declare what content it will accept for receiving;

[benstevens48]

Obviously you already have this on your radar, but for me the most important thing is to make files writable by default, especially if dragged from File Explorer.

[mfe-]

Obviously you already have this on your radar, but for me the most important thing is to make files writable by default, especially if dragged from File Explorer.

I'll provide some additional historic links (you could simply search for "uwp drag and drop readonly":

• https://social.msdn.microsoft.com/Forums/sqlserver/en-US/ffc881fe-e8ad-432a-b2e0-9a74f27c32d4/uwpwindows-10-dragampdrop-problem?forum=wpdevelop
• https://wpdev.uservoice.com/forums/110705-universal-windows-platform/suggestions/10911696-universal-apps-drag-drop-remove-read-only-attr

Off topic: It feels frustrating to renew these kind of old feature requests. We had uservoice, we had another platform (which was introduced in a build 20xx?) where we could view the status of such feature requests. All got abandoned. Now we have to repost them on Feedbackhub? or Project Reunion. Still I'm hopeful that those kind of issues will get addressed.

[metathinker]

Off topic: It feels frustrating to renew these kind of old feature requests. We had uservoice, we had another platform (which was introduced in a build 20xx?) where we could view the status of such feature requests. All got abandoned. Now we have to repost them on Feedbackhub? or Project Reunion. Still I'm hopeful that those kind of issues will get addressed.

I wish I could say anything better, but here is our best answer for now: We at Microsoft are sorry that we keep changing how we accept and request feedback from you and our other platform developers, and we're sorry that with each change we leave some old ideas and concerns behind even though they may still be live. We're trying to be better about this. (I have personally heard and participated in internal discussions regarding this.) We know you have no good reason to believe that, but even so we ask for your understanding as we figure out the best way we can talk to you.

[yaira2]

Obviously you already have this on your radar, but for me the most important thing is to make files writable by default, especially if dragged from File Explorer.

@benstevens48 you might want to check out microsoft/microsoft-ui-xaml#2421.

[wjk]

My main pain point with the drag/drop API is how hard it is to provide a proper drag image from a Windows Forms app, since the default WinForms IDataObject cannot properly communicate with the internal Windows drop subsystem. This means I will need to rewrite a lot of code to enable use of the SHDoDragDrop() function, which is the only flat-Win32 drag/drop code I know of that supports drag images. The "modern drag and drop" API (Windows.UI.Xaml.DragUI et al) is almost perfect; all I would want from it is to be able to write data (and drag images) to it even if I’m not using WinUI 3 for my GUI stack. Thanks!

[Ram-G]

Thank you everyone for the informative feedback so far!

My main pain point with the drag/drop API is how hard it is to provide a proper drag image from a Windows Forms app, since the default WinForms IDataObject cannot properly communicate with the internal Windows drop subsystem. This means I will need to rewrite a lot of code to enable use of the SHDoDragDrop() function, which is the only flat-Win32 drag/drop code I know of that supports drag images. The "modern drag and drop" API (Windows.UI.Xaml.DragUI et al) is almost perfect; all I would want from it is to be able to write data (and drag images) to it even if I’m not using WinUI 3 for my GUI stack. Thanks!

This is actually one of the exact scenarios we want to be enable in Reunion. Windows Forms uses the older drag & drop mechanism involving IDataObject which is harder to work with. In Reunion, we hope to provide an API for any app with an HWND or CoreWindow/AppWindow to call into the "modern drag & drop" implementation, how Windows.UI.Xaml et al do it, without an assumption on the GUI framework.

Obviously you already have this on your radar, but for me the most important thing is to make files writable by default, especially if dragged from File Explorer.

Thank you for highlighting this issue. We understand this is a request from a lot of developers; we will do our best to appropriately prioritize it for Reunion as well an update to the API in the Windows SDK.

Irrespective of platform framework, these are some of the things I would expect from a Drag and Drop experience

• Custom visual for the thing whilst it is being dragged;
• Changing the visual to reflect when a drag operation will be successful or will fail;
• Enable whole window as drop target, as well as a UI element acting as a drop target;
• Metadata attached to the dragged content, so apps can provide multiple experiences depending on what content it is receiving; (images, text or formatted content etc)
• Enable drop of multiple items; (batch processing?)
• Apps can declare what content it will accept for receiving;

This a great list of the features of a delightful drag & drop experience, thank you for sharing that. FWIW, a lot of these are already provided by Windows' modern drag & drop experience as mentioned above. We hope to expand modern drag & drop to be accessible by all Windows apps via Reunion; and make it easier to use.

[stevewri]

@Ram-G : what's the next step for your team with this (e.g., project proposal?)

[Ram-G]

@stevewri We have a draft API spec out for review here #141!

[maxkatz6]

I just found that in Win32 apps you can't drag files from non-elevated app to elevated app.
I believe it should be possible for any kinds of app, but for security it could be readonly of even path-only. "Anything" would be better than current "nothing".

https://social.technet.microsoft.com/Forums/windows/en-US/cba0e9b1-25f8-40e5-a888-1435d604f68d/quotrun-as-administratorquot-prevents-drag-and-drop-working

[lukeblevins]

Supporting on-demand drag and drop operations from storage item paths

In my app scenario, we avoid creating a StorageFile or StorageFolder until it's appropriate. This is best demonstrated by the concept of "extended properties" we've established for performance reasons. In our app, users have the ability to enumerate arbitrary filesystem locations like Windows Explorer, but we found that using the Windows.Storage APIs to create an item query that represents all the child items in a directory is slower than using the Win32 FindFirstFileExFromApp() method to enumerate the core parts of each item (FileName, DateTime, and Path) upfront. Only when an item is scrolled into the viewport will we create a StorageFile/StorageFolder for extended properties like thumbnails. For this reason, we have no guarantee that every item displayed in the ListView control is backed by an IStorageItem.

This alone makes reliable dragging of storage items into another app unreliable.

We are forced to generate many storage items depending on a user's selection. When they try to drag them outside the current window, there is a high risk of failure because it takes longer than the duration of the drag gesture to load all 50+ or so storage items.

For these reasons, it would be desirable to have a method on DataPackage like so:

public void SetStorageItemPaths(IEnumerable<string> value);

which would have the operating system handle generation of storage items from the paths provided.

[avraam-inside]

Hi!

I am an active user of a very cool third-party application - windows terminal. But this application has a bug, and the developers sent me here to support the topic at the request of the function.

The bug: microsoft/terminal#12017 (Cannot move tabs when Terminal is running as Administrator)

Apparently, developers need to allow "dragging & dropping XAML elements in elevated applications". There is not and cannot be a single reason to prevent me, the user, from simply taking and moving the application tabs with the mouse.

Please implement this functionality for developers. As a last resort, make a warning return - it's bad to do that, etc., I'll calmly forget about this message, say "still give the opportunity to move tabs with the mouse" and I'll feel safe.

Don't push security policies to the point of absurdity, please.

[rlwakefield]

I 100% agree with you here. I use Terminal all day every day while I am working, but I have to be running it as my elevated admin account in order to do some of the work I need to do. And so I needed to re-order some of the tabs in my admin Terminal window and found out I couldn't. Then I came and found out that it isn't able to be done due to limitations of the programming framework.

So I 100% agree with you that they need to fix this. It has been an issue since July of 2020 and here we are 3 years later, still waiting for a solution. Hopefully this is addressed soon because I know the Terminal team would love to be able to leverage this enhancement and all.

[Jamzy01]

Are there any updates on this? I would like to use the CoreDragDropManager in a win32 app instead of OLE Apis (since they are more memory safe and easier to use/more forgiving). I just looked at #141, which seems to implement exactly what I need, but it hasn't been merged and was since closed. Does anyone here know why it wasn't merged? I think this feature would be really useful and help move along the "modernizing" of windows Apis.

[DarranRowe]

CoreDragDropManager is available to desktop applications starting from Windows 11. You would need to use the IDragDropManagerInterop interface as the activation factory. But as you may guess, this means that the CoreDragDropManager runtime class is unusable for Windows 10.

[Jamzy01]

So how exactly am I supposed to use this although?

I found this code snippet online (from https://csharp.hotexamples.com/es/examples/-/IDragDropManagerInterop/-/php-idragdropmanagerinterop-class-examples.html):

public static CoreDragDropManager GetForWindow(IntPtr hWnd)
{
    IDragDropManagerInterop dragDropManagerInterop =(IDragDropManagerInterop)WindowsRuntimeMarshal.GetActivationFactory(typeof(CoreDragDropManager));
    Guid guid = typeof(CoreDragDropManager).GUID;
    return(dragDropManagerInterop.GetForWindow(hWnd, ref guid));
}

So it looks like I need the WindowsRuntimeMarshall class. My only problem is that I'm writing my app in rust, and I can't seem to find an implementation of the Windows Runtime Marshall in the windows crate. I'm not expecting you to know rust, but it would be great if you did, but am I doing this the right way and if so, have I now just hit a roadblock because it's not in the crate?

(edit, just realized that the WindowsRuntimeMarshal is a part of the .net apis, not winrt, but I'm still not sure what to do)

[DarranRowe]

Beyond pointing you towards windows-rs, I can't say much more on this subject.
The way that you would do it in C/C++ depends on what technology you are using. The low level way of obtaining the activation factory of a WinRT component is RoGetActivationFactory, so maybe there is some way to get things working in rust using some combination of these.

[Jamzy01]

Yeah so I've been trying to get this working but it's not easy. I've got this so far,

let drag_drop_manager_interop: IDragDropManagerInterop = RoGetActivationFactory(
    &HSTRING::from("IDragDropManagerInterop")
).expect("Failed to get drag drop manager interop");

let drag_drop_manager: CoreDragDropManager = drag_drop_manager_interop
    .GetForWindow(HWND(window.hwnd().expect("Failed to get window hwnd").0))
    .expect("Failed to get core drag drop manager for window");

let _target_requested_registration = drag_drop_manager.TargetRequested(
    &TypedEventHandler::new(move |_sender, _e| {
        println!("Winrt is requesting to add a drag drop target");
        Ok(())
    })
);

And I think the rest of the code should work except for the end, but I'm getting this error:

Failed to get drag drop manager interop: Error { code: HRESULT(0x80040154), message: "Class not registered" }

Now I assume that the IDragDropManagerInterop class is registered, so I think my problem is that I'm using the incorrect class id.

The official microsoft docs says this

But I've got no idea what they mean by "id of the activatable class", and at this point I should probably point out that I have like no C# experience outside of unity. Anyway as you can see in my code, I'm just passing along "IDragDropManagerInterop" as the activatable class id, which doesn't work (unsurprising). I can't really find any documentation on this and I'm probably looking in the wrong place, so I was wondering if you had any idea what they mean by "activatable class id" and how I can find the correct one for the IDragDropManagerInterop class?

[DarranRowe]

That would be the name of the runtime class itself.
You can find these in the header files in the Windows SDK. In this particular case, it is defined in the windows.applicationmodel.datatransfer.dragdrop.core.h header.

extern const __declspec(selectany) _Null_terminated_ WCHAR RuntimeClass_Windows_ApplicationModel_DataTransfer_DragDrop_Core_CoreDragDropManager[] = L"Windows.ApplicationModel.DataTransfer.DragDrop.Core.CoreDragDropManager";

This can also be found in the C++/WinRT projection headers, but you would have to look in the 0 header, windows.applicationmodel.datatrasnfer.dragdrop.core.0.h.

template <> inline constexpr auto& name_v<winrt::Windows::ApplicationModel::DataTransfer::DragDrop::Core::CoreDragDropManager> = L"Windows.ApplicationModel.DataTransfer.DragDrop.Core.CoreDragDropManager";

It is just the fully qualified name of the runtime class itself.

[mareksm]

Get this fixed first? microsoft/microsoft-ui-xaml#9061 and relevant microsoft/microsoft-ui-xaml#8450

[mareksm]

Sorry, I'm barking at wrong tree here. It's a discussion from 2020.