Switching tilesets overwrites current brush

[eishiya]

When switching tilesets in the Tilesets panel, the brush is set to the currently selected tiles in that tileset. This overwrites any currently set brush, and if the tileset previously had multiple tiles selected, as happens when painting with a larger brush, it's overwritten by the arrangement of those tiles in the tileset, which is rarely useful.

To Reproduce
This requires that at least two tilesets are available in the Tilesets panel.

1. Sample some tiles from a map into a brush, or choose a saved stamp. For the clearest results, this brush should consist of tiles that aren't all together in the tileset.
2. Go to another tileset tab.
3. Observe that the brush you just spent time making has been overwritten by whatever tiles were selected in that tileset.
4. Switch back to the previous tileset.
5. Observe that your brush is again changed, this time to the relevant tiles from your sampled brush, but as they occur in the tileset rather than your original brush.

Expected behaviour
Switching to a different tileset should not modify the brush if doing so would be destructive or would be likely to produce a garbage brush.
I can think of two concrete ways to make this decision:

• Simple: Do not modify the current brush if it has an area larger than 1x1 or has more than one layer.
  This would prevent complex brushes from being overwritten, but would also result in "false positives" - for example, if I select a 3x3 tree in Tileset A and switch to Tileset B where previously had a 5x5 house selected, I probably wouldn't mind having my tree overwritten by that house, and then having the house overwritten by the tree when I switch back to Tileset A. It would also overwrite 1x1 brushes that have been flipped/rotated, but this is not a big loss. Lastly, it would still produce nonsense brushes like in the above GIF, if the previous brush was a single tile - you wouldn't lose anything, but you'd still get garbage.
• Complex: Do not modify the current brush if the tile(s) in the newly selected tileset and/or in the current brush were selected by means other than manually selecting from a tileset or sampling a single tile from a map/selecting a single-tile, single-layer stamp. Brushes with multiple layers or using tiles from multiple tilesets should never be overwritten.
  This would prevent the aforementioned false positives, because it would allow Tiled to distinguish between multi-tile arrangements selected from the tileset by the user (=probably sensible) from multiple tiles that are selected from being in some other arrangement in a map or stamp.
  This one would probably not overwrite any brush that has been flipped, for better or worse.
  This one can be further improved by counting map selections and saved stamps that are identical to the tiles' arrangement in the tileset as direct selections. Perhaps trivial flips can also be ignored (i.e. if you select a bunch of tiles from a tileset and flip them, that can count as a simple brush that can be overwritten).

The above GIF is from Tiled 1.10.1 64-bit on Windows 10, but I'm pretty sure this has been happening since 1.8 and quite likely much earlier.

This is a usability problem rather than a bug, but it sure feels like a bug when your brush gets mangled by switching tilesets xP

[bjorn]

Hmm, I was thinking about how to implement the various suggestions here. It gets a little complicated. In the end, it seemed to me that we should store the stamp with the tileset, based upon which the tiles were selected, so that when you switch back to that tileset, it can just pick up that same stamp again.

However, a much easier solution would be, if switching tilesets would not touch the current brush at all. Maybe that is rather unexpected in the first place?

To make this more clear, we could, if you select some tiles from some tileset, automatically deselect any tiles selected in other tilesets. Though, this might make #3768 possibly a bit annoying, since the deselection would then also happen in the associated tileset editors. Maybe we can just keep the selected tiles, but just not change the tile stamp when switching tilesets.

[eishiya]

Storing stamps with the Tileset sounds weird to me, since a stamp can consist of tiles from several tilesets.

Maybe that is rather unexpected in the first place?

I think updating the brush can be convenient when all your brushes correspond to simple regions from the tileset and you use many tilesets, just not in more complex scenarios.

Perhaps there could be an option in the tilesets panel for this, e.g. "Automatically set brush to selected tiles"? It could affect the tile selection changing both due to the tileset being changed in the Tilesets view, and due to tiles being selected in a Tileset document due to sync (and it may need a tooltip to explain this). With that setting being off, my complaints about #3768 would be irrelevant, and the sync feature might not even need to be optional.

[bjorn]

Storing stamps with the Tileset sounds weird to me, since a stamp can consist of tiles from several tilesets.

Yeah, in this case we could store the same stamp on each of the tilesets though. But I'm not sure this approach is a good idea, anyway.

I'm not convinced that the convenience of switching brushes when switching tilesets is worth an option. I think it's rather rare people would be relying on that, and that it should be no problem to click in the tileset view again.

What I do worry about a bit is either:

• People being confused, seeing tiles selected in the tileset view, but seeing a different active stamp.
• Or, when we clear that selection, people being annoyed that the tile they had selected in the tileset editor is also no longer selected (with Share the selection between Tilesets view and Tileset Editor #3768 enabled).

However, maybe the latter is not a big problem, given that the "current tile" should stay, so stuff like collision editor, properties view and such will not switch to another tile at least. Well, unless multiple tiles were selected, which does affect the properties view.

[eishiya]

I'm not convinced that the convenience of switching brushes when switching tilesets is worth an option. I think it's rather rare people would be relying on that, and that it should be no problem to click in the tileset view again.

For a multi-tile brush (which is where this behaviour would be most useful), one would have to ctrl+click twice on a tile to update the brush, which isn't necessarily intuitive. I think some users, not knowing about ctrl+click, would complain about "having to" reselect their entire brush. This would be alleviated if these selection modifiers were documented within Tiled itself, or if the Tilesets view had selection options similar to the toolbar actions for the selection tools.

I agree that seeing different tiles selected versus what's in the brush is a weird, but I don't think the selection should be "cleared", I think it's better for the selection and brush to disagree when the syncing is added/enabled. The properties view doesn't allow you to edit tiles while in the Map Editor (which I think is a good thing since it enforces a conceptual separation between Map and Tileset, even if wasn't not a fully intentional decision) and the brush isn't visible in the Tileset Editor, so I think this disagreement isn't likely to cause any damage.

Somewhat related: Tiled has the Ctrl+[1-9] feature to save Tile Stamps, but this feature is hard to discover and requires foresight to use. Perhaps there could be some sort of automatic version of this that saves the last few used stamps that users could cycle through, perhaps via some modifier(s) + scrollwheel (I think shift+wheel might be free?)? This way, in the scenario where someone was actually relying on their brush being set to the selected tiles in a Tileset they switched to, they could instead scroll through their recent stamp history instead of changing tilesets. This would still be hard to discover, but at least it wouldn't require foresight.
Edit: And/or maybe there could be a "recent stamps" panel, which would be easier to discover, to allow quickly selecting recently used stamps in a visual manner similar to the Tile Stamps panel.