lxd recover : 'mountpoint' does not apply to datasets of this type.

[StormYudi]

Required information

• Distribution: Ubuntu
• Distribution version: Ubuntu 20.04 LTS
• The output of "lxc info" or if that fails:
  • Kernel version: 5.4.0-153-generic
  • LXC version: 5.15-002fa0f
  • LXD version: 5.15-002fa0f
  • Storage backend in use: ZFS

Issue description

I lost my os disk recently, so I reinstall ubuntu with lxd on a new disk. I was trying to use lxd recover command to restore my data zfs pool. all containers, vm are here on the zfs pool named data.

root@rvh-1:/var/snap/lxd/common/lxd/storage-pools/data/containers# lxd recover
This LXD server currently has the following storage pools:
Would you like to recover another storage pool? (yes/no) [default=no]: yes
Name of the storage pool: data
Name of the storage backend (btrfs, ceph, cephfs, cephobject, dir, lvm, zfs): zfs
Source of the storage pool (block device, volume group, dataset, path, ... as applicable): data
Additional storage pool configuration property (KEY=VALUE, empty when done): zfs.pool_name=data
Additional storage pool configuration property (KEY=VALUE, empty when done): 
Would you like to recover another storage pool? (yes/no) [default=no]: 
The recovery process will be scanning the following storage pools:
 - NEW: "data" (backend="zfs", source="data")
Would you like to continue with scanning for lost volumes? (yes/no) [default=yes]: 
Scanning for unknown volumes...
Error: Failed validation request: Failed checking volumes on pool "data": Failed to run: zfs set mountpoint=legacy canmount=noauto data/custom/default_rvh2-epbackup: exit status 1 (cannot set property for 'data/custom/default_rvh2-epbackup': 'mountpoint' does not apply to datasets of this type)

data/custom/default_rvh2-epbackup is a custom volume of zfs, I use this volume to attach to one of my vm.

It seems that LXD is trying to execute a command which is not supported by ZFS?

my zfs list:

[tomponline]

Please can you show the full zfs list -t all output?

Also, can you confirm if those volumes are block mode custom volumes?

[StormYudi]

Sorry the list is too long, so I uploaded that to ubuntu Pastebin:
https://pastebin.ubuntu.com/p/5XySMgKrjj/

I am pretty sure I create that volumes by the guide, so it should in block mode:

[tomponline]

Thanks, so looks like lxd recover needs to learn how to handle custom block volumes.

[tomponline]

@monstermunchkin please can you take a look at this and check its not a regression related to the changes we've made to zfs block mode filesystem volumes lately. Thanks

[StormYudi]

If that is a bug, could we fix that ASAP?
Because it is our production server, or do we have other methods to skip unsupported volumes and recover other containers, and virtual machines first?