Installation steps of kvdo + vdo on Debian 11(.3) #47
Assignees
Comments
This was referenced Apr 25, 2022
|
Script improved (thanks to this topic) : install_dependencies() {
(( ${#} > 1 )) && {
apt update && \
apt install -y "$@"
} || echo "No dependency to install"
}
clean_dependencies() {
(( ${#} > 1 )) && {
apt autoremove --purge -y "$@"
} || echo "No dependency to remove"
}
clone_repo() {
(( ${#} == 1 )) && {
local TEMPORARY_DIRECTORY=$(mktemp -d "/tmp/${1/*\/}.XXX") && \
git clone "$1" "${TEMPORARY_DIRECTORY}" && \
cd "${TEMPORARY_DIRECTORY}" && \
pwd -P
}
}
# Usage: clean $WORK_DIRECTORY $DEPENDENCIES
clean() {
(( ${#} >= 1 )) && {
clean_dependencies "${*:2}" > /dev/null 2>&1 &
rm -Rf "$1" &
wait
}
}
# First part : KVDO module
DEPENDENCIES="git make linux-headers-$(uname -r)"
install_dependencies $DEPENDENCIES > /dev/null 2>&1
WORK_DIRECTORY="$(clone_repo https://github.com/tigerblue77/kvdo.git)"
(
cd "${WORK_DIRECTORY}" && \
make -j"$(nproc)" -C "/usr/src/linux-headers-$(uname -r)" M="${WORK_DIRECTORY}"
) > kvdo_build.log 2>&1
cp "${WORK_DIRECTORY}/vdo/kvdo.ko" "${WORK_DIRECTORY}/uds/uds.ko" "/lib/modules/$(uname -r)"
clean "${WORK_DIRECTORY}" $DEPENDENCIES
# Second Part: VDO Module
DEPENDENCIES="git make gcc uuid-dev libz-dev libdevmapper-dev libblkid-dev"
install_dependencies $DEPENDENCIES > /dev/null 2>&1
WORK_DIRECTORY="$(clone_repo https://github.com/dm-vdo/vdo.git)"
(
cd "${WORK_DIRECTORY}" && \
make -j"$(nproc)" && \
make -j"$(nproc)" install
) > vdo_build.log 2>&1
clean "${WORK_DIRECTORY}" $DEPENDENCIES
depmod --quick
#update-initramfs -u
#echo uds >>/etc/modules && \
#echo kvdo >> /etc/modules
modprobe kvdo
modprobe udsIt now :
|
These lines are only needed if you intend to load the uds and kvdo modules at early boot time (e.g. if you put VDO underneath the rootfs, somehow, but that's not recommended right now)
Left a comment inline. Only needed if you intend to load the modules at early boot time, like if you were using VDO for a rootfs-like implementation. Regarding tweaks to the script, I don't have anything over and above what you have shared so far. If I get some spare time I can look a bit closer at it to see if anything stands out to me, but on the surface it looks like it should be fine. |
|
Thanks for taking a look and for explanation! If it seems fine to you, maybe we could add it to the documentation/wiki, creating a new "Debian VDO Integration Guide" section that would be useful for users of all Debian-based distributions at the same time. Of course, if you have time to test and improve the script you're welcome ! Also, do you have any idea about upgrading ? Do I just need to run the same script or do I have to delete/uninstall some things before upgrading ? I plan using VDO for my personal server storage so I want to be sure my data is safe 😅 thanks per advance ! |
|
Hello there ! I updated the script to use DKMS so that the UDS and KVDO kernel modules are kept on kernel upgrade. Don't hesitate to suggest any improvment ! 😁 This one works for kvdo 8.1.1.371 but not for 8.2.0.18 as install_dependencies() {
(( ${#} > 1 )) && {
apt update && \
apt install -y "$@"
} || echo "No dependency to install"
}
clean_dependencies() {
(( ${#} > 1 )) && {
apt autoremove --purge -y "$@"
} || echo "No dependency to remove"
}
clone_repo() {
(( ${#} == 1 )) && {
local TEMPORARY_DIRECTORY=$(mktemp -d "/tmp/${1/*\/}.XXX") && \
git clone "$1" "${TEMPORARY_DIRECTORY}" && \
cd "${TEMPORARY_DIRECTORY}" && \
pwd -P
}
}
# Usage: clean $WORK_DIRECTORY $DEPENDENCIES $OTHERS $...
clean() {
(( ${#} >= 1 )) && {
clean_dependencies "${*:2}" > /dev/null 2>&1 &
rm -Rf "$1" &
wait
}
}
# First part : KVDO module
DEPENDENCIES="make linux-headers-$(uname -r)"
install_dependencies $DEPENDENCIES dkms > /dev/null 2>&1
KVDO_module_version_to_install="8.1.1.371.fixed" # "8.2.0.18"
KVDO_ARCHIVE_NAME="kvdo.tar.gz"
KVDO_ARCHIVE_PATH="/tmp/$KVDO_ARCHIVE_NAME"
KVDO_MODULE_PATH="/usr/src"
wget -O $KVDO_ARCHIVE_PATH "https://github.com/tigerblue77/kvdo/archive/refs/tags/$KVDO_module_version_to_install.tar.gz" && \
tar -xf $KVDO_ARCHIVE_PATH -C $KVDO_MODULE_PATH
echo "PACKAGE_NAME=\"kvdo\"
PACKAGE_VERSION=\"$KVDO_module_version_to_install\"
AUTOINSTALL=\"yes\"
BUILT_MODULE_NAME[0]=\"uds\"
BUILT_MODULE_LOCATION[0]=\"uds\"
DEST_MODULE_LOCATION[0]=\"/kernel/drivers/block/\"
STRIP[0]=\"no\"
BUILT_MODULE_NAME[1]=\"kvdo\"
BUILT_MODULE_LOCATION[1]=\"vdo\"
DEST_MODULE_LOCATION[1]=\"/kernel/drivers/block/\"
STRIP[1]=\"no\"" > "$KVDO_MODULE_PATH/kvdo-$KVDO_module_version_to_install/dkms.conf"
dkms add "kvdo/$KVDO_module_version_to_install"
dkms build "kvdo/$KVDO_module_version_to_install"
dkms install "kvdo/$KVDO_module_version_to_install"
#cp "${WORK_DIRECTORY}/vdo/kvdo.ko" "${WORK_DIRECTORY}/uds/uds.ko" "/lib/modules/$(uname -r)"
clean $DEPENDENCIES $KVDO_ARCHIVE_PATH
# Second Part: VDO Module
DEPENDENCIES="git make gcc uuid-dev libz-dev libdevmapper-dev libblkid-dev"
install_dependencies $DEPENDENCIES > /dev/null 2>&1
WORK_DIRECTORY="$(clone_repo https://github.com/dm-vdo/vdo.git)"
(
cd "${WORK_DIRECTORY}" && \
make -j"$(nproc)" && \
make -j"$(nproc)" install
) > vdo_build.log 2>&1
clean "${WORK_DIRECTORY}" $DEPENDENCIES
depmod --quick
#update-initramfs -u
#echo uds >>/etc/modules && \
#echo kvdo >> /etc/modules
# No longer needed thanks to DKMS
#modprobe kvdo
#modprobe udsUsed sources: |
|
For v8.2.0.18, this should work : install_dependencies() {
(( ${#} > 1 )) && {
apt update && \
apt install -y "$@"
} || echo "No dependency to install"
}
clean_dependencies() {
(( ${#} > 1 )) && {
apt autoremove --purge -y "$@"
} || echo "No dependency to remove"
}
clone_repo() {
(( ${#} == 1 )) && {
local TEMPORARY_DIRECTORY=$(mktemp -d "/tmp/${1/*\/}.XXX") && \
git clone "$1" "${TEMPORARY_DIRECTORY}" && \
cd "${TEMPORARY_DIRECTORY}" && \
pwd -P
}
}
# Usage: clean $WORK_DIRECTORY $DEPENDENCIES $OTHERS $...
clean() {
(( ${#} >= 1 )) && {
clean_dependencies "${*:2}" > /dev/null 2>&1 &
rm -Rf "$1" &
wait
}
}
# First part : KVDO module
apt update && apt install -y dkms
DEPENDENCIES="make linux-headers-$(uname -r)"
install_dependencies $DEPENDENCIES > /dev/null 2>&1
KVDO_module_version_to_install="8.2.0.18.fixed" # "8.2.0.18"
KVDO_ARCHIVE_NAME="kvdo.tar.gz"
KVDO_ARCHIVE_PATH="/tmp/$KVDO_ARCHIVE_NAME"
KVDO_MODULE_PATH="/usr/src"
wget -O $KVDO_ARCHIVE_PATH "https://github.com/tigerblue77/kvdo/archive/refs/tags/$KVDO_module_version_to_install.tar.gz" && \
tar -xf $KVDO_ARCHIVE_PATH -C $KVDO_MODULE_PATH
echo "PACKAGE_NAME=\"kvdo\"
PACKAGE_VERSION=\"$KVDO_module_version_to_install\"
AUTOINSTALL=\"yes\"
BUILT_MODULE_NAME[0]=\"kvdo\"
BUILT_MODULE_LOCATION[0]=\"vdo\"
DEST_MODULE_LOCATION[0]=\"/kernel/drivers/block/\"
BUILD_DEPENDS[0]=LZ4_COMPRESS
BUILD_DEPENDS[0]=LZ4_DECOMPRESS
STRIP[0]=\"no\"" > "$KVDO_MODULE_PATH/kvdo-$KVDO_module_version_to_install/dkms.conf"
dkms add "kvdo/$KVDO_module_version_to_install"
dkms build "kvdo/$KVDO_module_version_to_install"
dkms install "kvdo/$KVDO_module_version_to_install"
#cp "${WORK_DIRECTORY}/vdo/kvdo.ko" "${WORK_DIRECTORY}/uds/uds.ko" "/lib/modules/$(uname -r)"
clean $DEPENDENCIES $KVDO_ARCHIVE_PATH
# Second Part: VDO Module
DEPENDENCIES="git make gcc uuid-dev libz-dev libdevmapper-dev libblkid-dev"
install_dependencies $DEPENDENCIES > /dev/null 2>&1
WORK_DIRECTORY="$(clone_repo https://github.com/dm-vdo/vdo.git)"
(
cd "${WORK_DIRECTORY}" && \
make -j"$(nproc)" && \
make -j"$(nproc)" install
) > vdo_build.log 2>&1
clean "${WORK_DIRECTORY}" $DEPENDENCIES
depmod --quick
#update-initramfs -u
#echo uds >>/etc/modules && \
#echo kvdo >> /etc/modules
# No longer needed thanks to DKMS
#modprobe kvdo
#modprobe udsBut give me the following errors : |
|
Hello @rhawalsh, |
|
Hello @tigerblue77, thanks for the effort here. I was able to build and install 8.1.1.371 with Your script. According to this BIO_MAX_VECS has been renamed to BIO_MAX_PAGES in the block subsystem. Could You try to apply a renaming patch on the sources, and give a try? Thanks in advance. |
|
Hello @kondas, thanks for your answer. I'm very motivated by this project but I don't manage to make it work perfectly on Debian... make -j"$(nproc)" -C "/usr/src/linux-headers-$(uname -r)" M=`pwd`And output for v8.2.0.18 : => I just checked KVDO v8.2.0.18/v8.2.0.21/v8.2.1.2 releases archives and this patch is not applied in any of them. That's why I also have this bug on my fork (which only include a fix for 5.10 kernel which is currently used by Debian 11 stable). |
|
I can compile the module successfully from Your 8.2.1.2.fixed archive, after changing BIO_MAX_VECS to BIO_MAX_PAGES and completely remove the failing include from num-utils.h. Disclaimer: I have no idea what am I doing, but it compiles without warnings. |
This was referenced Dec 4, 2022
|
This is what worked for me in debain 12.6 TO fix : error: ‘BIO_MAX_PAGES’ undeclared here (not in a function); did you mean ‘BIO_MAX_VECS’?
TO fix : Skipping BTF generation for /home/munish259272/Extrasoftware/test/kvdo/vdo/kvdo.ko due to unavailability of vmlinux
unix.stackexchange.com TO fix : /bin/sh: 1: ./tools/bpf/resolve_btfids/resolve_btfids: not foundnow we can compile and copy kvdo.kodid not face any issue with vdo |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello everyone,
I wrote this little script to install kvdo + vdo on Debian 11(.3) (tested with 5.10.0-13 Linux kernel)
VDO's part questions (second part) :
README.md?I tested this script with the following LVM volume creation script :
I wrote this script with help of those forum topics :
The text was updated successfully, but these errors were encountered: