Failure to connect to etcd server

[planetA]

Expected Behavior

Connecting to etcd should not end up in an error message

Current Behavior

sudo ./dist/flanneld
I0615 08:03:01.418560   91154 main.go:207] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:false kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:false ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
I0615 08:03:01.419274   91154 main.go:227] Created subnet manager: Etcd Local Manager with Previous Subnet: 10.5.65.0/24
I0615 08:03:01.419475   91154 main.go:230] Installing signal handlers
E0615 08:03:01.423054   91154 main.go:459] Couldn't fetch network config: rpc error: code = InvalidArgument desc = etcdserver: key not found
timed out
E0615 08:03:02.426129   91154 main.go:459] Couldn't fetch network config: rpc error: code = InvalidArgument desc = etcdserver: key not found
timed out

The last two lines are printed repeatedly.

Possible Solution

Git bisect point to the following two commits: b40d827 f4326fc

The versions before these commits work as expected. The ones after fail.

Steps to Reproduce (for bugs)

1. Install etcd from a repo
2. Launch flanneld manually on the same host

Context

Trying to launch flanneld manually, not using Kubernetes or Docker.

Your Environment

• Flannel version: v0.18.1
• Backend used (e.g. vxlan or udp): vxlan
• Etcd version:

etcd --version
etcd Version: 3.3.25
Git SHA: Not provided (use ./build instead of go build)
Go Version: go1.15.9
Go OS/Arch: linux/amd64

• Operating System and version:

lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

[salmon5]

https://etcd.io/docs/v3.3/dev-guide/interacting_v3/
"By default, etcdctl on master (3.4) uses the v3 API and earlier versions (3.3 and earlier) default to the v2 API."

maybe need to upgrade etcd to 3.4 or 3.5 version

[thomasferrandiz]

This looks like your flannel configuration was written as etcdv2 data but flannel now uses etcdv3.

Could you try using etcdctl with export ETCDCTL_API=3 to make sure etcdv3 is used?

More details are available in etcd documentation: https://etcd.io/docs/v3.5/op-guide/v2-migration/

[haifengjia]

I0615 08:03:01.419274 91154 main.go:227] Created subnet manager: Etcd Local Manager with Previous Subnet: 10.5.65.0/24
I0615 08:03:01.419475 91154 main.go:230] Installing signal handlers
E0615 08:03:01.423054 91154 main.go:459] Couldn't fetch network config: rpc error: code = InvalidArgument desc = etcdserver: key not found
timed out
E0615 08:03:02.426129 91154 main.go:459] Couldn't fetch network config: rpc error: code = InvalidArgument desc = etcdserver: key not found

I had the same issue with @planetA and solved it by resetting flanneld==v0.17.0.

This looks like your flannel configuration was written as etcdv2 data but flannel now uses etcdv3.

Thanks for the information provided by @thomasferrandiz , which made me realize that we could just reset the version of flannel early enough to use etcdv2 rather than etcdv3.

[planetA]

@thomasferrandiz Hi, setting v3 API works. But it would be nice to have a warning or an error message in this case.

[xyh15864643181]

I0615 08:03:01.419274 91154 main.go:227] 创建子网管理器：Etcd 本地管理器，以前的子网：10.5.65.0/24 I0615 08:03:01.419475
91154 main.go:230] 安装信号处理程序
E0615 08:03： 01.423054 91154 main.go:459] 无法获取网络配置：rpc 错误：代码 = InvalidArgument desc = etcdserver: 找不到密钥
超时
E0615 08:03:02.426129 91154 main.go:459] 无法获取网络配置： rpc 错误：代码 = InvalidArgument desc = etcdserver：找不到密钥

我也有同样的问题@planetA 并通过重置解决了它flanneld==v0.17.0。

看起来您的 flannel 配置是作为 etcdv2 数据编写的，但 flannel 现在使用 etcdv3。

感谢您提供的信息@thomasferrandiz，这让我意识到我们可以尽早重置 flannel 的版本来使用，etcdv2而不是etcdv3.

我的etcd是3.3.27 一开始用的flanneld是0,18，我将版本重置到0.17，就能开始成功了，谢谢