containerd 使用镜像加速站的问题

[guoguobaba]

在 cf 做了一个镜像加速站，docker.dockerimage.site ，然后 docker pull 完全没问题，rancher 要使用 containerd,但是 containerd 会有问题。首先 containerd 需要配置镜像加速站信息，在/etc/containerd/config.toml 增加

[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
    endpoint = ["https://docker.dockerimage.site"]

然后可以用下列命令拉镜像，但是总是拉不成功，后来发现是卡在它仍然要去 auth.docker.io 获取 token 才行

# ctr images pull docker.dockerimage.site/library/busybox:latest --http-dump
WARN[0000] DEPRECATION: CRI API v1alpha2 is deprecated since containerd v1.7 and removed in containerd v2.0. Use CRI API v1 instead.
INFO[0000] HEAD /v2/library/busybox/manifests/latest HTTP/1.1
INFO[0000] Host: docker.dockerimage.site
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] User-Agent: containerd/1.6.33
INFO[0000]
docker.dockerimage.site/library/busybox:latest: resolving      |--------------------------------------|
elapsed: 2.3 s                             total:   0.0 B (0.0 B/s)
INFO[0002] HTTP/1.1 401 Unauthorized
INFO[0002] Content-Length: 158
INFO[0002] Alt-Svc: h3=":443"; ma=86400
INFO[0002] Cf-Cache-Status: DYNAMIC
INFO[0002] Cf-Ray: 8dacb0f3ad3752a7-LAX
INFO[0002] Connection: keep-alive
INFO[0002] Content-Type: application/json
INFO[0002] Date: Wed, 30 Oct 2024 16:13:11 GMT
INFO[0002] Docker-Distribution-Api-Version: registry/2.0
INFO[0002] Docker-Ratelimit-Source: 172.69.34.71
INFO[0002] Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
INFO[0002] Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=eCxfmwJynLDUZ57Fsf1DW8e3gpQh9glOwIfkSle72jTtm8fOESra46%2B7tCEaJ44oh2dVfBTc5D%2BlRree5qSHjIawJYqJy242B0LyjKi%2BSTTZsKPaImz6q3GkRr%2FhIgfQRuXpc3Y%3D"}],"group":"cf-nel","max_age":604800}
INFO[0002] Server: cloudflare
INFO[0002] Server-Timing: cfL4;desc="?proto=TCP&rtt=235288&sent=8&recv=9&lost=0&retrans=2&sent_bytes=4543&recv_bytes=678&delivery_rate=4479&cwnd=246&unsent_bytes=0&cid=8029cb73bf98260e&ts=1014&x=0"
INFO[0002] Strict-Transport-Security: max-age=31536000
INFO[0002] Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:library/busybox:pull"
INFO[0002]
INFO[0002] GET /token?scope=repository%3Alibrary%2Fbusybox%3Apull&service=registry.docker.io HTTP/1.1
INFO[0002] Host: auth.docker.io
docker.dockerimage.site/library/busybox:latest: resolving      |--------------------------------------|
elapsed: 23.4s                             total:   0.0 B (0.0 B/s)
INFO[0023] trying next host                              error="failed to authorize: failed to fetch anonymous token: Get \"https://auth.docker.io/token?scope=repository%3Alibrary%2Fbusybox%3Apull&service=registry.docker.io\": dial tcp 199.59.149.231:443: connect: connection refused" host=docker.dockerimage.site
ctr: failed to resolve reference "docker.dockerimage.site/library/busybox:latest": failed to authorize: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fbusybox%3Apull&service=registry.docker.io": dial tcp 199.59.149.231:443: connect: connection refused

镜像加速站用的是这个 https://github.com/ciiiii/cloudflare-docker-proxy 在 cf 上创建的，看到有人提了一个类似的 issue https://github.com/ciiiii/cloudflare-docker-proxy/issues/79 。不知道是否有解决方案。

[evill]

可能是版本配置问题，有两种配置方式
这是我目前使用的

# /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"

#/etc/containerd/certs.d/
#└── docker.io/
# ├── ca.crt # CA 证书
# ├── client.cert # 客户端证书（如果需要）
# ├── client.key # 客户端密钥（如果需要）
# └── hosts.toml # Harbor 映射配置文件
#/etc/containerd/certs.d/docker.io/hosts.toml
# docker 为代理 project 名称
server = "https://harbor.xxxxxxxxx.cn"

[host."https://harbor.xxxxxx.cn/v2/docker"]
capabilities = ["pull","resolve"]
override_path = true
[host."https://harbor.xxxxxxx.cn".header]
Authorization = ["Basic <password-base64>"]

[guoguobaba]

@evill 这个不影响啊，无非就是配置 registry ，你这个是私有的，估计不会认证 auth.docker.io ，我这个只是相当于代理。

[hongyexiaoqing]

镜像代理服务器问题，它只是个代理，不是 registry mirror ，你无法解决，除非服务端帮你完成验证 token

这个服务端直接透传给你，没有帮你跳过取得 token 步骤
```
Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:library/busybox:pull"
```

[guoguobaba]

@hongyexiaoqing 所以我想看看有没有别的镜像加速器的方案，目前搜到最简单的是这个，docker pull 没问题，containerd 就有问题，所以应该还有别的解决方案。

[xuyan1994]

endpoint = ["http://docker.1panel.live", "https://dockerhub.icu", "https://hub.rat.dev", "https://docker.awsl9527.cn", "https://docker.awsl9527.cn"]

[xuyan1994]

用我这个试试看

[guoguobaba]

@xuyan1994 你这个原理和我一样，docker pull 没有问题，containerd 不行。我走代理的方案了。

[guoguobaba]

https://github.com/cmliu/CF-Workers-docker.io/blob/main/_worker.js 换了这个 worker ，支持 containerd 了。

[suofeiya]

加个代理直接拉吧，有的镜像站镜像不全，还有的镜像不在白名单内拉不了.

[aru]

我现在用的 sniproxy + hosts 解决
192.168.1.11 是运行 sniproxy 的主机

192.168.1.11 docker.io
192.168.1.11 auth.docker.io
192.168.1.11 registry-1.docker.io
192.168.1.11 production.cloudflare.docker.com