mirror of
https://github.com/labring/sealos.git
synced 2026-09-24 15:46:19 +08:00
299 lines
14 KiB
Markdown
299 lines
14 KiB
Markdown
# `sealos exec` 命令
|
|
|
|
> 支持在指定节点执行自定义命令, 拷贝一个文件到一些指定节点
|
|
|
|
如在所有master节点创建一个目录
|
|
|
|
```
|
|
sealos exec --cmd "mkdir /data" --label node-role.kubernetes.io/master=""
|
|
sealos exec --cmd "mkdir /data" --node x.x.x.x
|
|
sealos exec --cmd "mkdir /data" --node dev-k8s-mater
|
|
```
|
|
|
|
拷贝一个文件到一些指定节点
|
|
|
|
```
|
|
sealos exec --src /data/foo --dst /root/foo --label node-role.kubernetes.io/master=""
|
|
sealos exec --src /data/foo --dst /root/foo --node x.x.x.x
|
|
sealos exec --src /data/foo --dst /root/foo --node dev-k8s-mater
|
|
```
|
|
|
|
## 实现方法
|
|
|
|
### 获取待执行的`ip`序列逻辑
|
|
|
|
将 `--label , --node hostname` 所关联的对象全部转为`ip`, 以`ip`为基点去执行`copy`或者是`cmd`.
|
|
|
|
使用`ListOptions`, 通过`labelSelector`直接定位到相关的`nodeList`节点. 如果`label`为空, 则返回空. 如果不为空, 则返回的`ip` 加入到待执行的`ip`序列.
|
|
|
|
```
|
|
func GetNodeListByLabel(k8sClient *kubernetes.Clientset, label string) (*v1.NodeList, error) {
|
|
listOption := &metav1.ListOptions{LabelSelector: label}
|
|
return k8sClient.CoreV1().Nodes().List(context.TODO(), *listOption)
|
|
}
|
|
func GetNodeIpByLabel(k8sClient *kubernetes.Clientset, label string) ([]string, error) {
|
|
var ips []string
|
|
if label == "" {
|
|
return ips, nil
|
|
}
|
|
nodes, err := GetNodeListByLabel(k8sClient, label)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
for _, node := range nodes.Items {
|
|
for _, v := range node.Status.Addresses {
|
|
if v.Type == v1.NodeInternalIP {
|
|
ips = append(ips, v.Address)
|
|
}
|
|
}
|
|
}
|
|
if len(ips) != 0 {
|
|
return ips, nil
|
|
}
|
|
return nil, fmt.Errorf("label %s is not fount in kubernetes nodes", label)
|
|
}
|
|
```
|
|
|
|
将 `--node` 进行区分, 如果是`ip` ,则加入执行`ip`序列, 如果是`hostname`, 则加入到`hostname`序列, 通过使用Get方法, 获取K8s的ClientSet资源对象.通过`nodename`直接找到定位到node节点. 在通过一次`for loop` 找到`hostname`对应的`ip`, 将得到的`ip`加入到待执行的`ip`序列.
|
|
|
|
```
|
|
node, err := k8sClient.CoreV1().Nodes().Get(context.TODO(), nodeName, metav1.GetOptions{})
|
|
|
|
for _, node := range resHost {
|
|
ip, err := GetNodeIpByName(k8sClient, node)
|
|
if err == nil {
|
|
ips = append(ips, ip)
|
|
}
|
|
}
|
|
```
|
|
|
|
这里对`ips`没有进行过滤. 只要满足是ipv4, 即加入到执行序列, 没有对`master ips`及`nodes ips`进行比对.
|
|
|
|
```
|
|
// 集群
|
|
k8s-master 192.168.0.31
|
|
huohua-test 192.168.0.30
|
|
server65 192.168.0.65
|
|
server88-new 192.168.0.88
|
|
|
|
// sealos exec --cmd "hostname" --node 192.168.0.21
|
|
// 192.168.0.21 is not in your kubernetes. but if your ssh access 192.168.0.21. the command will be exec in 192.168.0.21.
|
|
```
|
|
|
|
### 执行命令逻辑
|
|
|
|
判断是执行`copy` 或者是 `cmd`方法如下. 如果两者都存在, 则先执行`copy`逻辑, 再执行`cmd`逻辑.
|
|
|
|
```
|
|
type ExecFlag struct {
|
|
Dst string
|
|
Src string
|
|
Cmd string
|
|
Label string
|
|
ExecNode []string
|
|
SealConfig
|
|
}
|
|
|
|
func (e *ExecFlag) IsUseCopy() bool {
|
|
return FileExist(e.Src) && e.Dst != ""
|
|
}
|
|
|
|
func (e *ExecFlag) IsUseCmd() bool {
|
|
return e.Cmd != ""
|
|
}
|
|
```
|
|
|
|
### 远程命令和复制实现
|
|
|
|
实现`scp`复制, 则是通过复制单个文件, 然后递归复制即可. 查看具体的[源码](https://github.com/fanux/sealos/blob/develop/pkg/sshcmd/sshutil/scp.go)
|
|
如果`--dst`在目标机器存在, 则不执行copy动作, 直接就跳过了.
|
|
|
|
// todo
|
|
这里是否需要添加一个flag, 比如`--force, -f`, 直接覆盖? 或者先删除再复制?
|
|
|
|
```
|
|
// CopyLocalToRemote is copy file or dir to remotePath
|
|
func (ss *SSH) CopyLocalToRemote(host, localPath, remotePath string) {
|
|
|
|
}
|
|
// ssh session is a problem, 复用ssh链接
|
|
func (ss *SSH) copyLocalDirToRemote(sftpClient *sftp.Client, localPath, remotePath string) {
|
|
|
|
}
|
|
// solve the session
|
|
func (ss *SSH) copyLocalFileToRemote(sftpClient *sftp.Client, localPath, remotePath string) {
|
|
|
|
}
|
|
```
|
|
|
|
实现执行命令的逻辑. 这里是用的ssh. 具体不赘述.
|
|
|
|
## 使用方法
|
|
|
|
```
|
|
$ ./sealos exec -h
|
|
support exec cmd or copy file by Label/nodes
|
|
|
|
Usage:
|
|
sealos exec [flags]
|
|
|
|
Examples:
|
|
|
|
# exec cmd by label or nodes. when --label and --node is Exist, get Union of both.
|
|
sealos exec --cmd "mkdir /data" --label node-role.kubernetes.io/master= --node 192.168.0.2
|
|
sealos exec --cmd "mkdir /data" --node 192.168.0.2 --nodes dev-k8s-mater
|
|
|
|
# exec copy src file to dst by label or nodes. when --label and --node is Exist, get Union of both.
|
|
sealos exec --src /data/foo --dst /root/foo --label node-role.kubernetes.io/master=""
|
|
sealos exec --src /data/foo --dst /root/foo --node 192.168.0.2
|
|
|
|
|
|
Flags:
|
|
--cmd string exec command string
|
|
--dst string dest file location
|
|
-h, --help help for exec
|
|
--label string kubernetes labels like node-role.kubernetes.io/master=
|
|
--node strings node ip or hostname in kubernetes
|
|
--src string source file location
|
|
|
|
Global Flags:
|
|
--config string config file (default is $HOME/.sealos/config.yaml)
|
|
```
|
|
|
|
选项说明:
|
|
|
|
- `--cmd`: 执行的命令.
|
|
- `--src`: 本地文件路径, 可以使文件或者是文件夹, 配合`--dst`使用.
|
|
- `--dst`: 目标文件路径, 配合`--src`使用.
|
|
- `--label`: `kubernetes`集群的`label`. 支持`label`逻辑表达式, 如 `kubernetes.io/arch!=amd64`
|
|
- `--node`: 节点的`ip`或者是`hostname`
|
|
|
|
## 测试相关
|
|
|
|
集群如下
|
|
|
|
```
|
|
$ kubectl get nodes --show-labels
|
|
NAME STATUS ROLES AGE VERSION LABELS
|
|
huohua-test Ready <none> 93d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=huohua-test,kubernetes.io/os=linux,name=huohua
|
|
k8s-master Ready master 93d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master,kubernetes.io/os=linux,node-role.kubernetes.io/master=
|
|
server65 Ready <none> 89d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server65,kubernetes.io/os=linux
|
|
server88-new Ready <none> 90d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server88-new,kubernetes.io/os=linux,name=front
|
|
```
|
|
|
|
#### 仅使用 `--label` 执行 `cmd` 命令
|
|
|
|
<details><summary><code>1. Use --label to exec cmd </code> Output</summary><br><pre>
|
|
$ ./sealos exec --cmd "hostname " --label beta.kubernetes.io/arch=amd64
|
|
19:09:35 [INFO] [ssh.go:57] [ssh][192.168.0.88] cd /tmp && hostname
|
|
19:09:35 [INFO] [ssh.go:57] [ssh][192.168.0.31] cd /tmp && hostname
|
|
19:09:35 [INFO] [ssh.go:57] [ssh][192.168.0.30] cd /tmp && hostname
|
|
19:09:35 [INFO] [ssh.go:57] [ssh][192.168.0.65] cd /tmp && hostname
|
|
19:09:35 [INFO] [ssh.go:50] [192.168.0.88] server88-new
|
|
19:09:35 [INFO] [ssh.go:50] [192.168.0.30] huohua-test
|
|
19:09:35 [INFO] [ssh.go:50] [192.168.0.65] server65
|
|
19:09:36 [INFO] [ssh.go:50] [192.168.0.31] k8s-master
|
|
</pre></details>
|
|
|
|
#### 仅使用 `--node` 执行 `cmd` 命令
|
|
|
|
<details><summary><code>2. Use --node to exec cmd </code> Output</summary><br><pre>
|
|
$ ./sealos exec --cmd "hostname -i" --node huohua-test --node 192.168.0.65
|
|
19:20:47 [INFO] [ssh.go:57] [ssh][192.168.0.30] cd /tmp && hostname -i
|
|
19:20:47 [INFO] [ssh.go:57] [ssh][192.168.0.65] cd /tmp && hostname -i
|
|
19:20:47 [INFO] [ssh.go:50] [192.168.0.30] 192.168.0.30
|
|
19:20:47 [INFO] [ssh.go:50] [192.168.0.65] 192.168.0.65
|
|
</pre></details>
|
|
|
|
#### 使用 `--label` 和 `--node`执行 `cmd` 命令
|
|
|
|
<details><summary><code>3. Use --node & --label to exec cmd </code> Output</summary><br><pre>
|
|
$ ./sealos exec --cmd "hostname -i" --node huohua-test --label node-role.kubernetes.io/master=
|
|
19:21:44 [INFO] [ssh.go:57] [ssh][192.168.0.30] cd /tmp && hostname -i
|
|
19:21:44 [INFO] [ssh.go:57] [ssh][192.168.0.31] cd /tmp && hostname -i
|
|
19:21:44 [INFO] [ssh.go:50] [192.168.0.30] 192.168.0.30
|
|
19:21:45 [INFO] [ssh.go:50] [192.168.0.31] 192.168.0.31
|
|
</pre></details>
|
|
|
|
#### 使用 `--label` 和 `--node`执行 `cmd` 命令和 复制文件
|
|
|
|
<details><summary><code>4. Use --node & --label to exec cmd & copy files </code> Output</summary><br><pre>
|
|
$ ./sealos exec --cmd "ls -lh /data/01.txt" --src /root/01.txt --dst /data/01.txt --node huohua-test --label node-role.kubernetes.io/master=
|
|
19:23:01 [INFO] [ssh.go:12] [ssh][192.168.0.30] ls -l /data/01.txt 2>/dev/null |wc -l
|
|
19:23:01 [INFO] [ssh.go:12] [ssh][192.168.0.31] ls -l /data/01.txt 2>/dev/null |wc -l
|
|
19:23:01 [DEBG] [ssh.go:24] [ssh][192.168.0.30]command result is: 0
|
|
19:23:02 [INFO] [scp.go:328] [ssh]transfer [/root/01.txt] total size is: 2.11MB ;speed is 2MB
|
|
19:23:02 [DEBG] [ssh.go:24] [ssh][192.168.0.31]command result is: 0
|
|
19:23:02 [INFO] [scp.go:328] [ssh]transfer [/root/01.txt] total size is: 2.11MB ;speed is 2MB
|
|
19:23:02 [INFO] [ssh.go:57] [ssh][192.168.0.30] cd /tmp && ls -lh /data/01.txt
|
|
19:23:02 [INFO] [ssh.go:57] [ssh][192.168.0.31] cd /tmp && ls -lh /data/01.txt
|
|
19:23:02 [INFO] [ssh.go:50] [192.168.0.30] -rw-r--r-- 1 root root 2.2M 9月 4 19:23 /data/01.txt
|
|
19:23:03 [INFO] [ssh.go:50] [192.168.0.31] -rw-r--r--. 1 root root 2.2M Sep 4 19:23 /data/01.txt
|
|
</pre></details>
|
|
|
|
#### 使用 `--label` 和 `--node`执行 `cmd` 命令和 复制文件夹
|
|
|
|
<details><summary><code>5. Use --node & --label to exec cmd & copy dir </code> Output</summary><br><pre>
|
|
$ ./sealos exec --cmd "ls -lh /data/test" --src /root/test --dst /data/test --node huohua-test --label node-role.kubernetes.io/master=
|
|
19:24:24 [INFO] [ssh.go:12] [ssh][192.168.0.30] ls -l /data/test 2>/dev/null |wc -l
|
|
19:24:24 [INFO] [ssh.go:12] [ssh][192.168.0.31] ls -l /data/test 2>/dev/null |wc -l
|
|
19:24:24 [DEBG] [ssh.go:24] [ssh][192.168.0.30]command result is: 0
|
|
19:24:24 [INFO] [scp.go:328] [ssh]transfer [/root/test/crontab.yaml] total size is: 1.19KB ;speed is 1KB
|
|
19:24:24 [INFO] [scp.go:328] [ssh]transfer [/root/test/crontab.yaml.bak] total size is: 2.23KB ;speed is 2KB
|
|
19:24:24 [DEBG] [ssh.go:24] [ssh][192.168.0.31]command result is: 0
|
|
19:24:25 [INFO] [scp.go:328] [ssh]transfer [/root/test/crontab.yaml] total size is: 1.19KB ;speed is 1KB
|
|
19:24:25 [INFO] [scp.go:328] [ssh]transfer [/root/test/crontab.yaml.bak] total size is: 2.23KB ;speed is 2KB
|
|
19:24:25 [INFO] [ssh.go:57] [ssh][192.168.0.30] cd /tmp && ls -lh /data/test
|
|
19:24:25 [INFO] [ssh.go:57] [ssh][192.168.0.31] cd /tmp && ls -lh /data/test
|
|
19:24:25 [INFO] [ssh.go:50] [192.168.0.30] 总用量 8.0K
|
|
19:24:25 [INFO] [ssh.go:50] [192.168.0.30] -rw-r--r-- 1 root root 1.2K 9月 4 19:24 crontab.yaml
|
|
19:24:25 [INFO] [ssh.go:50] [192.168.0.31] total 8.0K
|
|
</pre></details>
|
|
|
|
#### 使用 `--label`时, `label` 不存在
|
|
|
|
<details><summary><code>6. Use --label if label not exist </code> Output</summary><br><pre>
|
|
$ ./sealos exec --cmd "hostname -i" --node huohua-test --label node-role.kubernete.
|
|
12:48:25 [EROR] [exec.go:53] get ips err: unable to parse requirement: invalid label key "node-role.kubernete.": name part must consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyName', or 'my.name', or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]')
|
|
$ ./sealos exec --cmd "hostname -i" --node huohua-test --label node-role.kubernete
|
|
12:48:42 [EROR] [exec.go:53] get ips err: label node-role.kubernete is not fount in kubernetes nodes
|
|
</pre></details>
|
|
|
|
#### 使用 `--node`时, `node`不存在
|
|
|
|
<details><summary><code>7. Use --node if node not exist in kuernetes </code> Output</summary><br><pre>
|
|
12:50:05 [INFO] [ssh.go:50] [192.168.0.88] server88-new
|
|
12:50:05 [INFO] [ssh.go:50] [192.168.0.30] huohua-test
|
|
12:50:05 [INFO] [ssh.go:50] [192.168.0.65] server65
|
|
12:50:05 [INFO] [ssh.go:50] [192.168.0.31] k8s-master
|
|
## this will output nothing
|
|
$ ./sealos exec --cmd "hostname -i" --node huohua-test031
|
|
## only exec on exsit nodes.
|
|
./sealos exec --cmd "hostname -i" --node huohua-test031 --node 192.168.0.65
|
|
12:51:28 [INFO] [ssh.go:57] [ssh][192.168.0.65] cd /tmp && hostname -i
|
|
12:51:29 [INFO] [ssh.go:50] [192.168.0.65] 192.168.0.65
|
|
## when nodes ip is format but is not in kubernetes, will appear ssh session error , timeout.
|
|
$ ./sealos exec --cmd "hostname -i" --node huohua-test031 --node 192.168.9.65
|
|
12:52:14 [INFO] [ssh.go:57] [ssh][192.168.9.65] cd /tmp && hostname -i
|
|
12:53:14 [EROR] [ssh.go:60] [ssh][192.168.9.65]Error create ssh session failed,dial tcp 192.168.9.65:22: i/o timeout
|
|
</pre></details>
|
|
|
|
#### 支持`kubernetes label`的逻辑表达式.
|
|
|
|
<details><summary><code>8. support kubernetes label exp like kubernetes.io/arch!=amd64 </code> Output</summary><br><pre>
|
|
$ kubectl get nodes --show-labels
|
|
NAME STATUS ROLES AGE VERSION LABELS
|
|
huohua-test Ready <none> 93d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=huohua-test,kubernetes.io/os=linux,name=huohua
|
|
k8s-master Ready master 93d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master,kubernetes.io/os=linux,node-role.kubernetes.io/master=
|
|
server65 Ready <none> 89d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server65,kubernetes.io/os=linux
|
|
server88-new Ready <none> 89d v1.18.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server88-new,kubernetes.io/os=linux,name=front
|
|
$ ./sealos exec --cmd "hostname -i" --label name!=front
|
|
09:17:21 [INFO] [ssh.go:57] [ssh][192.168.0.65] cd /tmp && hostname -i
|
|
09:17:21 [INFO] [ssh.go:57] [ssh][192.168.0.30] cd /tmp && hostname -i
|
|
09:17:21 [INFO] [ssh.go:57] [ssh][192.168.0.31] cd /tmp && hostname -i
|
|
09:17:21 [INFO] [ssh.go:50] [192.168.0.30] 192.168.0.30
|
|
09:17:21 [INFO] [ssh.go:50] [192.168.0.65] 192.168.0.65
|
|
09:17:21 [INFO] [ssh.go:50] [192.168.0.31] 192.168.0.31
|
|
</pre></details>
|
|
|