文档首页

全部产品

容器安全（TCSS）

租户端安全容器安全（TCSS）操作指南-新

自建集群

最近更新时间: 2024-10-17 17:10:00

接入自建集群

本文介绍接入自建集群的步骤，您可以将自建集群接入容器安全服务进行统一管理，对自建集群开展集群风险检查和管理。

限制条件

接入自建集群节点规模小于500节点。

操作步骤

登录 [容器安全服务控制台]，在左侧导航中，单击集群安全管理 > 集群检查。
在集群检查页面，单击接入自建集群。

在集群信息设置页面，配置相关参数，单击下一步。

参数说明：

参数组	参数	说明	可选项
基础信息设置	集群名称	输入自建集群的名称，64字符以内	-
	集群环境	选择自建集群的类型	Kubernetes,Openshift
	集群版本	选择集群环境的集群版本	K8s 集群支持1.13以上版本
网络信息设置	网络类型	选择通过公网或通过 VPC 网络接入自建集群	公网、VPC
	所在地域	选择自建集群所在的地域，公网类型无地域限制	-
	VPC ID	当网络类型使用 VPC 时，选择集群所在网络的 VPC 信息	-
	API Server地址	当网络类型使用 VPC 时，选择集群 API Server 后端服务类型	服务器、负载均衡
集群检查组件	安装检查组件	选择自动或者自行手动安装集群检查的组件	- 自动安装检查组件并进行一次集群检查 - 不安装检查组件，接入后自行安装组件并下发集群安装
	自动检查	是否开启集群的自动检查功能	- 开启 - 关闭

在上传配置文件，单击选择文件，上传本地文件后，单击完成接入即可接入自建集群。

注意：

公网方式接入自建集群，如果您的集群有设置访问控制策略，需单击 IP 白名单地址添加页面中的 IP 地址。

您需要在服务器上生成 K8s 配置文件后，才能上传该配置文件。生成K8s 配置文件的具体操作，请参见生成 K8S 配置文件。

上传配置文件，大小需要在 1M 以内。

生成 K8s 配置文件

本文指导您生成容器安全需要的最小化权限 K8s 配置文件。您可参照文档步骤生成配置文件，或者参见 一键脚本。

前提条件

• 已在服务器上搭建 K8s 集群。具体操作，请参见 K8s 中文官方文档。 • 已安装 Docker 服务。

操作步骤

以 root 身份登录 k8s 集群 master 所在服务器。
输入如下命令，创建命名空间和权限绑定。

# 1. 创建命名空间： tcss
# 2. 创建命名空间tcss下的管理角色： tcss-admin
# 3. 绑定角色tcss-admin和用户tcss
# 4. 创建秘钥并绑定服务账号： tcss-agent-secret，tcss-agent
# 5. 创建只读的集群角色： security-clusterrole
# 6. 绑定集群角色security-clusterrole到服务账号tcss-agent

---
apiVersion: v1
kind: Namespace
metadata:
  name: tcss


---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tcss
  name: tcss-admin
rules:
- apiGroups: ["extensions", "apps", ""]
  resources: ["*"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tcss-admin-rb
  namespace: tcss
subjects:
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tcss-admin
  apiGroup: rbac.authorization.k8s.io


---
apiVersion: v1
kind: Secret
metadata:
  name: tcss-agent-secret
  namespace: tcss
  annotations:
    kubernetes.io/service-account.name: tcss-agent
type: kubernetes.io/service-account-token


---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tcss-agent
  namespace: tcss
secrets:
  - name: tcss-agent-secret
    namespace: tcss


---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: security-clusterrole
rules:
- apiGroups: ["", "v1"]
  resources: ["namespaces", "pods", "nodes"]
  verbs: ["get", "list"]
- apiGroups: ["apps"]
  resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
  verbs: ["get", "list"]
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get", "list"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["get"]
- apiGroups: ["networking.k8s.io", "extensions"]
  resources: ["ingresses"]
  verbs: ["get", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: security-clusterrolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: security-clusterrole
subjects:
- kind: ServiceAccount
  name: tcss-agent
  namespace: tcss
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io

说明：

执行上述命令，如果能显示 pod 或者显示当前命名空间下没有相关资源，则表示该集群配置是可用的，上传该文件 /root/tcss.conf 即可。

一键脚本

在 mater 节点中，您可基于以下一键脚本代码一键快速生成集群配置文件：

说明：

环境需要提前安装 openssl。

#!/bin/bash

set -e;

# API_SERVER 需要设置为公网可访问的地址和端口
# API_SERVER=https://xx.xx.xx.xx:xxxx

# 以下路径,用户根据集群实际情况设定
KUBECONFIG_TARGET=/root/tcss.conf
CA_FILE=/etc/kubernetes/ca.crt
CAKEY_FILE=/etc/kubernetes/ca.key
TCSS_TMPDIR=/tmp/tcss
# 如果是OpenShift环境，可以更换为 oc
KUBECTL_CMD=kubectl


if [ ! $API_SERVER ]; then
    echo "API_SERVER does not set.";
    exit 1;
fi
if ! which kubectl ; then
    echo "kubectl does not exist.";
    exit 1;
fi
if [ ! -f "$CA_FILE" ]; then
    echo "$CA_FILE does not exist.";
    exit 1;
fi
if [ ! -f "$CAKEY_FILE" ]; then
    echo "$CAKEY_FILE does not exist.";
    exit 1;
fi
if [ ! -d $TCSS_TMPDIR ]; then
    mkdir -p $TCSS_TMPDIR;
fi

cat <<EOF  > $TCSS_TMPDIR/tcss_res.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: tcss

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tcss
  name: tcss-admin
rules:
- apiGroups: ["extensions", "apps", ""]
  resources: ["*"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tcss-admin-rb
  namespace: tcss
subjects:
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tcss-admin
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: v1
kind: Secret
metadata:
  name: tcss-agent-secret
  namespace: tcss
  annotations:
    kubernetes.io/service-account.name: tcss-agent
type: kubernetes.io/service-account-token

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tcss-agent
  namespace: tcss
secrets:
  - name: tcss-agent-secret
    namespace: tcss

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: security-clusterrole
rules:
- apiGroups: ["", "v1"]
  resources: ["namespaces", "pods", "nodes"]
  verbs: ["get", "list"]
- apiGroups: ["apps"]
  resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
  verbs: ["get", "list"]
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get", "list"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["get"]
- apiGroups: ["networking.k8s.io", "extensions"]
  resources: ["ingresses"]
  verbs: ["get", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: security-clusterrolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: security-clusterrole
subjects:
- kind: ServiceAccount
  name: tcss-agent
  namespace: tcss
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io
EOF

# echo "generate tcss resource file ($TCSS_TMPDIR/tcss_res.yaml) success."

$KUBECTL_CMD apply -f $TCSS_TMPDIR/tcss_res.yaml;

# 创建User私钥 tcss.key。
openssl genrsa -out $TCSS_TMPDIR/tcss.key 2048
# 创建证书签署请求 tcss.csr
openssl req -new -key $TCSS_TMPDIR/tcss.key -out $TCSS_TMPDIR/tcss.csr -subj "/O=K8s/CN=tcss"
# 签署证书 生成 tcss.crt
openssl x509 -req -in $TCSS_TMPDIR/tcss.csr -CA $CA_FILE -CAkey $CAKEY_FILE -CAcreateserial -out $TCSS_TMPDIR/tcss.crt -days 365

# 创建并设置集群配置
$KUBECTL_CMD config set-cluster tcss --server=$API_SERVER --certificate-authority=$CA_FILE --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
# 创建并设置用户配置
$KUBECTL_CMD config set-credentials tcss --client-certificate=$TCSS_TMPDIR/tcss.crt --client-key=$TCSS_TMPDIR/tcss.key --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
# 设置context配置
$KUBECTL_CMD config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=$KUBECONFIG_TARGET
# 切换context配置
$KUBECTL_CMD config use-context tcss@tcss --kubeconfig=$KUBECONFIG_TARGET

echo "generate KUBECONFIG file success. $KUBECONFIG_TARGET"

生成 Openshift 配置文件

本文指导您生成容器安全需要的最小化权限 OpenShift 配置文件。您可参照文档步骤生成配置文件，或者参见 一键脚本。

前提条件

已在服务器上搭建 K8s 集群。具体操作，请参见 K8s 中文官方文档。
已安装 Docker 服务。
暂仅支持 OpenShift3.0及以上版本接入，低于该版本可能存在不确定性问题。

操作步骤

说明：

整体接入思路和 Kubernetes 类似，只涉及相关路径和命令行工具区别，如果集群 master 节点上已经安装的 kubectl 工具，则可以完全同 Kubernetes 集群接入方式进行接入。

以 root 身份登录 OpenShift 集群 master 所在服务器。
输入如下命令，创建命名空间和权限绑定。

# 1. 创建命名空间： tcss
# 2. 创建命名空间tcss下的管理角色： tcss-admin
# 3. 绑定角色tcss-admin和用户tcss
# 4. 创建秘钥并绑定服务账号： tcss-agent-secret，tcss-agent
# 5. 创建只读的集群角色： security-clusterrole
# 6. 绑定集群角色security-clusterrole到服务账号tcss-agent

---
apiVersion: v1
kind: Namespace
metadata:
  name: tcss


---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tcss
  name: tcss-admin
rules:
- apiGroups: ["extensions", "apps", ""]
  resources: ["*"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tcss-admin-rb
  namespace: tcss
subjects:
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tcss-admin
  apiGroup: rbac.authorization.k8s.io


---
apiVersion: v1
kind: Secret
metadata:
  name: tcss-agent-secret
  namespace: tcss
  annotations:
    kubernetes.io/service-account.name: tcss-agent
type: kubernetes.io/service-account-token


---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tcss-agent
  namespace: tcss
secrets:
  - name: tcss-agent-secret
    namespace: tcss


---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: security-clusterrole
rules:
- apiGroups: ["", "v1"]
  resources: ["namespaces", "pods", "nodes"]
  verbs: ["get", "list"]
- apiGroups: ["apps"]
  resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
  verbs: ["get", "list"]
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get", "list"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["get"]
- apiGroups: ["networking.k8s.io", "extensions"]
  resources: ["ingresses"]
  verbs: ["get", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: security-clusterrolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: security-clusterrole
subjects:
- kind: ServiceAccount
  name: tcss-agent
  namespace: tcss
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io

进入配置目录（/etc/origin/master/），输入如下命令，创建证书。

# 创建User私钥 tcss.key。
openssl genrsa -out tcss.key 2048

# 创建证书签署请求 tcss.csr
openssl req -new -key tcss.key -out tcss.csr -subj "/O=K8s/CN=tcss"

# 签署证书 生成 tcss.crt
openssl x509 -req -in tcss.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tcss.crt -days 365

输入如下命令，创建集群配置文件。

# 创建并设置集群配置, 其中需要主要 server 地址必须为公网可访问地址
KUBECONFIG=/root/tcss.conf oc config set-cluster tcss --server=https://xx.xx.xx.xx:60002 --certificate-authority=/etc/origin/master/ca.crt --embed-certs=true --kubeconfig=/root/tcss.conf

# 创建并设置用户配置
KUBECONFIG=/root/tcss.conf oc config set-credentials tcss --client-certificate=tcss.crt --client-key=tcss.key --embed-certs=true --kubeconfig=/root/tcss.conf

# 设置context配置
KUBECONFIG=/root/tcss.conf oc config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=/root/tcss.conf

# 切换context配置
KUBECONFIG=/root/tcss.conf oc config use-context tcss@tcss --kubeconfig=/root/tcss.conf

输入如下命令，验证集群配置文件并上传配置。

KUBECONFIG=/root/tcss.conf oc -n tcss get pod

说明：

执行上述命令，如果能显示pod或者显示当前命名空间下没有相关资源则表示该集群配置是可用的，上传该文件 /root/tcss.conf 即可。

一键脚本

在 mater 节点中，您可基于以下一键脚本代码一键快速生成集群配置文件：

说明：

环境需要提前安装 openssl。

#!/bin/bash

set -e;

# API_SERVER 需要设置为公网可访问的地址和端口
# API_SERVER=https://xx.xx.xx.xx:xxxx

# 以下路径,用户根据集群实际情况设定
KUBECONFIG_TARGET=/root/tcss.conf
CA_FILE=/etc/kubernetes/ca.crt
CAKEY_FILE=/etc/kubernetes/ca.key
TCSS_TMPDIR=/tmp/tcss
KUBECTL_CMD=oc


if [ ! $API_SERVER ]; then
    echo "API_SERVER does not set.";
    exit 1;
fi
if ! which $KUBECTL_CMD ; then
    echo "$KUBECTL_CMD does not exist.";
    exit 1;
fi
if [ ! -f "$CA_FILE" ]; then
    echo "$CA_FILE does not exist.";
    exit 1;
fi
if [ ! -f "$CAKEY_FILE" ]; then
    echo "$CAKEY_FILE does not exist.";
    exit 1;
fi
if [ ! -d $TCSS_TMPDIR ]; then
    mkdir -p $TCSS_TMPDIR;
fi

cat <<EOF  > $TCSS_TMPDIR/tcss_res.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: tcss

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tcss
  name: tcss-admin
rules:
- apiGroups: ["extensions", "apps", ""]
  resources: ["*"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tcss-admin-rb
  namespace: tcss
subjects:
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tcss-admin
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: v1
kind: Secret
metadata:
  name: tcss-agent-secret
  namespace: tcss
  annotations:
    kubernetes.io/service-account.name: tcss-agent
type: kubernetes.io/service-account-token

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tcss-agent
  namespace: tcss
secrets:
  - name: tcss-agent-secret
    namespace: tcss

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: security-clusterrole
rules:
- apiGroups: ["", "v1"]
  resources: ["namespaces", "pods", "nodes"]
  verbs: ["get", "list"]
- apiGroups: ["apps"]
  resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
  verbs: ["get", "list"]
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get", "list"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["get"]
- apiGroups: ["networking.k8s.io", "extensions"]
  resources: ["ingresses"]
  verbs: ["get", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: security-clusterrolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: security-clusterrole
subjects:
- kind: ServiceAccount
  name: tcss-agent
  namespace: tcss
- kind: User
  name: tcss
  apiGroup: rbac.authorization.k8s.io
EOF

# echo "generate tcss resource file ($TCSS_TMPDIR/tcss_res.yaml) success."

$KUBECTL_CMD apply -f $TCSS_TMPDIR/tcss_res.yaml;
$KUBECTL_CMD adm policy add-scc-to-user privileged -n tcss -z tcss-agent;
$KUBECTL_CMD adm policy add-scc-to-user hostaccess -n tcss -z tcss-agent;
$KUBECTL_CMD adm policy add-scc-to-user privileged tcss;
$KUBECTL_CMD adm policy add-scc-to-user hostaccess tcss;
oc adm policy add-cluster-role-to-user cluster-reader tcss;

# 创建User私钥 tcss.key。
openssl genrsa -out $TCSS_TMPDIR/tcss.key 2048
# 创建证书签署请求 tcss.csr
openssl req -new -key $TCSS_TMPDIR/tcss.key -out $TCSS_TMPDIR/tcss.csr -subj "/O=K8s/CN=tcss"
# 签署证书 生成 tcss.crt
openssl x509 -req -in $TCSS_TMPDIR/tcss.csr -CA $CA_FILE -CAkey $CAKEY_FILE -CAcreateserial -out $TCSS_TMPDIR/tcss.crt -days 365

# 创建并设置集群配置
KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config set-cluster tcss --server=$API_SERVER --certificate-authority=$CA_FILE --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
# 创建并设置用户配置
KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config set-credentials tcss --client-certificate=$TCSS_TMPDIR/tcss.crt --client-key=$TCSS_TMPDIR/tcss.key --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
# 设置context配置
KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=$KUBECONFIG_TARGET
# 切换context配置
KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config use-context tcss@tcss --kubeconfig=$KUBECONFIG_TARGET

echo "generate KUBECONFIG file success. $KUBECONFIG_TARGET"