目录
摘要:
docker镜像:
开源镜像:
redhat地址:
需自制镜像:
下载打包镜像:
离线镜像目录:
编译csiplugin镜像:
os镜像:
csiplugin
cfs:
客户端软件:
运行csiplugin测试
环境:
相关配置文件:
instorage.yaml
部署文件:
X86平台:
部署出错记录:
一. csi-snapshotter 创建 VolumeSnapshotResource 失败
问题描述:
摘要:
记录部署浪潮所需资源
AS13000
docker镜像:
开源镜像:
quay.io/k8scsi/csi-provisioner:v1.4.0 quay.io/k8scsi/csi-attacher:v2.2.0 quay.io/k8scsi/csi-snapshotter:v1.2.2 quay.io/k8scsi/snapshot-controller:v2.1.0 quay.io/k8scsi/csi-resizer:v0.5.0 quay.io/k8scsi/csi-node-driver-registrar:v1.3.0 |
redhat地址:
Quay
Quay
Quay
Quay
Quay
Quay
需自制镜像:
csiplugin:v2.1.1 |
下载打包镜像:
docker pull quay.io/k8scsi/csi-resizer:v0.5.0 docker pull quay.io/k8scsi/csi-attacher:v2.2.0 docker pull quay.io/k8scsi/csi-node-driver-registrar:v1.3.0 docker pull quay.io/k8scsi/csi-snapshotter:v1.2.2 docker pull quay.io/k8scsi/csi-provisioner:v1.4.0 docker pull harbor.ceclouddyn.com/paas/csi-lc-csiplugin-test:0.0.1 docker save -o csi-resizer.tar quay.io/k8scsi/csi-resizer:v0.5.0 docker save -o csi-attacher.tar quay.io/k8scsi/csi-attacher:v2.2.0 docker save -o csi-node-driver-registrar.tar quay.io/k8scsi/csi-node-driver-registrar:v1.3.0 docker save -o csi-snapshotter.tar quay.io/k8scsi/csi-snapshotter:v1.2.2 docker save -o csi-provisioner.tar quay.io/k8scsi/csi-provisioner:v1.4.0 docker save -o csi-lc-csiplugin-test.tar harbor.ceclouddyn.com/paas/csi-lc-csiplugin-test:0.0.1 |
podman load -i csi-attacher.tar podman load -i csi-lc-csiplugin-test.tar podman load -i csi-node-driver-registrar.tar podman load -i csi-provisioner.tar podman load -i csi-resizer.tar podman load -i csi-snapshotter.tar |
离线镜像目录:
\share.cecdev.cnsharezslimage |
编译csiplugin镜像:
- 代码库: https://code.cestc.cn/ccos-ops/operator-redis-ha/-/tree/trunk/csi-lc-csiplugin
- CDP: http://cdp.cestc.cn/product/#/project/pipeline/result?projectId=1509439873001431042&pipelineId=1517065513689088001
- 镜像名: harbor.ceclouddyn.com/paas/csi-lc-csiplugin-test:0.0.1
os镜像:
csiplugin
提供了二进制csiplugin, 但是需要自己制作镜像
centos-7.6.1812:for-csiplugin |
cfs:
目的是挂载然后从里边装软件
cfs-3.7.40-linux-3.10.0.iso |
客户端软件:
yum install librados2 librbd1 rbd-nbd -y |
yum install librados2 librbd1 rbd-nbd python-rados python-rbd icfs-common -y |
运行csiplugin测试
环境:
10.253.15.103
相关配置文件:
instorage.yaml
log:
enabled: false
logdir: log
level: ""
logrotatemaxsize: 0
host:
link: iscsi
scsiScanRetryTimes: 3
scsiScanWaitInterval: 0
fcNVMEScanWaitInterval: 3
fcNVMEPathCheckRetryTimes: 3
fcNVMEPathCheckWaitInterval: 3
fcScanLevel: lun
iscsiPathCheckRetryTimes: 3
iscsiPathCheckWaitInterval: 0
forceUseMultipath: false
multipathSearchRetryTimes: 3
multipathSearchWaitInterval: 3
multipathResizeDelay: 1
hostnamePrefix: ""
hostCommandTimeout: 0
deviceCleanEnabled: false
deviceCleanWaitInternal: 30
storageVendor: ""
storage:
- name: storage-01
type: AS13000
host: 10.253.20.24:8080
username: cecloud
password: "%csibmc@ceclouD99%"
domainName: ""
shadow: ""
deviceUsername: cecloud
devicePassword: "%csibmc@ceclouD99%"
deviceShadow: ""
cifsUsername: ""
cifsPassword: ""
barrierPath: ""
portalTable:
table: {}
default: []
as13000cfg:
maxTargetNumber: 0
maxLunPerTarget: 0
priorTargetNumber: 0
targetNamePrefix: ""
emptyTargetDeleted: false
directoryForceDeleted: false
moveLvmWaitInterval: 0
moveLvmRetryTimes: 0
fillLvmWaitInterval: 0
fillLvmRetryTimes: 0
cloneDirectoryWaitInterval: 0
cloneDirectoryRetryTimes: 0
as18000cfg:
lcMapWaitInterval: 0
lcMapRetryTimes: 0
task:
maxCreateVolumeNumber: 0
maxDeleteVolumeNumber: 0
maxPublishVolumeNumber: 0
maxUnpublishVolumeNumber: 0
maxStageVolumeNumber: 0
maxUnstageVolumeNumber: 0
maxCreateSnapshotNumber: 0
maxDeleteSnapshotNumber: 0
maxExpandVolumeNumber: 0
|
部署文件:
X86平台:
csiplugin.zip
部署出错记录:
一. csi-snapshotter 创建 VolumeSnapshotResource 失败
问题描述:
[root@master0 deploy]# kubectl logs csi-instorage-controller-868b58894f-d78gm -c csi-snapshotter
I0421 10:10:38.342449 1 main.go:89] Version: v1.2.2-0-gd5da3c1e
F0421 10:10:38.412889 1 create_crd.go:50] failed to create VolumeSnapshotResource: &v1beta1.CustomResourceDefinition{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1beta1.CustomResourceDefinitionSpec{Group:"", Version:"", Names:v1beta1.CustomResourceDefinitionNames{Plural:"", Singular:"", ShortNames:[]string(nil), Kind:"", ListKind:"", Categories:[]string(nil)}, Scope:"", Validation:(*v1beta1.CustomResourceValidation)(nil), Subresources:(*v1beta1.CustomResourceSubresources)(nil), Versions:[]v1beta1.CustomResourceDefinitionVersion(nil), AdditionalPrinterColumns:[]v1beta1.CustomResourceColumnDefinition(nil), Conversion:(*v1beta1.CustomResourceConversion)(nil)}, Status:v1beta1.CustomResourceDefinitionStatus{Conditions:[]v1beta1.CustomResourceDefinitionCondition(nil), AcceptedNames:v1beta1.CustomResourceDefinitionNames{Plural:"", Singular:"", ShortNames:[]string(nil), Kind:"", ListKind:"", Categories:[]string(nil)}, StoredVersions:[]string(nil)}}, err: &errors.StatusError{ErrStatus:v1.Status{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ListMeta:v1.ListMeta{SelfLink:"", ResourceVersion:"", Continue:""}, Status:"Failure", Message:"the server could not find the requested resource", Reason:"NotFound", Details:(*v1.StatusDetails)(0xc0002d8b40), Code:404}}
|
Status:"Failure", Message:"the server could not find the requested resource" |
部署CRD的时候提示版本不支持CRD
[root@master0 external-snapshotter-2.1-crd]# kubectl apply -f . --validate=false unable to recognize "snapshot.storage.k8s.io_volumesnapshotclasses.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1" unable to recognize "snapshot.storage.k8s.io_volumesnapshotcontents.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1" unable to recognize "snapshot.storage.k8s.io_volumesnapshots.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1" |
换了CRD的apiVersion
[root@master0 external-snapshotter-2.1-crd]# kubectl apply -f . --validate=false
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"api-approved.kubernetes.io":"https://github.com/kubernetes-csi/external-snapshotter/pull/260","controller-gen.kubebuilder.io/version":"v0.2.5","kubectl.kubernetes.io/last-applied-configuration":"{"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","metadata":{"annotations":{"api-approved.kubernetes.io":"https://github.com/kubernetes-csi/external-snapshotter/pull/260","controller-gen.kubebuilder.io/version":"v0.2.5"},"creationTimestamp":null,"name":"volumesnapshotclasses.snapshot.storage.k8s.io"},"spec":{"additionalPrinterColumns":[{"JSONPath":".driver","name":"Driver","type":"string"},{"JSONPath":".deletionPolicy","description":"Determines whether a VolumeSnapshotContent created through the VolumeSnapshotClass should be deleted when its bound VolumeSnapshot is deleted.","name":"DeletionPolicy","type":"string"},{"JSONPath":".metadata.creationTimestamp","name":"Age","type":"date"}],"group":"snapshot.storage.k8s.io","names":{"kind":"VolumeSnapshotClass","listKind":"VolumeSnapshotClassList","plural":"volumesnapshotclasses","singular":"volumesnapshotclass"},"preserveUnknownFields":false,"scope":"Cluster","subresources":{},"validation":{"openAPIV3Schema":{"description":"VolumeSnapshotClass specifies parameters that a underlying storage system uses when creating a volume snapshot. A specific VolumeSnapshotClass is used by specifying its name in a VolumeSnapshot object. VolumeSnapshotClasses are non-namespaced","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"deletionPolicy":{"description":"deletionPolicy determines whether a VolumeSnapshotContent created through the VolumeSnapshotClass should be deleted when its bound VolumeSnapshot is deleted. Supported values are \"Retain\" and \"Delete\". \"Retain\" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are kept. \"Delete\" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are deleted. Required.","enum":["Delete","Retain"],"type":"string"},"driver":{"description":"driver is the name of the storage driver that handles this VolumeSnapshotClass. Required.","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"parameters":{"additionalProperties":{"type":"string"},"description":"parameters is a key-value map with storage driver specific parameters for creating snapshots. These values are opaque to Kubernetes.","type":"object"}},"required":["deletionPolicy","driver"],"type":"object"}},"version":"v1beta1","versions":[{"name":"v1beta1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}n"}},"spec":{"additionalPrinterColumns":[{"JSONPath":".driver","name":"Driver","type":"string"},{"JSONPath":".deletionPolicy","description":"Determines whether a VolumeSnapshotContent created through the VolumeSnapshotClass should be deleted when its bound VolumeSnapshot is deleted.","name":"DeletionPolicy","type":"string"},{"JSONPath":".metadata.creationTimestamp","name":"Age","type":"date"}],"preserveUnknownFields":false,"subresources":{},"validation":{"openAPIV3Schema":{"description":"VolumeSnapshotClass specifies parameters that a underlying storage system uses when creating a volume snapshot. A specific VolumeSnapshotClass is used by specifying its name in a VolumeSnapshot object. VolumeSnapshotClasses are non-namespaced","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"deletionPolicy":{"description":"deletionPolicy determines whether a VolumeSnapshotContent created through the VolumeSnapshotClass should be deleted when its bound VolumeSnapshot is deleted. Supported values are "Retain" and "Delete". "Retain" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are kept. "Delete" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are deleted. Required.","enum":["Delete","Retain"],"type":"string"},"driver":{"description":"driver is the name of the storage driver that handles this VolumeSnapshotClass. Required.","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"parameters":{"additionalProperties":{"type":"string"},"description":"parameters is a key-value map with storage driver specific parameters for creating snapshots. These values are opaque to Kubernetes.","type":"object"}},"required":["deletionPolicy","driver"],"type":"object"}},"version":"v1beta1","versions":[{"name":"v1beta1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}
to:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"
Name: "volumesnapshotclasses.snapshot.storage.k8s.io", Namespace: ""
for: "snapshot.storage.k8s.io_volumesnapshotclasses.yaml": CustomResourceDefinition.apiextensions.k8s.io "volumesnapshotclasses.snapshot.storage.k8s.io" is invalid: spec.versions[0].schema.openAPIV3Schema: Required value: schemas are required
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"api-approved.kubernetes.io":"https://github.com/kubernetes-csi/external-snapshotter/pull/260","controller-gen.kubebuilder.io/version":"v0.2.5","kubectl.kubernetes.io/last-applied-configuration":"{"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","metadata":{"annotations":{"api-approved.kubernetes.io":"https://github.com/kubernetes-csi/external-snapshotter/pull/260","controller-gen.kubebuilder.io/version":"v0.2.5"},"creationTimestamp":null,"name":"volumesnapshotcontents.snapshot.storage.k8s.io"},"spec":{"additionalPrinterColumns":[{"JSONPath":".status.readyToUse","description":"Indicates if a snapshot is ready to be used to restore a volume.","name":"ReadyToUse","type":"boolean"},{"JSONPath":".status.restoreSize","description":"Represents the complete size of the snapshot in bytes","name":"RestoreSize","type":"integer"},{"JSONPath":".spec.deletionPolicy","description":"Determines whether this VolumeSnapshotContent and its physical snapshot on the underlying storage system should be deleted when its bound VolumeSnapshot is deleted.","name":"DeletionPolicy","type":"string"},{"JSONPath":".spec.driver","description":"Name of the CSI driver used to create the physical snapshot on the underlying storage system.","name":"Driver","type":"string"},{"JSONPath":".spec.volumeSnapshotClassName","description":"Name of the VolumeSnapshotClass to which this snapshot belongs.","name":"VolumeSnapshotClass","type":"string"},{"JSONPath":".spec.volumeSnapshotRef.name","description":"Name of the VolumeSnapshot object to which this VolumeSnapshotContent object is bound.","name":"VolumeSnapshot","type":"string"},{"JSONPath":".metadata.creationTimestamp","name":"Age","type":"date"}],"group":"snapshot.storage.k8s.io","names":{"kind":"VolumeSnapshotContent","listKind":"VolumeSnapshotContentList","plural":"volumesnapshotcontents","singular":"volumesnapshotcontent"},"preserveUnknownFields":false,"scope":"Cluster","subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"VolumeSnapshotContent represents the actual \"on-disk\" snapshot object in the underlying storage system","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"spec":{"description":"spec defines properties of a VolumeSnapshotContent created by the underlying storage system. Required.","properties":{"deletionPolicy":{"description":"deletionPolicy determines whether this VolumeSnapshotContent and its physical snapshot on the underlying storage system should be deleted when its bound VolumeSnapshot is deleted. Supported values are \"Retain\" and \"Delete\". \"Retain\" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are kept. \"Delete\" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are deleted. In dynamic snapshot creation case, this field will be filled in with the \"DeletionPolicy\" field defined in the VolumeSnapshotClass the VolumeSnapshot refers to. For pre-existing snapshots, users MUST specify this field when creating the VolumeSnapshotContent object. Required.","enum":["Delete","Retain"],"type":"string"},"driver":{"description":"driver is the name of the CSI driver used to create the physical snapshot on the underlying storage system. This MUST be the same as the name returned by the CSI GetPluginName() call for that driver. Required.","type":"string"},"source":{"description":"source specifies from where a snapshot will be created. This field is immutable after creation. Required.","properties":{"snapshotHandle":{"description":"snapshotHandle specifies the CSI \"snapshot_id\" of a pre-existing snapshot on the underlying storage system. This field is immutable.","type":"string"},"volumeHandle":{"description":"volumeHandle specifies the CSI \"volume_id\" of the volume from which a snapshot should be dynamically taken from. This field is immutable.","type":"string"}},"type":"object"},"volumeSnapshotClassName":{"description":"name of the VolumeSnapshotClass to which this snapshot belongs.","type":"string"},"volumeSnapshotRef":{"description":"volumeSnapshotRef specifies the VolumeSnapshot object to which this VolumeSnapshotContent object is bound. VolumeSnapshot.Spec.VolumeSnapshotContentName field must reference to this VolumeSnapshotContent's name for the bidirectional binding to be valid. For a pre-existing VolumeSnapshotContent object, name and namespace of the VolumeSnapshot object MUST be provided for binding to happen. This field is immutable after creation. Required.","properties":{"apiVersion":{"description":"API version of the referent.","type":"string"},"fieldPath":{"description":"If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: \"spec.containers{name}\" (where \"name\" refers to the name of the container that triggered the event) or if no container name is specified \"spec.containers[2]\" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.","type":"string"},"kind":{"description":"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"name":{"description":"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names","type":"string"},"namespace":{"description":"Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/","type":"string"},"resourceVersion":{"description":"Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency","type":"string"},"uid":{"description":"UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids","type":"string"}},"type":"object"}},"required":["deletionPolicy","driver","source","volumeSnapshotRef"],"type":"object"},"status":{"description":"status represents the current information of a snapshot.","properties":{"creationTime":{"description":"creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in with the \"creation_time\" value returned from CSI \"CreateSnapshotRequest\" gRPC call. For a pre-existing snapshot, this field will be filled with the \"creation_time\" value returned from the CSI \"ListSnapshots\" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.","format":"int64","type":"integer"},"error":{"description":"error is the latest observed error during snapshot creation, if any.","properties":{"message":{"description":"message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.","type":"string"},"time":{"description":"time is the timestamp when the error was encountered.","format":"date-time","type":"string"}},"type":"object"},"readyToUse":{"description":"readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in with the \"ready_to_use\" value returned from CSI \"CreateSnapshotRequest\" gRPC call. For a pre-existing snapshot, this field will be filled with the \"ready_to_use\" value returned from the CSI \"ListSnapshots\" gRPC call if the driver supports it, otherwise, this field will be set to \"True\". If not specified, it means the readiness of a snapshot is unknown.","type":"boolean"},"restoreSize":{"description":"restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in with the \"size_bytes\" value returned from CSI \"CreateSnapshotRequest\" gRPC call. For a pre-existing snapshot, this field will be filled with the \"size_bytes\" value returned from the CSI \"ListSnapshots\" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.","format":"int64","minimum":0,"type":"integer"},"snapshotHandle":{"description":"snapshotHandle is the CSI \"snapshot_id\" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.","type":"string"}},"type":"object"}},"required":["spec"],"type":"object"}},"version":"v1beta1","versions":[{"name":"v1beta1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}n"}},"spec":{"additionalPrinterColumns":[{"JSONPath":".status.readyToUse","description":"Indicates if a snapshot is ready to be used to restore a volume.","name":"ReadyToUse","type":"boolean"},{"JSONPath":".status.restoreSize","description":"Represents the complete size of the snapshot in bytes","name":"RestoreSize","type":"integer"},{"JSONPath":".spec.deletionPolicy","description":"Determines whether this VolumeSnapshotContent and its physical snapshot on the underlying storage system should be deleted when its bound VolumeSnapshot is deleted.","name":"DeletionPolicy","type":"string"},{"JSONPath":".spec.driver","description":"Name of the CSI driver used to create the physical snapshot on the underlying storage system.","name":"Driver","type":"string"},{"JSONPath":".spec.volumeSnapshotClassName","description":"Name of the VolumeSnapshotClass to which this snapshot belongs.","name":"VolumeSnapshotClass","type":"string"},{"JSONPath":".spec.volumeSnapshotRef.name","description":"Name of the VolumeSnapshot object to which this VolumeSnapshotContent object is bound.","name":"VolumeSnapshot","type":"string"},{"JSONPath":".metadata.creationTimestamp","name":"Age","type":"date"}],"preserveUnknownFields":false,"subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"VolumeSnapshotContent represents the actual "on-disk" snapshot object in the underlying storage system","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"spec":{"description":"spec defines properties of a VolumeSnapshotContent created by the underlying storage system. Required.","properties":{"deletionPolicy":{"description":"deletionPolicy determines whether this VolumeSnapshotContent and its physical snapshot on the underlying storage system should be deleted when its bound VolumeSnapshot is deleted. Supported values are "Retain" and "Delete". "Retain" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are kept. "Delete" means that the VolumeSnapshotContent and its physical snapshot on underlying storage system are deleted. In dynamic snapshot creation case, this field will be filled in with the "DeletionPolicy" field defined in the VolumeSnapshotClass the VolumeSnapshot refers to. For pre-existing snapshots, users MUST specify this field when creating the VolumeSnapshotContent object. Required.","enum":["Delete","Retain"],"type":"string"},"driver":{"description":"driver is the name of the CSI driver used to create the physical snapshot on the underlying storage system. This MUST be the same as the name returned by the CSI GetPluginName() call for that driver. Required.","type":"string"},"source":{"description":"source specifies from where a snapshot will be created. This field is immutable after creation. Required.","properties":{"snapshotHandle":{"description":"snapshotHandle specifies the CSI "snapshot_id" of a pre-existing snapshot on the underlying storage system. This field is immutable.","type":"string"},"volumeHandle":{"description":"volumeHandle specifies the CSI "volume_id" of the volume from which a snapshot should be dynamically taken from. This field is immutable.","type":"string"}},"type":"object"},"volumeSnapshotClassName":{"description":"name of the VolumeSnapshotClass to which this snapshot belongs.","type":"string"},"volumeSnapshotRef":{"description":"volumeSnapshotRef specifies the VolumeSnapshot object to which this VolumeSnapshotContent object is bound. VolumeSnapshot.Spec.VolumeSnapshotContentName field must reference to this VolumeSnapshotContent's name for the bidirectional binding to be valid. For a pre-existing VolumeSnapshotContent object, name and namespace of the VolumeSnapshot object MUST be provided for binding to happen. This field is immutable after creation. Required.","properties":{"apiVersion":{"description":"API version of the referent.","type":"string"},"fieldPath":{"description":"If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.","type":"string"},"kind":{"description":"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"name":{"description":"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names","type":"string"},"namespace":{"description":"Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/","type":"string"},"resourceVersion":{"description":"Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency","type":"string"},"uid":{"description":"UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids","type":"string"}},"type":"object"}},"required":["deletionPolicy","driver","source","volumeSnapshotRef"],"type":"object"},"status":{"description":"status represents the current information of a snapshot.","properties":{"creationTime":{"description":"creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in with the "creation_time" value returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.","format":"int64","type":"integer"},"error":{"description":"error is the latest observed error during snapshot creation, if any.","properties":{"message":{"description":"message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.","type":"string"},"time":{"description":"time is the timestamp when the error was encountered.","format":"date-time","type":"string"}},"type":"object"},"readyToUse":{"description":"readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in with the "ready_to_use" value returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.","type":"boolean"},"restoreSize":{"description":"restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in with the "size_bytes" value returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.","format":"int64","minimum":0,"type":"integer"},"snapshotHandle":{"description":"snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.","type":"string"}},"type":"object"}},"required":["spec"],"type":"object"}},"version":"v1beta1","versions":[{"name":"v1beta1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}
to:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"
Name: "volumesnapshotcontents.snapshot.storage.k8s.io", Namespace: ""
for: "snapshot.storage.k8s.io_volumesnapshotcontents.yaml": CustomResourceDefinition.apiextensions.k8s.io "volumesnapshotcontents.snapshot.storage.k8s.io" is invalid: spec.versions[0].schema.openAPIV3Schema: Required value: schemas are required
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"api-approved.kubernetes.io":"https://github.com/kubernetes-csi/external-snapshotter/pull/260","controller-gen.kubebuilder.io/version":"v0.2.5","kubectl.kubernetes.io/last-applied-configuration":"{"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","metadata":{"annotations":{"api-approved.kubernetes.io":"https://github.com/kubernetes-csi/external-snapshotter/pull/260","controller-gen.kubebuilder.io/version":"v0.2.5"},"creationTimestamp":null,"name":"volumesnapshots.snapshot.storage.k8s.io"},"spec":{"additionalPrinterColumns":[{"JSONPath":".status.readyToUse","description":"Indicates if a snapshot is ready to be used to restore a volume.","name":"ReadyToUse","type":"boolean"},{"JSONPath":".spec.source.persistentVolumeClaimName","description":"Name of the source PVC from where a dynamically taken snapshot will be created.","name":"SourcePVC","type":"string"},{"JSONPath":".spec.source.volumeSnapshotContentName","description":"Name of the VolumeSnapshotContent which represents a pre-provisioned snapshot.","name":"SourceSnapshotContent","type":"string"},{"JSONPath":".status.restoreSize","description":"Represents the complete size of the snapshot.","name":"RestoreSize","type":"string"},{"JSONPath":".spec.volumeSnapshotClassName","description":"The name of the VolumeSnapshotClass requested by the VolumeSnapshot.","name":"SnapshotClass","type":"string"},{"JSONPath":".status.boundVolumeSnapshotContentName","description":"The name of the VolumeSnapshotContent to which this VolumeSnapshot is bound.","name":"SnapshotContent","type":"string"},{"JSONPath":".status.creationTime","description":"Timestamp when the point-in-time snapshot is taken by the underlying storage system.","name":"CreationTime","type":"date"},{"JSONPath":".metadata.creationTimestamp","name":"Age","type":"date"}],"group":"snapshot.storage.k8s.io","names":{"kind":"VolumeSnapshot","listKind":"VolumeSnapshotList","plural":"volumesnapshots","singular":"volumesnapshot"},"preserveUnknownFields":false,"scope":"Namespaced","subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"VolumeSnapshot is a user's request for either creating a point-in-time snapshot of a persistent volume, or binding to a pre-existing snapshot.","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"spec":{"description":"spec defines the desired characteristics of a snapshot requested by a user. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshots#volumesnapshots Required.","properties":{"source":{"description":"source specifies where a snapshot will be created from. This field is immutable after creation. Required.","properties":{"persistentVolumeClaimName":{"description":"persistentVolumeClaimName specifies the name of the PersistentVolumeClaim object in the same namespace as the VolumeSnapshot object where the snapshot should be dynamically taken from. This field is immutable.","type":"string"},"volumeSnapshotContentName":{"description":"volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object. This field is immutable.","type":"string"}},"type":"object"},"volumeSnapshotClassName":{"description":"volumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. If not specified, the default snapshot class will be used if one exists. If not specified, and there is no default snapshot class, dynamic snapshot creation will fail. Empty string is not allowed for this field. TODO(xiangqian): a webhook validation on empty string. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes","type":"string"}},"required":["source"],"type":"object"},"status":{"description":"status represents the current information of a snapshot. NOTE: status can be modified by sources other than system controllers, and must not be depended upon for accuracy. Controllers should only use information from the VolumeSnapshotContent object after verifying that the binding is accurate and complete.","properties":{"boundVolumeSnapshotContentName":{"description":"boundVolumeSnapshotContentName represents the name of the VolumeSnapshotContent object to which the VolumeSnapshot object is bound. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: Specified boundVolumeSnapshotContentName alone does not mean binding is valid. Controllers MUST always verify bidirectional binding between VolumeSnapshot and VolumeSnapshotContent to avoid possible security issues.","type":"string"},"creationTime":{"description":"creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in with the \"creation_time\" value returned from CSI \"CreateSnapshotRequest\" gRPC call. For a pre-existing snapshot, this field will be filled with the \"creation_time\" value returned from the CSI \"ListSnapshots\" gRPC call if the driver supports it. If not specified, it indicates that the creation time of the snapshot is unknown.","format":"date-time","type":"string"},"error":{"description":"error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported.","properties":{"message":{"description":"message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.","type":"string"},"time":{"description":"time is the timestamp when the error was encountered.","format":"date-time","type":"string"}},"type":"object"},"readyToUse":{"description":"readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in with the \"ready_to_use\" value returned from CSI \"CreateSnapshotRequest\" gRPC call. For a pre-existing snapshot, this field will be filled with the \"ready_to_use\" value returned from the CSI \"ListSnapshots\" gRPC call if the driver supports it, otherwise, this field will be set to \"True\". If not specified, it means the readiness of a snapshot is unknown.","type":"boolean"},"restoreSize":{"anyOf":[{"type":"integer"},{"type":"string"}],"description":"restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in with the \"size_bytes\" value returned from CSI \"CreateSnapshotRequest\" gRPC call. For a pre-existing snapshot, this field will be filled with the \"size_bytes\" value returned from the CSI \"ListSnapshots\" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","x-kubernetes-int-or-string":true}},"type":"object"}},"required":["spec"],"type":"object"}},"version":"v1beta1","versions":[{"name":"v1beta1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}n"}},"spec":{"additionalPrinterColumns":[{"JSONPath":".status.readyToUse","description":"Indicates if a snapshot is ready to be used to restore a volume.","name":"ReadyToUse","type":"boolean"},{"JSONPath":".spec.source.persistentVolumeClaimName","description":"Name of the source PVC from where a dynamically taken snapshot will be created.","name":"SourcePVC","type":"string"},{"JSONPath":".spec.source.volumeSnapshotContentName","description":"Name of the VolumeSnapshotContent which represents a pre-provisioned snapshot.","name":"SourceSnapshotContent","type":"string"},{"JSONPath":".status.restoreSize","description":"Represents the complete size of the snapshot.","name":"RestoreSize","type":"string"},{"JSONPath":".spec.volumeSnapshotClassName","description":"The name of the VolumeSnapshotClass requested by the VolumeSnapshot.","name":"SnapshotClass","type":"string"},{"JSONPath":".status.boundVolumeSnapshotContentName","description":"The name of the VolumeSnapshotContent to which this VolumeSnapshot is bound.","name":"SnapshotContent","type":"string"},{"JSONPath":".status.creationTime","description":"Timestamp when the point-in-time snapshot is taken by the underlying storage system.","name":"CreationTime","type":"date"},{"JSONPath":".metadata.creationTimestamp","name":"Age","type":"date"}],"preserveUnknownFields":false,"subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"VolumeSnapshot is a user's request for either creating a point-in-time snapshot of a persistent volume, or binding to a pre-existing snapshot.","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds","type":"string"},"spec":{"description":"spec defines the desired characteristics of a snapshot requested by a user. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshots#volumesnapshots Required.","properties":{"source":{"description":"source specifies where a snapshot will be created from. This field is immutable after creation. Required.","properties":{"persistentVolumeClaimName":{"description":"persistentVolumeClaimName specifies the name of the PersistentVolumeClaim object in the same namespace as the VolumeSnapshot object where the snapshot should be dynamically taken from. This field is immutable.","type":"string"},"volumeSnapshotContentName":{"description":"volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object. This field is immutable.","type":"string"}},"type":"object"},"volumeSnapshotClassName":{"description":"volumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. If not specified, the default snapshot class will be used if one exists. If not specified, and there is no default snapshot class, dynamic snapshot creation will fail. Empty string is not allowed for this field. TODO(xiangqian): a webhook validation on empty string. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes","type":"string"}},"required":["source"],"type":"object"},"status":{"description":"status represents the current information of a snapshot. NOTE: status can be modified by sources other than system controllers, and must not be depended upon for accuracy. Controllers should only use information from the VolumeSnapshotContent object after verifying that the binding is accurate and complete.","properties":{"boundVolumeSnapshotContentName":{"description":"boundVolumeSnapshotContentName represents the name of the VolumeSnapshotContent object to which the VolumeSnapshot object is bound. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: Specified boundVolumeSnapshotContentName alone does not mean binding is valid. Controllers MUST always verify bidirectional binding between VolumeSnapshot and VolumeSnapshotContent to avoid possible security issues.","type":"string"},"creationTime":{"description":"creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in with the "creation_time" value returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates that the creation time of the snapshot is unknown.","format":"date-time","type":"string"},"error":{"description":"error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported.","properties":{"message":{"description":"message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.","type":"string"},"time":{"description":"time is the timestamp when the error was encountered.","format":"date-time","type":"string"}},"type":"object"},"readyToUse":{"description":"readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in with the "ready_to_use" value returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.","type":"boolean"},"restoreSize":{"anyOf":[{"type":"integer"},{"type":"string"}],"description":"restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in with the "size_bytes" value returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.","pattern":"^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$","x-kubernetes-int-or-string":true}},"type":"object"}},"required":["spec"],"type":"object"}},"version":"v1beta1","versions":[{"name":"v1beta1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}
to:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"
Name: "volumesnapshots.snapshot.storage.k8s.io", Namespace: ""
for: "snapshot.storage.k8s.io_volumesnapshots.yaml": CustomResourceDefinition.apiextensions.k8s.io "volumesnapshots.snapshot.storage.k8s.io" is invalid: spec.versions[0].schema.openAPIV3Schema: Required value: schemas are required
|
