Kfserving Deployment_数据挖掘与分析

Kfserving Deployment Environment

Istio v1.9.3
Knative v0.20.0
Cert Manager v1.3.0
kfserving v0.5.1
install istioctl v1

wget https://github.com/istio/istio/releases/download/1.9.3/istio-1.9.3-linux-amd64.tar.gz
tar xf istio-1.9.3-linux-amd64.tar.gz
cd istio-1.9.3/
cp bin/istioctl /usr/local/bin

install knative

Serving Component

1、Install the Custom Resource Definitions(aka CRDs):

kubectl apply --filename https://github.com/knative/serving/releases/download/v0.20.0/serving-crds.yaml

2、Install the core components of Serving (see below for optional extensions)

kubectl apply --filename https://github.com/knative/serving/releases/download/v0.20.0/serving-core.yaml

3、Pick a networking layer:Istio

cat << EOF > ./istio-minimal-operator.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      proxy:
        autoInject: disabled
      useMCP: false
      # The third-party-jwt is not enabled on all k8s.
      # See: https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens
      jwtPolicy: first-party-jwt

  addonComponents:
    pilot:
      enabled: true

  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
EOF

istioctl install -f istio-minimal-operator.yaml
#Enable sidecar container on knative-serving system namespace.
kubectl label namespace knative-serving istio-injection=enabled

Set PeerAuthentication to PERMISSIVE on knative-serving system namespace.

cat < 
 Updating the config-istio configmap to use a non-default local gateway 
  If you create a custom service and deployment for local gateway with a name other than cluster-local-gateway, you need to update gateway configmap config-istio under the knative-serving namespace.Edit the config-istio configmap.
 kubectl edit configmap config-istio -n knative-serving
 
  Replace the local-gateway.knative-serving.cluster-local-gateway field with the custom service. As an example, if you name both the service and deployment custom-local-gateway under the namespace istio-system, it should be updated to.
 custom-local-gateway.istio-system.svc.cluster.local
 
  if both the custom service and deployment are labeled with custom: custom-local-gateway, not the default istio: cluster-local-gateway, you must update gateway instance cluster-local-gateway in the knative-serving namespace.
 kubectl edit gateway cluster-local-gateway -n knative-serving
 Replace the label selector with the label of your service:
 istio: cluster-local-gateway
 For the service above, it should be updated to:
 custom: custom-local-gateway
 If there is a change in service ports (compared to that of cluster-local-gateway), update the port info in the gateway accordingly.
 
 configuring DNS 
  look external IP
 kubectl get svc -nistio-system
 
  This external IP can be used with your DNS provider with a wildcard A record. However, for a basic non-production set up, this external IP address can be used with xip.io in the config-domain ConfigMap in knative-serving.You can edit this by using the following command
 kubectl edit cm config-domain --namespace knative-serving
 
  Given the external IP above, change the content to
 apiVersion: v1
kind: ConfigMap
metadata:
  name: config-domain
  namespace: knative-serving
data:
  # xip.io is a "magic" DNS provider, which resolves all DNS lookups for:
  # *.{ip}.xip.io to {ip}.
  34.83.80.117.xip.io: ""
  
4、Configure DNS 
option 1: Real DNS 
To configure DNS for Knative, take the External IP or CNAME from setting up networking, and configure it with your DNS provider as follows: 
If the networking layer produced an External IP address, then configure a wildcard A record for the domain: 
# Here knative.example.com is the domain suffix for your cluster
*.knative.example.com == A 35.233.41.212
 
 If the networking layer produced a CNAME, then configure a CNAME record for the domain:
 # Here knative.example.com is the domain suffix for your cluster
*.knative.example.com == CNAME a317a278525d111e89f272a164fd35fb-1510370581.eu-central-1.elb.amazonaws.com
  
once your DNS provider has been configured, direct Knative to use that domain: 
# Replace knative.example.com with your domain suffix
kubectl patch configmap/config-domain 
  --namespace knative-serving 
  --type merge 
  --patch '{"data":{"knative.example.com":""}}'
 
option 2: Magic DNS 
We ship a simple Kubernetes Job called “default domain” that will (see caveats) configure Knative Serving to use xip.ioas the default DNS suffix. 
kubectl apply --filename https://github.com/knative/serving/releases/download/v0.20.0/serving-default-domain.yaml
 
EventingComponent 
5、Install the Custom Resource Definitions(aka CRDs): 
kubectl apply --filename https://github.com/knative/eventing/releases/download/v0.20.0/eventing-crds.yaml
 
6、Install the core components of Eventing (see below for optional extensions) 
kubectl apply --filename https://github.com/knative/eventing/releases/download/v0.20.0/eventing-core.yaml
 
7、Install a default Channel (messaging) layer (alphabetical). 
Apache Kafka Channel 
 Install Apache Kafka for Kubernetes
 
 Create a namespace for your Apache Kafka installation, like kafka:
 kubectl create namespace kafka
  
  
- Install the Strimzi operator, like:
  
  ```bash
  curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.16.2/strimzi-cluster-operator-0.16.2.yaml" 
    | sed 's/namespace: .*/namespace: kafka/' 
    | kubectl -n kafka apply -f -
 
 Describe the size of your Apache Kafka installation in kafka.yaml, like:
 apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 2.4.0
    replicas: 1
    listeners:
      plain: {}
      tls: {}
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      log.message.format.version: "2.4"
    storage:
      type: ephemeral
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}
  
  
- Deploy the Apache Kafka cluster
  
  ```bash
  kubectl apply -n kafka -f kafka.yaml
 
install the Apache Kafka Channel: 
curl -L "https://github.com/knative-sandbox/eventing-kafka/releases/download/v0.20.0/channel-consolidated.yaml" 
 | sed 's/REPLACE_WITH_CLUSTER_URL/my-cluster-kafka-bootstrap.kafka:9092/' 
 | kubectl apply --filename -
 
8、Install a Broker (eventing) layer 
Apache Kafka Broker 
Install the Kafka controlle 
kubectl apply --filename https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/v0.20.0/eventing-kafka-controller.yaml
 
Install the Kafka Broker data plane 
kubectl apply --filename https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/v0.20.0/eventing-kafka-broker.yaml
 
9、install cert-manager 
Install the CustomResourceDefinitions and cert-manager itself 
$ kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.3.0/cert-manager.yaml
 
Note: When running on GKE (Google Kubernetes Engine), you may encounter a ‘permission denied’ error when creating some of these resources. This is a nuance of the way GKE handles RBAC and IAM permissions, and as such you should ‘elevate’ your own privileges to that of a ‘cluster-admin’ before running the above command. If you have already run the above command, you should run them again after elevating your permissions: 
kubectl create clusterrolebinding cluster-admin-binding 
--clusterrole=cluster-admin 
--user=$(gcloud config get-value core/account)
 
10、Install KFServing CRD and Controller 
kubectl apply -f https://github.com/kubeflow/kfserving/releases/download/0.5.1/kfserving.yaml
 
11、Setup Ingress Gateway 
If the default ingress gateway setup does not fit your need, you can choose to setup a custom ingress gateway 
 Configure Custom Ingress Gateway 
  Create Gateway Service and Deployment Instance，You’ll need to create the gateway service and deployment instance to handle traffic first. Let’s say you customized the default istio-ingressgateway to custom-ingressgateway as follows.
 apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      proxy:
        autoInject: disabled
      useMCP: false
      # The third-party-jwt is not enabled on all k8s.
      # See: https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens
      jwtPolicy: first-party-jwt

  addonComponents:
    pilot:
      enabled: true
    prometheus:
      enabled: false

  components:
    ingressGateways:
      - name: custom-ingressgateway
        enabled: true
        namespace: custom-ns
        label:
          istio: custom-gateway
      - name: cluster-local-gateway
        enabled: true
        label:
          istio: cluster-local-gateway
          app: cluster-local-gateway
        k8s:
          service:
            type: ClusterIP
            ports:
            - port: 15020
              name: status-port
            - port: 80
              name: http2
            - port: 443
              name: https
 
  Update Knative Gateway，Update gateway instance knative-ingress-gateway under knative-serving namespace:
 kubectl edit gateway knative-ingress-gateway -n knative-serving
 Replace the label selector with the label of your service:
 istio: ingressgateway
 For the service above, it should be updated to:
 istio: custom-gateway
 If there is a change in service ports (compared with that of istio-ingressgateway), update the port info in the gateway accordingly 
  Update Gateway Configmap
 Update gateway configmap config-istio under knative-serving namespace:
 kubectl edit configmap config-istio -n knative-serving
 Replace the istio-ingressgateway.istio-system.svc.cluster.local field with the fully qualified url of your service.
 gateway.knative-serving.knative-ingress-gateway: "istio-ingressgateway.istio-system.svc.cluster.local"
 For the service above, it should be updated to:
 gateway.knative-serving.knative-ingress-gateway: custom-ingressgateway.custom-ns.svc.cluster.local
 
  In addition you need to update KFServing configmap to use the custom ingress gateway.
 apiVersion: v1
kind: ConfigMap
metadata:
  name: inferenceservice-config
  namespace: kfserving-system
data:
  predictors: |-
    {
        "tensorflow": {
            "image": "tensorflow/serving",
            "defaultImageVersion": "1.14.0",
            "defaultGpuImageVersion": "1.14.0-gpu",
            "defaultTimeout": "60",
            "supportedframeworks": [
              "tensorflow"
            ],
            "multiModelServer": false
        },
        "onnx": {
            "image": "mcr.microsoft.com/onnxruntime/server",
            "defaultImageVersion": "v1.0.0",
            "supportedframeworks": [
              "onnx"
            ],
            "multiModelServer": false
        },
        "sklearn": {
          "v1": {
            "image": "gcr.io/kfserving/sklearnserver",
            "defaultImageVersion": "latest",
            "supportedframeworks": [
              "sklearn"
            ],
            "multiModelServer": true
          },
          "v2": {
            "image": "docker.io/seldonio/mlserver",
            "defaultImageVersion": "0.2.1",
            "supportedframeworks": [
              "sklearn"
            ],
            "multiModelServer": false
          }
        },
        "xgboost": {
          "v1": {
            "image": "gcr.io/kfserving/xgbserver",
            "defaultImageVersion": "latest",
            "supportedframeworks": [
              "xgboost"
            ],
            "multiModelServer": true
          },
          "v2": {
            "image": "docker.io/seldonio/mlserver",
            "defaultImageVersion": "0.2.1",
            "supportedframeworks": [
              "xgboost"
            ],
            "multiModelServer": false
          }
        },
        "pytorch": {
          "v1" : {
            "image": "gcr.io/kfserving/pytorchserver",
            "defaultImageVersion": "latest",
            "defaultGpuImageVersion": "latest-gpu",
            "supportedframeworks": [
              "pytorch"
            ],
            "multiModelServer": false
          },
          "v2" : {
            "image": "pytorch/torchserve-kfs",
            "defaultImageVersion": "0.4.0",
            "defaultGpuImageVersion": "0.4.0-gpu",
            "supportedframeworks": [
              "pytorch"
            ],
            "multiModelServer": false
          }
        },
        "triton": {
            "image": "nvcr.io/nvidia/tritonserver",
            "defaultImageVersion": "20.08-py3",
            "supportedframeworks": [
              "tensorrt",
              "tensorflow",
              "onnx",
              "pytorch",
              "caffe2"
            ],
            "multiModelServer": true
        },
        "pmml": {
            "image": "kfserving/pmmlserver",
            "defaultImageVersion": "latest",
            "supportedframeworks": [
              "pmml"
            ],
            "multiModelServer": false
        },
        "lightgbm": {
            "image": "kfserving/lgbserver",
            "defaultImageVersion": "latest",
            "supportedframeworks": [
              "lightgbm"
            ],
            "multiModelServer": false
        },
        "paddle": {
            "image": "kfserving/paddleserver",
            "defaultImageVersion": "latest",
            "supportedframeworks": [
              "paddle"
            ],
            "multiModelServer": false
        }
    }
  transformers: |-
    {
    }
  explainers: |-
    {
        "alibi": {
            "image" : "kfserving/alibi-explainer",
            "defaultImageVersion": "latest"
        },
        "aix": {
            "image" : "kfserving/aix-explainer",
            "defaultImageVersion": "latest"
        },
        "art": {
            "image" : "kfserving/art-explainer",
            "defaultImageVersion": "latest"
        }
    }
  storageInitializer: |-
    {
        "image" : "gcr.io/kfserving/storage-initializer:latest",
        "memoryRequest": "100Mi",
        "memoryLimit": "1Gi",
        "cpuRequest": "100m",
        "cpuLimit": "1"
    }
  credentials: |-
    {
       "gcs": {
           "gcsCredentialFileName": "gcloud-application-credentials.json"
       },
       "s3": {
           "s3AccessKeyIDName": "AWS_ACCESS_KEY_ID",
           "s3SecretAccessKeyName": "AWS_SECRET_ACCESS_KEY"
       }
    }
  ingress: |-
    {
        "ingressGateway" : "$(ingressGateway)",
        "ingressService" : "istio-ingressgateway.istio-system.svc.cluster.local",
        "localGateway" : "knative-serving/knative-local-gateway",
        "localGatewayService" : "knative-local-gateway.istio-system.svc.cluster.local"
    }
  logger: |-
    {
        "image" : "kfserving/agent:latest",
        "memoryRequest": "100Mi",
        "memoryLimit": "1Gi",
        "cpuRequest": "100m",
        "cpuLimit": "1",
        "defaultUrl": "http://default-broker"
    }
  batcher: |-
    {
        "image" : "kfserving/agent:latest",
        "memoryRequest": "1Gi",
        "memoryLimit": "1Gi",
        "cpuRequest": "1",
        "cpuLimit": "1"
    }
  agent: |-
    {
        "image" : "kfserving/agent:latest",
        "memoryRequest": "100Mi",
        "memoryLimit": "1Gi",
        "cpuRequest": "100m",
        "cpuLimit": "1"
    }
 
 Configure Custom Domain 
  Edit the domain configuration config-map to replace example.com with your own domain, for example mydomain.com:
 kubectl edit cm config-domain --namespace knative-serving
 This command opens your default text editor and allows you to edit the config map.
 apiVersion: v1
data:
  _example: |
    ################################
    #                              #
    #    EXAMPLE ConFIGURATION     #
    #                              #
    ################################
    # ...
    example.com: |    
kind: ConfigMap
 
  Edit the file to replace example.com with the domain you’d like to use, remove the _example key and save your changes. In this example, we configure mydomain.com for all routes:
 apiVersion: v1
data:
  mydomain.com: ""
kind: ConfigMap
[...]
 
  Create a new file, config-domain.yaml and paste the following text, replacing the example.org and example.com values with the new domain you want to use:
 apiVersion: v1
kind: ConfigMap
metadata:
  name: config-domain
  namespace: knative-serving
data:
  # These are example settings of domain.
  # example.org will be used for routes having app=prod.
  example.org: |
    selector:
      app: prod    
  # Default value for domain, for routes that does not have app=prod labels.
  # Although it will match all routes, it is the least-specific rule so it
  # will only be used if no other domain matches.
  example.com: ""
 
  Apply updated domain configuration to your cluster:
 kubectl apply --filename config-domain.yaml
 
  You can map the domain to the IP address of your Knative gateway in your local machine with:
 INGRESSGATEWAY=istio-ingressgateway

export GATEWAY_IP=`kubectl get svc $INGRESSGATEWAY --namespace istio-system --output jsonpath="{.status.loadBalancer.ingress[*]['ip']}"`

# Add the record of Gateway IP and domain name into file "/etc/hosts"
echo -e "$GATEWAY_IPt$DOMAIN_NAME" | sudo tee -a /etc/hosts
  
Publish your Domain 
Follow these steps to make your domain publicly accessible: 
Set static IP for Knative Gateway 
You might want to set a static IP for your Knative gateway, so that the gateway IP does not change each time your cluster is restarted. 
Update your DNS records 
To publish your domain, you need to update your DNS provider to point to the IP address for your service ingress. 
 Create a wildcard record for the namespace and custom domain to the ingress IP Address, which would enable hostnames for multiple services in the same namespace to work without creating additional DNS entries.
 *.default.mydomain.com                   59     IN     A   35.237.28.44
 
 Create an A record to point from the fully qualified domain name to the IP address of your Knative gateway. This step needs to be done for each Knative Service or Route created.
 helloworld-go.default.mydomain.com        59     IN     A   35.237.28.44
  
If you are using Google Cloud DNS, you can find step-by-step instructions in the Cloud DNS quickstart. 
once the domain update has propagated, you can access your app using the fully qualified domain name of the deployed route, for example http://helloworld-go.default.mydomain.com 
Configure HTTPS Connection 
Determine the ingress IP and ports 
Execute the following command to determine if your kubernetes cluster is running in an environment that supports external load balancers 
$ kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)   AGE
istio-ingressgateway   LoadBalancer   172.21.109.129   130.211.10.121   ...       17h
 
If the EXTERNAL-IP value is set, your environment has an external load balancer that you can use for the ingress gateway. 
export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
 
If the EXTERNAL-IP value is none (or perpetually pending), your environment does not provide an external load balancer for the ingress gateway. In this case, you can access the gateway using the service’s node port. 
# GKE
export INGRESS_HOST=worker-node-address
# Minikube
export INGRESS_HOST=$(minikube ip)
# Other environment(On Prem)
export INGRESS_HOST=$(kubectl get po -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].status.hostIP}')

export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
 
Alternatively you can do Port Forward for testing purpose 
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80
# start another terminal
export INGRESS_HOST=localhost
export INGRESS_PORT=8080

Kfserving Deployment

数据挖掘与分析相关栏目本月热门文章