Browse Source

init commits

iwita 5 years ago
parent
commit
ca6a8e7bf1
4 changed files with 184 additions and 6 deletions
  1. 36 6
      README.md
  2. 125 0
      new_scheduler/add_new_scheduler.md
  3. 15 0
      new_scheduler/create_scheduler.sh
  4. 8 0
      new_scheduler/update_scheduler.sh

+ 36 - 6
README.md

@@ -2,29 +2,29 @@
 
 ## 1. Create your custom plugin
 There is a list of stages during the scheduling procedure you can interve as it also stated in [scheduling framework- v1.17](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/):
-#### Queue sort[](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#queue-sort)
+#### [Queue sort](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#queue-sort)
 
 These plugins are used to sort Pods in the scheduling queue. A queue sort plugin essentially will provide a “less(Pod1, Pod2)” function. Only one queue sort plugin may be enabled at a time.
 
-#### Pre-filter[](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#pre-filter)
+#### [Pre-filter](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#pre-filter)
 
 These plugins are used to pre-process info about the Pod, or to check certain conditions that the cluster or the Pod must meet. If a pre-filter plugin returns an error, the scheduling cycle is aborted.
 
-#### Filter[](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#filter)
+#### [Filter](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#filter)
 
 These plugins are used to filter out nodes that cannot run the Pod. For each node, the scheduler will call filter plugins in their configured order. If any filter plugin marks the node as infeasible, the remaining plugins will not be called for that node. Nodes may be evaluated concurrently.
 
-#### Post-filter[](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#post-filter)
+#### [Post-filter](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#post-filter)
 
 This is an informational extension point. Plugins will be called with a list of nodes that passed the filtering phase. A plugin may use this data to update internal state or to generate logs/metrics.
 
 **Note:**  Plugins wishing to perform “pre-scoring” work should use the post-filter extension point.
 
-#### Scoring[](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#scoring)
+#### [Scoring](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#scoring)
 
 These plugins are used to rank nodes that have passed the filtering phase. The scheduler will call each scoring plugin for each node. There will be a well defined range of integers representing the minimum and maximum scores. After the  [normalize scoring](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#normalize-scoring)  phase, the scheduler will combine node scores from all plugins according to the configured plugin weights.
 
-#### Normalize scoring[](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#normalize-scoring)
+#### [Normalize scoring](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#normalize-scoring)
 
 These plugins are used to modify scores before the scheduler computes a final ranking of Nodes. A plugin that registers for this extension point will be called with the  [scoring](https://kubernetes.io/docs/concepts/configuration/scheduling-framework/#scoring)  results from the same plugin. This is called once per plugin per scheduling cycle.
 
@@ -72,3 +72,33 @@ func  NewInTreeRegistry() framework.Registry {
 	}
 }
 ```
+
+## 3. Enable the plugin
+You can enable or disable plugins using the configuration yaml file on runtime. However, here we will show the hard-coded way. 
+
+By editing the `pkg/scheduler/algorithmprovider/registry.go` file. 
+
+```go
+func  getDefaultConfig() *schedulerapi.Plugins {
+
+	return &schedulerapi.Plugins{
+
+...
+
+		Score: &schedulerapi.PluginSet{
+
+			Enabled: []schedulerapi.Plugin{
+
+				{Name: noderesources.CustomAllocatedName, Weight: 100000},
+
+				{Name: imagelocality.Name, Weight: 1},
+
+				...
+			},
+
+		},
+...
+	}
+
+}
+```

+ 125 - 0
new_scheduler/add_new_scheduler.md

@@ -0,0 +1,125 @@
+## Deploy the customized scheduler
+
+Run:
+```bash
+./create_scheduler
+```
+
+Create a yaml file:
+
+```yaml
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: evolve-scheduler
+  namespace: kube-system
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: evolve-scheduler-as-kube-scheduler
+subjects:
+- kind: ServiceAccount
+  name: evolve-scheduler
+  namespace: kube-system
+roleRef:
+  kind: ClusterRole
+  name: system:kube-scheduler
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: evolve-scheduler
+  labels:
+    component: scheduler
+    tier: control-plane
+  name: evolve-scheduler
+  namespace: kube-system
+spec:
+  selector:
+    matchLabels:
+      component: scheduler
+      tier: control-plane
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        component: scheduler
+        tier: control-plane
+        version: second
+    spec:
+      serviceAccountName: evolve-scheduler
+      containers:
+      - command:
+        - kube-scheduler
+        - --authentication-kubeconfig=/etc/kubernetes/scheduler.conf
+        - --authorization-kubeconfig=/etc/kubernetes/scheduler.conf
+        - --bind-address=0.0.0.0
+        - --kubeconfig=/etc/kubernetes/scheduler.conf 
+        - --leader-elect=false
+        - --scheduler-name=evolve-scheduler
+#        - --config=/etc/kubernetes/config/kube-scheduler-conf.yaml
+        image: docker.io/iwita/scheduler:latest
+        imagePullPolicy: Always
+        livenessProbe:
+          httpGet:
+            path: /healthz
+            port: 10251
+          initialDelaySeconds: 15
+        name: evolve-scheduler
+        readinessProbe:
+          httpGet:
+            path: /healthz
+            port: 10251
+        resources:
+          requests:
+            cpu: '0.1'
+        securityContext:
+          privileged: false
+        volumeMounts:
+        - mountPath: /etc/kubernetes/
+          name: kubeconfig
+          readOnly: true
+        env:
+          - name: NODE_NAME
+            valueFrom:
+              fieldRef:
+                fieldPath: spec.nodeName
+      nodeSelector:
+        kubernetes.io/hostname: kube-00
+      hostNetwork: false
+      hostPID: false
+      volumes: 
+      - hostPath:
+          path: /etc/kubernetes/
+        name: kubeconfig
+
+```
+
+Update 
+
+```bash
+./update_scheduler.sh custom-scheduler.yaml
+```
+
+Verify
+```bash
+kubectl get pods -n kube-system
+```
+
+Finally add the following lines 
+
+- Open the config file for the cluster role `kubectl edit clusterrole system:kube-scheduler`
+- Append the following lines in the end
+
+```yaml
+- apiGroups:
+  - storage.k8s.io
+  resources:
+  - storageclasses
+  verbs:
+  - watch
+  - list
+  - get
+  ```

+ 15 - 0
new_scheduler/create_scheduler.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+KUBERNETES_PATH="kubernetes-v1.17/"
+DOCKERFILE_PATH="${KUBERNETES_PATH}/_output/bin/"
+
+echo $KUBERNETES_PATH
+echo $DOCKERFILE_PATH
+
+cd $KUBERNETES_PATH
+sudo make all WHAT=cmd/kube-scheduler/
+cd $DOCKERFILE_PATH
+sudo docker build -t iwita/scheduler:latest .
+sudo docker push iwita/scheduler:latest
+cd
+echo "END"

+ 8 - 0
new_scheduler/update_scheduler.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+
+SCHEDULER_FILE=$1
+kubectl delete -f ${SCHEDULER_FILE}
+kubectl taint nodes kube-00 node-role.kubernetes.io/master:NoSchedule-
+kubectl create -f ${SCHEDULER_FILE}
+kubectl taint nodes kube-00 node-role.kubernetes.io/master=:NoSchedule
+