My goal was to create a redis-cluster with one command. I was able to accomplish this by using a kubernetes job. It wasn’t the ideal solution due to all the additional Terminated: Error pods it leaves behind, but it is however a solution.
There are two things you need to pay attention to.
- namespace: itsmetommy
- storageClassName: fast
Create namespace
You can update the namespace to whatever you’d like or delete the namespace line completely to set it to default.
kubectl create ns itsmetommy
storageClassName
I created a storageClass named fast that uses SSD. This is why you will see storageClassName: fast within the volumeClaimTemplates section. You can simply remove this line to set it back to your default settings.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-ssd
Clone git repository
https://github.com/itsmetommy/kubernetes-redis-clusterCreate
ONE COMMAND TO RULE THEM ALL!
kubectl create -f .
Example
kubectl create -f .
configmap "redis-cluster-conf" created
job.batch "redis-join" created
service "redis-service" created
statefulset.apps "redis-cluster" created
Check pods
kubectl get pods -n itsmetommy -l app=redis
Example
kubectl get pods -n itsmetommy -l app=redis
NAME READY STATUS RESTARTS AGE
redis-cluster-0 1/1 Running 0 19m
redis-cluster-1 1/1 Running 0 19m
redis-cluster-2 1/1 Running 0 19m
redis-cluster-3 1/1 Running 0 18m
redis-cluster-4 1/1 Running 0 18m
redis-cluster-5 1/1 Running 0 17m
Logs
kubectl logs redis-cluster-0 -n itsmetommy
Example
kubectl logs redis-cluster-0 -n itsmetommy
1:C 09 Jun 20:08:58.268 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 09 Jun 20:08:58.268 # Redis version=4.0.2, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 09 Jun 20:08:58.268 # Configuration loaded
1:M 09 Jun 20:08:58.269 * No cluster configuration found, I'm face3a65719530427ad09a188a70fea150d22ea1
1:M 09 Jun 20:08:58.272 * Running mode=cluster, port=6379.
1:M 09 Jun 20:08:58.272 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 09 Jun 20:08:58.272 # Server initialized
1:M 09 Jun 20:08:58.272 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 09 Jun 20:08:58.272 * Ready to accept connections
1:M 09 Jun 20:10:53.031 # configEpoch set to 1 via CLUSTER SET-CONFIG-EPOCH
1:M 09 Jun 20:10:53.043 # IP address for this node updated to 10.43.12.235
1:M 09 Jun 20:10:57.392 * Slave 10.43.11.13:6379 asks for synchronization
1:M 09 Jun 20:10:57.392 * Partial resynchronization not accepted: Replication ID mismatch (Slave asked for '2e3eac549dfc8815a6eae482e5884caec12837f9', my replication IDs are '6749228b457a1208ba105fa3130f8820860ba731' and '0000000000000000000000000000000000000000')
1:M 09 Jun 20:10:57.392 * Starting BGSAVE for SYNC with target: disk
1:M 09 Jun 20:10:57.392 * Background saving started by pid 10
10:C 09 Jun 20:10:57.395 * DB saved on disk
10:C 09 Jun 20:10:57.395 * RDB: 0 MB of memory used by copy-on-write
1:M 09 Jun 20:10:57.463 * Background saving terminated with success
1:M 09 Jun 20:10:57.463 * Synchronization with slave 10.43.11.13:6379 succeeded
1:M 09 Jun 20:10:57.964 # Cluster state changed: ok
Connect
Note: -c Enable cluster mode (follow -ASK and -MOVED redirections)
kubectl exec redis-cluster-0 -n itsmetommy -it -- redis-cli -c
Create keys
127.0.0.1:6379> set mykey1 1
OK
127.0.0.1:6379> set mykey2 2
-> Redirected to slot [14119] located at 10.43.13.244:6379
OK
10.43.13.244:6379> set mykey3 3
-> Redirected to slot [9990] located at 10.43.1.53:6379
OK
10.43.1.53:6379>
View Nodes
> cluster nodes
Example
10.43.1.53:6379> cluster nodes
5f4b8d887e5ab846fd2cd441ce444132190e60e2 10.43.11.13:6379@16379 slave face3a65719530427ad09a188a70fea150d22ea1 0 1528576197000 5 connected
834d17ef54a792cf561591c32687cfc316ac877a 10.43.10.149:6379@16379 slave e8cbe17e5e6024536c49729ddf1bd4860d966d73 0 1528576198514 4 connected
fcd20223a3c6e02993c5aad23f84d8222825d6a8 10.43.1.53:6379@16379 myself,master - 0 1528576197000 2 connected 5461-10922
face3a65719530427ad09a188a70fea150d22ea1 10.43.12.235:6379@16379 master - 0 1528576197000 1 connected 0-5460
e8cbe17e5e6024536c49729ddf1bd4860d966d73 10.43.13.244:6379@16379 master - 0 1528576198000 3 connected 10923-16383
958a6d195c24670f4d73a94972b49b29d417dc74 10.43.13.252:6379@16379 slave fcd20223a3c6e02993c5aad23f84d8222825d6a8 0 1528576198114 6 connected
10.43.1.53:6379>
View slots
> cluster slots
Example
10.43.1.53:6379> cluster slots
1) 1) (integer) 5461
2) (integer) 10922
3) 1) "10.43.1.53"
2) (integer) 6379
3) "fcd20223a3c6e02993c5aad23f84d8222825d6a8"
4) 1) "10.43.13.252"
2) (integer) 6379
3) "958a6d195c24670f4d73a94972b49b29d417dc74"
2) 1) (integer) 0
2) (integer) 5460
3) 1) "10.43.12.235"
2) (integer) 6379
3) "face3a65719530427ad09a188a70fea150d22ea1"
4) 1) "10.43.11.13"
2) (integer) 6379
3) "5f4b8d887e5ab846fd2cd441ce444132190e60e2"
3) 1) (integer) 10923
2) (integer) 16383
3) 1) "10.43.13.244"
2) (integer) 6379
3) "e8cbe17e5e6024536c49729ddf1bd4860d966d73"
4) 1) "10.43.10.149"
2) (integer) 6379
3) "834d17ef54a792cf561591c32687cfc316ac877a"
10.43.1.53:6379>
Clean up
kubectl delete -f .
Example
kubectl delete -f .
configmap "redis-cluster-conf" deleted
job.batch "redis-join" deleted
service "redis-service" deleted
statefulset.apps "redis-cluster" deleted
Delete Persistent Volumes
kubectl delete pvc -l app=redis -n itsmetommy
Example
kubectl delete pvc -l app=redis -n itsmetommy
persistentvolumeclaim "redis-data-redis-cluster-0" deleted
persistentvolumeclaim "redis-data-redis-cluster-1" deleted
persistentvolumeclaim "redis-data-redis-cluster-2" deleted
persistentvolumeclaim "redis-data-redis-cluster-3" deleted
persistentvolumeclaim "redis-data-redis-cluster-4" deleted
persistentvolumeclaim "redis-data-redis-cluster-5" deleted