testbed-automator

This project provides scripts to automate the setup and teardown of a Kubernetes cluster for deploying a 5G core network.

This repo is designed to be used with free5gc-k8s and open5gs-k8s.

Requirements

• Supported OS: Ubuntu 22.04 LTS (recommended); Ubuntu 20.04 is compatible but unmaintained.
• Minimum hardware: 4 cores, 4GB RAM, and internet access.

Quick Start

# Clone the repository
git clone https://github.com/niloysh/testbed-automator.git
cd testbed-automator

# Run the installation script
./install.sh

Note: Run scripts as a regular user (not as root).

The install.sh script should take approximately 5 mintues to run to completion. On successful deployment, you should see all pods in Running state as shown below.

Scripts Overview

Descriptions of the scripts in this repo are given below.

• install.sh : Automates the deployment of a single-node k8s cluster, configures cluster, installs various CNIs, configures ovs bridges and sets everything up for deployment of 5G core.

• uninstall.sh: Reverts changes made by install.sh, removing the Kubernetes cluster and configurations.

• cleanup.sh: Removes remants (old configuration files, CNIs) etc from previous installations of Kubernetes. Use this after uninstall.sh before running install.sh again.

• worker-join-token.sh: Run this on the master node to print out the command for adding worker nodes to the cluster.

Note: Run scripts as regular user with sudo permissions if needed. Do not use root.

Multi-node deployment

To deploy a multi-node Kubernetes cluster, start by setting up the master node using install.sh, and then add worker nodes.

1. Set up the Master Node: Run install.sh on the intended master node as shown in the Quick Start section.

2. Set up each Worker Node: On each worker node, run the install.sh script with the --worker flag as follows.

   # Clone the repository
   git clone https://github.com/niloysh/testbed-automator.git
   cd testbed-automator
   
   # Run the installation script for the worker node
   sudo ./install.sh --worker

3. Generate Join Token: On the master node, execute:

   sudo ./worker-join-token.sh

   This will output a command that includes the token and IP address for the worker node to join the cluster.

4. Join the Worker Nodes: On each worker node, run (with sudo) the join command generated by worker-join-token.sh on the master node. The command will look something like this:

   sudo kubeadm join <master-ip>:<port> --token <token> --discovery-token-ca-cert-hash <hash>

5. Verify Node Status: Back on the master node, check the status of all nodes:

   kubectl get nodes

   Each worker node should appear with the status Ready.

6. Connect every worker node to the master node using VXLAN tunnels:

   sudo python3 setup-tunnels.py

   Note: The setup-tunnels.py script is a work in progress and may not work as expected, as it makes several assumptions such as passwordless SSH being enabled. The idea is to create VXLAN tunnels between worker nodes and the master node to enable communication between them for the secondary Multus interface. If the script does not work, you can manually create VXLAN tunnels between the nodes.

Useful Resources and Commands

This project relies on several networking and Kubernetes tools that are essential for 5G deployments. Below are resources to help understand these tools better, along with some common commands.

Key Technologies

• Kubernetes: The orchestrator for containerized applications.
• Multus CNI: Enables multiple network interfaces in Kubernetes.
• OpenVSwitch (OVS): A virtual switch for network virtualization.
• OVS-CNI: A plugin that integrates OpenVSwitch with Kubernetes.

Common Kubernetes Commands

# View all nodes in the cluster
kubectl get nodes

# View all pods in the default namespace
kubectl get pods

# View detailed info about a specific pod
kubectl describe pod <pod-name>

# List services in the namespace
kubectl get svc

# Delete a pod
kubectl delete pod <pod-name>

Useful OVS Commands

# Show OVS bridges
ovs-vsctl show

# Add a new bridge
sudo ovs-vsctl add-br <bridge-name>

# Delete an OVS bridge
sudo ovs-vsctl del-br <bridge-name>

Troubleshooting

Common issues

• Check Logs: Each function logs output to help trace errors. Look for “RED” messages indicating critical failures.
• Network Requirements: Some components require network access (e.g., Helm, Kubernetes packages). Ensure the server can access the internet during the installation process.
• Resource Limits: Verify system resources meet minimum requirements. Kubernetes installations may fail with less than the recommended CPU and RAM.

For additional help, see troubleshooting.md

Contributing

Contributions are welcome! Please open an issue or submit a pull request. For larger changes, open an issue first to discuss the proposed updates.

License

This repository is licensed under the MIT License.