This is the first article in the IBM Cloud Pak for Integration (CP4I) series. It describes how to deploy OpenShift 4.3 cluster as a platform to run IBM Cloud Pak for Integration. This article does not consider many important production things and can be used for PoC, MVP or other demo cases. This article is not covering VMs deployment, therefore I consider you have your machines ran on a hypervisor with OS installed. I would recommend to create snapshots for prepared VMs in case you need to rollback to the first step.
The Architecture of the installation
The installation includes 10 machines ran in VmWare: 3 Masters based on RHCOS; 5 Workers based on RHEL7.6; 1 Bootstrap; 1 Helpernode for cluster management.
Prepare the Helpernode
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm -E %rhel).noarch.rpm
Install ansible and git and clone this repo
yum -y install ansible git
git clone https://github.com/RedHatOfficial/ocp4-helpernode
Get the Mac addresses for the cluster machines and write them down somewhere
Edit the vars.yaml file. Use the example below.
cp docs/examples/vars.yaml .
Run the main.yaml playbook
ansible-playbook -e @vars.yaml tasks/main.yml
Run the following to get info about your environment. 'haproxy' is just one of the arguments you can use with this command. For now, what we need is the address of a console which shows if the cluster is ready.
Create Ignition config files
Create an install dir
Create a place to store your pull-secret
mkdir -p ~/.openshift
Visit https://cloud.redhat.com/openshift/install and select "Bare Metal" or "vSphere", depends on what you are using. Download your pull secret and save it under ~/.openshift/pull-secret.
The playbook run before creates an sshkey for you; it's under ~/.ssh/helper_rsa. You can use this key or create/use another one if you wish.
Create install-config.yaml (I used id_rsa here)
Create installation manifests
openshift-install create manifests
If you need masters to be schedulable enable this:
Edit the manifests/cluster-scheduler-02-config.yml Kubernetes manifest file to prevent Pods from being scheduled on the control plane machines by setting mastersSchedulable to false.
sed -i 's/mastersSchedulable: true/mastersSchedulable: false/g' manifests/cluster-scheduler-02-config.yml
I didn't do this for my installation as I didn't want any pods be scheduled on my masters.
Generate the ignition configs
openshift-install create ignition-configs
Copy the ignition files in the ignition directory for the websever
cp ~/ocp4/*.ign /var/www/html/ignition/
restorecon -vR /var/www/html/
chmod o+r /var/www/html/ignition/*.ign
Launch vSphere, and boot the VMs into the boot menu; and select PXE. The vms should boot into the proper PXE profile, based on their IP address. I used Fn+F12 while RHEL 7.6 loaded to load PXE mode and chose Bootstrap and Master installation on VMs. I didn't install Workers on this step, because I planned to run them on RHEL.
Boot/install the VMs in the following order
On your laptop/workstation visit the status page for haproxy. Use this command to get the URL:
You'll see the bootstrap turn "green" and then the masters turn "green", then the bootstrap turn "red". This is your indication that you can continue.
Wait for install. The boostrap VM actually does the install for you; you can track it with the following command.
openshift-install wait-for bootstrap-complete --log-level debug
Once you see this message below...
DEBUG OpenShift Installer v4.2.0-201905212232-dirty
DEBUG Built from commit 71d8978039726046929729ad15302973e3da18ce
INFO Waiting up to 30m0s for the Kubernetes API at https://api.ocp4.example.com:6443...
INFO API v1.13.4+838b4fa up
INFO Waiting up to 30m0s for bootstrapping to complete...
DEBUG Bootstrap status: complete
INFO It is now safe to remove the bootstrap resources
...you can continue....at this point you can delete the bootstrap server.
Add workers which are on RHEL7.6 in my case
Keep in mind that you have less than 24 hours to add Worker nodes without additional steps. After that time you will have to update worker.ign because of expired certificate. You can find details on this link: https://access.redhat.com/solutions/4799921
Verify that you have access to the console. Your login will be kubeadmin and the password can be found under the file ocp4/auth/kubeadmin-password
Login to OCP
oc login -u kubeadmin
Get your console URL
oc whoami --show-console=true
Open the link in the browser and use the same credentials to login
If something went wrong and you want to start the process from the step one be sure to clean up before trying again
rm -rf /var/www/cgi-bin/ /var/www/html/ rm -rf ocp4 ocp4-helpernode/ ansible/ openshift-ansible.log install-config.yaml.orig rm -rf .openshift .openshift_install.log .ansible/ rm -rf .ssh/known_hosts .ssh/helper_rsa.pub .ssh/helper_rsa .ssh/config rm -rf /usr/local/src/openshift-install-linux.tar.gz rm -rf /var/www/html/install/bios.raw.gz rm -rf /var/lib/tftpboot/rhcos/initramfs.img rm -rf /var/lib/tftpboot/rhcos/kernel