# OpenShift 설치 ## SSH Issue [![Screenshot_20230629_173459_Notion.jpg](http://138.2.116.150/uploads/images/gallery/2023-06/scaled-1680-/screenshot-20230629-173459-notion.jpg)](http://138.2.116.150/uploads/images/gallery/2023-06/screenshot-20230629-173459-notion.jpg) ## Files for installation [pre-installation.tar](https://drive.google.com/file/d/1eT3c7k4PzdaLL0YFTeqY9fPUCfVNXbQc/view?usp=sharing) [roles](https://drive.google.com/drive/folders/1UL4SpY7r6x2DLbLTx5s_tZJxmXPRajHN?usp=sharing) ## Installation Steps: Firstly, get the packages tar file ```bash scp .\pre-installation.tar root@10.1.10.230:/root/ ``` (download the pre-installation.tar file from the notion or from the link above) Then we will get the following files ```bash ls /root 01-packages.sh 03-check-rootpermit.sh 05-ansible-directory.sh anaconda-ks.cfg 02-vim-config.sh 04-ansible-config.sh 06-ssh-key.sh pre-installation.tar ``` run all the above scripts first Then we will get ansible installed. We need to move all of the ansible files inside the new ‘ansible’ directory that we will create manually ```bash mv group_vars playbook roles ansible.cfg inventory ansible ``` We need to include the following in the /etc/hosts of the Bastion VM ```bash api.ocp.spelix2.com api-ocp-spelix2-com ``` Then we need the ocp.yaml file under, ```bash vim ansible/playbook/ocp.yaml - name: Install HAProxy hosts: localhost roles: - { role: install-haproxy, tags: ['haproxy'] } vars_files: - ../group_vars/all.yaml - name: Install DNS hosts: localhost roles: - { role: install-dns, tags: ['dns'] } vars_files: - ../group_vars/all.yaml - name: Install TFTPBOOT hosts: localhost roles: - { role: install-tftpboot, tags: ['tftp'] } vars_files: - ../group_vars/all.yaml - name: Install DHCP hosts: localhost roles: - { role: install-dhcp, tags: ['dhcp'] } vars_files: - ../group_vars/all.yaml - name: Install web server (httpd) hosts: localhost roles: - { role: install-httpd, tags: ['httpd'] } vars_files: - ../group_vars/all.yaml - name: Configuration Firewall (if you have two networks, it must be done) hosts: localhost roles: - { role: config-firewall, tags: ['firewall'] } vars_files: - ../group_vars/all.yaml - name: Download installation files hosts: localhost roles: - { role: download-files, tags: ['download'] } vars_files: - ../group_vars/all.yaml - name: Create OCP Directory hosts: localhost roles: - { role: create-ocp-directory, tags: ['ocp-dir'] } vars_files: - ../group_vars/all.yaml - name: Create OCP install-config.yaml hosts: localhost roles: - { role: create-ocp-config, tags: ['ocp-config'] } vars_files: - ../group_vars/all.yaml ``` Then we need to include all.yaml ```bash vim ansible/group_vars/all.yaml ### For 9th Cluster --- client_files_url: "" installation_files_url: "" ocp_client_files: - openshift-client-linux.tar.gz - openshift-install-linux.tar.gz rhcos_binaries: - rhcos-live-initramfs.x86_64.img - rhcos-live-kernel-x86_64 - rhcos-live-rootfs.x86_64.img ipconfig: "dhcp" ppc64le: false uefi: true disk: sda #disk where you are installing RHCOS on the masters/workers networkname: external: "ens192" internal: "ens224" ocp: root_dir: "/root" network: "OVNKubernetes" # openshift network OpenShiftSDN/OVNKubernetes sshkey: 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDUs0fFARly9xt87GwDCpYmmJW17kIedUcOsYn6cxQIklMm4o0H0C7lZA3XZhOJ/j1PXn5wpTOkEX0x7eyl0mxXNlvGneyhrXRvsy/VCzRzBAoUDihIVqsCyi6ahT0UzLRDJB7X+g3JDRpWEN2IzcWZ9yvlizq6s5S3Q0EXIh/4g2c+LXzl/mzaeYG1BvR6DSNDRSCxguPSG/+bmzZDjTozUhz5vrLtEvpAQEzejWXp45PxThUhi27Qu285i/DtGuMnEeWIUX0+yiVMIwZe0jhdVxoE1lucR4dpt/OM7dPhxo950YkYoA48bsDZVFittlLMvUIFoMkWAcJ8k8IMmfk3GqgcKoRnAf4yEmRr5+eKGXc7DOc9uvxqeaFEq8WP2e4jXuev6MqH19zq7579Tno04Ik/x2/TcJ9sPFORpRjPLum9OtZXiEFf6yoRYkjFEsLH3OnUJDBy+wQiwnYi1UuuTyBSQd/vXWQ0Bns0uXJMHGd3wrIgvGFsnX/ssqIlX+v77m/sCvxJo5W39FQBOSQ1/giAvs6KRXD3LdIG/GVWJxi+Q2OA42ylJQq2O9ZAfDENl4KkpgehRB0ahxU0GMiR09GUkki4Tw8JSDx3ft8BncIGoLbpF8mBNU1ChSx8Ukk9TW6RJWXi7nClF+tfzyBH2wWMssfndn8W66RFlxD86Q== root@bastion' secret: '{"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K29jbV9hY2Nlc3NfMGYzMmNjZjRmMjI5NGFjNWEzOTU4ZWJiZjM5Y2VjZWQ6Q0dVUjRKVTRaNUNRMUtIRVJEVlA2VDFOTFBKM1lRMFJISUdaREw5UzBENDhaMDgxUVoxQ0ZQUlhRRkpSNDlKTw==","email":"wajiwos16@gmail.com"},"quay.io":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K29jbV9hY2Nlc3NfMGYzMmNjZjRmMjI5NGFjNWEzOTU4ZWJiZjM5Y2VjZWQ6Q0dVUjRKVTRaNUNRMUtIRVJEVlA2VDFOTFBKM1lRMFJISUdaREw5UzBENDhaMDgxUVoxQ0ZQUlhRRkpSNDlKTw==","email":"wajiwos16@gmail.com"},"registry.connect.redhat.com":{"auth":"fHVoYy1wb29sLTZiN2U1YjE4LWViMDItNDhkYy1hZTY1LWU1ZDFhMGI0ZGI3MTpleUpoYkdjaU9pSlNVelV4TWlKOS5leUp6ZFdJaU9pSXdZelU0TldJMFpqQm1NekUwTjJZME9HVTVNV0U1T1RFMllUSXdOelk0WlNKOS52Q01sTnlXNVpyMjBiNXRyTm1UazNDQmZlTnZ3SERuQXh2SVNUSDJnTm1wT3pOSUppa29RUnFJMjZtcFF0QzktelFXSUVyZkV0clM0ZmgxTnBPVm5nRGEtTnltT3V0RG5FU3VUOXlJZ3B2VDRxMFFBVWhFdGdTei1aMk9oN3BXOVZzbWpxUjJwb2lJNzZhMUFjc1p2X3lHVDZaOXE5Um1HWGRpV0VFaG5WUUl2VmxvbDRxQS14aDJjOE9LMkc2LVR6VnNJSndzc09kVndjcWItX2JhRm5pV1JsS2lZNFpFQ3NCUjlJMmZHR3ZaSnBET3ZCcndlSEtuUFlwTS10WmtkTXVkeV9ZSWlxZ2ctbUg3ajR0dXhHRTdPck00bERjZk55R1BERGM4OHdZZExzMTNxS3NleVNuVmxmMjFRMFM0RDRiRWl0OUtCalZsR0p6alJ5YTFOSThLYk8za2hkdjhIT3hGLVZ1cXRocUJkV0NYcTVDYXF3ZlZ6T2ZPTnB1Zlc1VjJ0VmtwbFlFWmlhejczSjdpeEc2dHBrNnQyWWYtYmUzSjNRdHA0Q1pyX0hOVmkzZVBGRHMzX3JpbnA0UUs3LV9KWG16cHdkUGtLLWZYZ3UwSFZwZmlGM3ZYaTNmR204Q2t6MXBWUlpjR1dveTYzUmxZd210ZTAwZjlVT280RV9Tb1BpZFFReERGUDVtUWF3RDc5ZU53eDVKd1hDYWNVTXNMeFFNNGpSYjNiT0lqUXY5amo1QUNpQzJYSENRZDZKZzJhekxQWWNlWEJpRnRISEhwY2Q4Yl9zRkt0d1lZRjM4UGN6cFJFNzdodGp6NExEdGwzRFdmS0p6dnRPU0dIQ1pPdmJPLWpKOGRQWHBfUzhza25JNl93Nm1wajkxMm5GVFFKemwwY00yWQ==","email":"wajiwos16@gmail.com"},"registry.redhat.io":{"auth":"fHVoYy1wb29sLTZiN2U1YjE4LWViMDItNDhkYy1hZTY1LWU1ZDFhMGI0ZGI3MTpleUpoYkdjaU9pSlNVelV4TWlKOS5leUp6ZFdJaU9pSXdZelU0TldJMFpqQm1NekUwTjJZME9HVTVNV0U1T1RFMllUSXdOelk0WlNKOS52Q01sTnlXNVpyMjBiNXRyTm1UazNDQmZlTnZ3SERuQXh2SVNUSDJnTm1wT3pOSUppa29RUnFJMjZtcFF0QzktelFXSUVyZkV0clM0ZmgxTnBPVm5nRGEtTnltT3V0RG5FU3VUOXlJZ3B2VDRxMFFBVWhFdGdTei1aMk9oN3BXOVZzbWpxUjJwb2lJNzZhMUFjc1p2X3lHVDZaOXE5Um1HWGRpV0VFaG5WUUl2VmxvbDRxQS14aDJjOE9LMkc2LVR6VnNJSndzc09kVndjcWItX2JhRm5pV1JsS2lZNFpFQ3NCUjlJMmZHR3ZaSnBET3ZCcndlSEtuUFlwTS10WmtkTXVkeV9ZSWlxZ2ctbUg3ajR0dXhHRTdPck00bERjZk55R1BERGM4OHdZZExzMTNxS3NleVNuVmxmMjFRMFM0RDRiRWl0OUtCalZsR0p6alJ5YTFOSThLYk8za2hkdjhIT3hGLVZ1cXRocUJkV0NYcTVDYXF3ZlZ6T2ZPTnB1Zlc1VjJ0VmtwbFlFWmlhejczSjdpeEc2dHBrNnQyWWYtYmUzSjNRdHA0Q1pyX0hOVmkzZVBGRHMzX3JpbnA0UUs3LV9KWG16cHdkUGtLLWZYZ3UwSFZwZmlGM3ZYaTNmR204Q2t6MXBWUlpjR1dveTYzUmxZd210ZTAwZjlVT280RV9Tb1BpZFFReERGUDVtUWF3RDc5ZU53eDVKd1hDYWNVTXNMeFFNNGpSYjNiT0lqUXY5amo1QUNpQzJYSENRZDZKZzJhekxQWWNlWEJpRnRISEhwY2Q4Yl9zRkt0d1lZRjM4UGN6cFJFNzdodGp6NExEdGwzRFdmS0p6dnRPU0dIQ1pPdmJPLWpKOGRQWHBfUzhza25JNl93Nm1wajkxMm5GVFFKemwwY00yWQ==","email":"wajiwos16@gmail.com"}}}' # openshift pull secret file: helper: name: "bastion" #hostname for your helper node ipaddr: "192.168.228.1" #current IP address of the helper networkifacename: "ens224" #interface of the helper node,ACTUAL name of the interface, NOT the NetworkManager name dns: domain: "spelix2.com" #DNS server domain. Should match baseDomain inside the install-config.yaml file. clusterid: "ocp" #needs to match what you will for metadata.name inside the install-config.yaml file forwarder1: "192.168.228.1" #DNS forwarder forwarder2: "8.8.8.8" #second DNS forwarder lb_ipaddr: "{{ helper.ipaddr }}" #Load balancer IP, it is optional, the default value is helper.ipaddr dhcp: router: "192.168.228.1" #default gateway of the network assigned to the masters/workers bcast: "192.168.228.255" #broadcast address for your network netmask: "255.255.255.0" #netmask that gets assigned to your masters/workers poolstart: "192.168.228.200" #First address in your dhcp address pool poolend: "192.168.228.220" #Last address in your dhcp address pool ipid: "192.168.228.0" #ip network id for the range netmaskid: "255.255.255.0" #networkmask id for the range. ntp: "192.168.228.1" #ntp server address dns: "" #domain name server, it is optional, the default value is set to helper.ipaddr bootstrap: name: "bootstrap" #hostname (WITHOUT the fqdn) of the bootstrap node ipaddr: "192.168.228.200" #IP address that you want set for bootstrap node macaddr: "00:50:56:bd:e3:eb" #The mac address for dhcp reservation masters: - name: "master01" #hostname (WITHOUT the fqdn) of the master node (x of 3) ipaddr: "192.168.228.201" #The IP address (x of 3) that you want set macaddr: "00:50:56:bd:bb:e6" #The mac address for dhcp reservation - name: "master02" ipaddr: "192.168.228.202" macaddr: "00:50:56:bd:9b:7b" - name: "master03" ipaddr: "192.168.228.203" macaddr: "00:50:56:bd:ee:19" workers: - name: "worker01" #hostname (WITHOUT the fqdn) of the worker node you want to set ipaddr: "192.168.228.211" #The IP address that you want set (1st node) macaddr: "00:50:56:bd:23:b7" #The mac address for dhcp reservation (1st node) - name: "worker02" ipaddr: "192.168.228.212" macaddr: "00:50:56:bd:f7:9c" - name: "worker03" ipaddr: "192.168.228.213" macaddr: "00:50:56:bd:77:69" ``` Recheck all the values (IP address, MAC, pull secret, sshkey, domain name and then continue) Then we need all of the installation files (download from the above link) ```bash # From Windows Powershell we have a 'roles' folder with all of the installation files scp -r .\\roles\\ root@10.1.10.230:/root/ansible/ root@10.1.10.230's password: main.yaml 100% 1874 328.4KB/s 00:00 main.yaml 100% 169 33.7KB/s 00:00 install-config.yaml.j2 100% 583 114.0KB/s 00:00 main.yaml 100% 155 15.2KB/s 00:00 main.yaml 100% 1793 350.2KB/s 00:00 default.j2 100% 1484 159.1KB/s 00:00 main.yaml 100% 285 28.3KB/s 00:00 dhcpd.conf.j2 100% 1894 189.7KB/s 00:00 main.yaml 100% 1397 133.5KB/s 00:00 dns.tar 100% 11KB 734.8KB/s 00:00 named.conf.j2 100% 1728 168.9KB/s 00:00 named.rfc1912.zones.j2 100% 1856 378.1KB/s 00:00 ocp.zones.j2 100% 2507 240.8KB/s 00:00 reverse.rev.j2 100% 1372 134.0KB/s 00:00 main.yaml 100% 274 26.8KB/s 00:00 haproxy.cfg.j2 100% 4691 466.7KB/s 00:00 main.yaml 100% 280 54.7KB/s 00:00 httpd.conf.j2 100% 12KB 783.3KB/s 00:00 ldlinux.c32 100% 113KB 5.5MB/s 00:00 libutil.c32 100% 22KB 2.2MB/s 00:00 menu.c32 100% 26KB 2.5MB/s 00:00 pxelinux.0 100% 41KB 2.7MB/s 00:00 tftp.tar 100% 207KB 10.1MB/s 00:00 main.yaml 100% 564 53.7KB/s 00:00 ``` Then we need to add the localhost as below ```bash vim inventory/ocp [localhost] 127.0.0.1 ``` Then inside the ansible directory, we need to use the following to install and check all of the yamls ```bash # To list and see the 순서of installation ansible-playbook -i inventory/ocp playbook/ocp.yaml --list-tags # Installing ansible-playbook -i inventory/ocp playbook/ocp.yaml --tags ## one command install ansible-playbook -i inventory/ocp playbook/ocp.yaml ``` After doing all ansible-playbook commands, we need to rename the openshift installer yaml file like the following: ```bash mv /root/openshift/config/install-cnfig.yaml /root/openshift/config/install-config.yaml ``` The install-config.yaml file looks like this: ```bash apiVersion: v1 baseDomain: spelix2.com compute: - hyperthreading: Enabled name: worker replicas: 0 controlPlane: hyperthreading: Enabled name: master replicas: 3 metadata: name: ocp networking: clusterNetworks: - cidr: 10.128.0.0/14 hostPrefix: 23 networkType: OVNKubernetes serviceNetwork: - 172.30.0.0/16 platform: none: {} fips: false pullSecret: '{"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K29jbV9hY2Nlc3NfMGYzMmNjZjRmMjI5NGFjNWEzOTU4ZWJiZjM5Y2VjZWQ6Q0dVUjRKVTRaNUNRMUtIRVJEVlA2VDFOTFBKM1lRMFJISUdaREw5UzBENDhaMDgxUVoxQ0ZQUlhRRkpSNDlKTw==","email":"wajiwos16@gmail.com"},"quay.io":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K29jbV9hY2Nlc3NfMGYzMmNjZjRmMjI5NGFjNWEzOTU4ZWJiZjM5Y2VjZWQ6Q0dVUjRKVTRaNUNRMUtIRVJEVlA2VDFOTFBKM1lRMFJISUdaREw5UzBENDhaMDgxUVoxQ0ZQUlhRRkpSNDlKTw==","email":"wajiwos16@gmail.com"},"registry.connect.redhat.com":{"auth":"fHVoYy1wb29sLTZiN2U1YjE4LWViMDItNDhkYy1hZTY1LWU1ZDFhMGI0ZGI3MTpleUpoYkdjaU9pSlNVelV4TWlKOS5leUp6ZFdJaU9pSXdZelU0TldJMFpqQm1NekUwTjJZME9HVTVNV0U1T1RFMllUSXdOelk0WlNKOS52Q01sTnlXNVpyMjBiNXRyTm1UazNDQmZlTnZ3SERuQXh2SVNUSDJnTm1wT3pOSUppa29RUnFJMjZtcFF0QzktelFXSUVyZkV0clM0ZmgxTnBPVm5nRGEtTnltT3V0RG5FU3VUOXlJZ3B2VDRxMFFBVWhFdGdTei1aMk9oN3BXOVZzbWpxUjJwb2lJNzZhMUFjc1p2X3lHVDZaOXE5Um1HWGRpV0VFaG5WUUl2VmxvbDRxQS14aDJjOE9LMkc2LVR6VnNJSndzc09kVndjcWItX2JhRm5pV1JsS2lZNFpFQ3NCUjlJMmZHR3ZaSnBET3ZCcndlSEtuUFlwTS10WmtkTXVkeV9ZSWlxZ2ctbUg3ajR0dXhHRTdPck00bERjZk55R1BERGM4OHdZZExzMTNxS3NleVNuVmxmMjFRMFM0RDRiRWl0OUtCalZsR0p6alJ5YTFOSThLYk8za2hkdjhIT3hGLVZ1cXRocUJkV0NYcTVDYXF3ZlZ6T2ZPTnB1Zlc1VjJ0VmtwbFlFWmlhejczSjdpeEc2dHBrNnQyWWYtYmUzSjNRdHA0Q1pyX0hOVmkzZVBGRHMzX3JpbnA0UUs3LV9KWG16cHdkUGtLLWZYZ3UwSFZwZmlGM3ZYaTNmR204Q2t6MXBWUlpjR1dveTYzUmxZd210ZTAwZjlVT280RV9Tb1BpZFFReERGUDVtUWF3RDc5ZU53eDVKd1hDYWNVTXNMeFFNNGpSYjNiT0lqUXY5amo1QUNpQzJYSENRZDZKZzJhekxQWWNlWEJpRnRISEhwY2Q4Yl9zRkt0d1lZRjM4UGN6cFJFNzdodGp6NExEdGwzRFdmS0p6dnRPU0dIQ1pPdmJPLWpKOGRQWHBfUzhza25JNl93Nm1wajkxMm5GVFFKemwwY00yWQ==","email":"wajiwos16@gmail.com"},"registry.redhat.io":{"auth":"fHVoYy1wb29sLTZiN2U1YjE4LWViMDItNDhkYy1hZTY1LWU1ZDFhMGI0ZGI3MTpleUpoYkdjaU9pSlNVelV4TWlKOS5leUp6ZFdJaU9pSXdZelU0TldJMFpqQm1NekUwTjJZME9HVTVNV0U1T1RFMllUSXdOelk0WlNKOS52Q01sTnlXNVpyMjBiNXRyTm1UazNDQmZlTnZ3SERuQXh2SVNUSDJnTm1wT3pOSUppa29RUnFJMjZtcFF0QzktelFXSUVyZkV0clM0ZmgxTnBPVm5nRGEtTnltT3V0RG5FU3VUOXlJZ3B2VDRxMFFBVWhFdGdTei1aMk9oN3BXOVZzbWpxUjJwb2lJNzZhMUFjc1p2X3lHVDZaOXE5Um1HWGRpV0VFaG5WUUl2VmxvbDRxQS14aDJjOE9LMkc2LVR6VnNJSndzc09kVndjcWItX2JhRm5pV1JsS2lZNFpFQ3NCUjlJMmZHR3ZaSnBET3ZCcndlSEtuUFlwTS10WmtkTXVkeV9ZSWlxZ2ctbUg3ajR0dXhHRTdPck00bERjZk55R1BERGM4OHdZZExzMTNxS3NleVNuVmxmMjFRMFM0RDRiRWl0OUtCalZsR0p6alJ5YTFOSThLYk8za2hkdjhIT3hGLVZ1cXRocUJkV0NYcTVDYXF3ZlZ6T2ZPTnB1Zlc1VjJ0VmtwbFlFWmlhejczSjdpeEc2dHBrNnQyWWYtYmUzSjNRdHA0Q1pyX0hOVmkzZVBGRHMzX3JpbnA0UUs3LV9KWG16cHdkUGtLLWZYZ3UwSFZwZmlGM3ZYaTNmR204Q2t6MXBWUlpjR1dveTYzUmxZd210ZTAwZjlVT280RV9Tb1BpZFFReERGUDVtUWF3RDc5ZU53eDVKd1hDYWNVTXNMeFFNNGpSYjNiT0lqUXY5amo1QUNpQzJYSENRZDZKZzJhekxQWWNlWEJpRnRISEhwY2Q4Yl9zRkt0d1lZRjM4UGN6cFJFNzdodGp6NExEdGwzRFdmS0p6dnRPU0dIQ1pPdmJPLWpKOGRQWHBfUzhza25JNl93Nm1wajkxMm5GVFFKemwwY00yWQ==","email":"wajiwos16@gmail.com"}}}' sshKey: 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDMkx7b+ZEpfQhWs0SrCeUXiR8d00ST+f43tEpaLtNVOSE+GwlwmbuaOPqOBM4Efjw8BxiZQ2JX506t6W7zDZ4UK2XbpkjnLbYJxAy4gPK7/ZVr6MI6Fn7zwhOFC2sB31bb9RtfYcNeXQ4tCHdNy/DZsIrE/rEH3CrCzYiQspKHYL5ZvNiQo/9eu7cJ/MeZTB+TvkFV9m/GabQtr72q2FdnLdic0B3a4tM42M2WVbMUlP5zFyr8SeDLrTOOP6nygqlViqfvF8KUIJkNNqwvIkchaepZ82FVnnrpM81o3n3UohhRRnuHe/LAOiGbQDrXgxWGRan9xdS3i0IV4hwtxKoc/d+a68AgyBQ/j5jANUWtQ2z5cls4nJo4g7d+h4UZ+MOVQvrN+QHJMq2Nb1QmfPi8OAaFIfUldFwcgoDeZ8A0UllUVmYE1jcehbCSS9VqCX5wr9jT8oVzf8E7iN2WEbh/nwxX4OGjYQwZo+iXegE7HG44WCaFtYugz5MQjVcxvya4X3kpUOztK89GK5R4LYO9yekBjQcopcSUS6MEj6zbZcb5+yiO/ovelZTCdi/W2THsvQHmeibhb1W896oSTLG8xDbKP7z/V4LJTvBd1DuQ17VO2UaFcUIZirmggTvV0c+HQtDZdKWujaIk94esirm2MCJQykPATXxvjiLSG8rfYw== root@bastion' ``` Then ```bash cd /root/openshift/config [root@bastion config]# ls install-config.yaml [root@bastion config]# openshift-install create manifests INFO Consuming Install Config from target directory WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings INFO Manifests created in: manifests and openshift [root@bastion config]# ls manifests openshift ``` Then we need to install ignition files ```bash openshift-install create ignition-configs INFO Consuming Master Machines from target directory INFO Consuming Openshift Manifests from target directory INFO Consuming OpenShift Install (Manifests) from target directory INFO Consuming Worker Machines from target directory INFO Consuming Common Manifests from target directory INFO Ignition-Configs created in: . and auth [root@bastion config]# ls auth bootstrap.ign master.ign metadata.json worker.ign ``` Then we need to move the .ign files to /var/www/html/ign, ```bash ls auth bootstrap.ign master.ign metadata.json worker.ign [root@bastion config]# mv /root/openshift/config/worker.ign /var/www/html/ign/ [root@bastion config]# mv /root/openshift/config/master.ign /var/www/html/ign/ [root@bastion config]# mv /root/openshift/config/bootstrap.ign /var/www/html/ign/ [root@bastion config]# ls auth metadata.json [root@bastion config]# ls /var/www/html/ign/ bootstrap.ign master.ign worker.ign ``` Then give apache ownership ```bash chown -R apache:apache /var/www/html/* ``` After all settings on Bastion is done, we can turn on the Bootstrap node and install it [![Screenshot_20230629_173643_Notion.jpg](http://138.2.116.150/uploads/images/gallery/2023-06/scaled-1680-/screenshot-20230629-173643-notion.jpg)](http://138.2.116.150/uploads/images/gallery/2023-06/screenshot-20230629-173643-notion.jpg) Then we need to check if we can ssh to bootstrap node from our bastion ```bash ssh core@bootstrap.ocp.spelix.com [core@bootstrap ~]$ ``` Then we need to turn on our 3 master nodes one by one and select the ‘Install Master’ option The same goes for the 3 worker nodes ‘Install Node’ option to be selected Then we need to export the KUBECONFIG file to use the ‘oc’ command ```bash export KUBECONFIG=/root/openshift/config/auth/kubeconfig source /root/.bashrc ## or add the above line inside the .bashrc file vim /root/.bashrc ``` Then approve the certificates ```bash oc get csr -o name | xargs oc adm certificate approve ``` We can check the approve status using the following command ```bash oc get csr ``` the values should be ‘Approved’ instead of ‘Pending’ so we can use the approve certificates command again if we see any pending

keep checking the certs using the 'oc get csr' command, as it will keep updating new certificates that will be in 'pending' state. We need to approve them as well.

Then if we see the nodes ```bash oc get nodes NAME STATUS ROLES AGE VERSION master01.ocp.spelix.com Ready master,worker 12m v1.23.5+8471591 master02.ocp.spelix.com Ready master,worker 12m v1.23.5+8471591 master03.ocp.spelix.com Ready master,worker 12m v1.23.5+8471591 worker01.ocp.spelix.com Ready worker 78s v1.23.5+8471591 worker02.ocp.spelix.com Ready worker 77s v1.23.5+8471591 worker03.ocp.spelix.com Ready worker 88s v1.23.5+8471591 ``` Then we need to add the bastion IP and the following hostnames to the windows host file ```bash api.ocp.cpf.com console-openshift-console.apps.ocp.cpf.com oauth-openshift.apps.ocp.cpf.com downloads-openshift-console.apps.ocp.cpf.com alertmanager-main-openshift-monitoring.apps.ocp.cpf.com grafana-openshift-monitoring.apps.ocp.cpf.com prometheus-k8s-openshift-monitoring.apps.ocp.cpf.com thanos-querier-openshift-monitoring.apps.ocp.cpf.com ``` Then we can change the master nodes to be 'not schedulable' ```yaml oc patch schedulers.config.openshift.io/cluster --type merge -p '{"spec":{"mastersSchedulable":false}}' ``` The above command should change the status of master nodes to be only master nodes ```bash oc get nodes NAME STATUS ROLES AGE VERSION master01.ocp.spelix2.com Ready master 6h39m v1.23.5+8471591 master02.ocp.spelix2.com Ready master 6h39m v1.23.5+8471591 master03.ocp.spelix2.com Ready master 6h39m v1.23.5+8471591 worker01.ocp.spelix2.com Ready worker 6h27m v1.23.5+8471591 worker02.ocp.spelix2.com Ready worker 6h28m v1.23.5+8471591 worker03.ocp.spelix2.com Ready worker 6h27m v1.23.5+8471591 ``` After all the above steps, we need to follow the notion page from the Openshift 교육 '설치 후 작업' part as it is Just remember to apply all of the nfs yaml files while doing the nfs part deployment.yaml rbac.yaml class.yaml test-pod.yaml test-pvc.yaml