Coherence update 2022

From NURDspace
Coherence update 2022
Name Coherence update 2022
Date 2022/01/06
DateEnd 2022/01/31
Location Space
Contact Melan
Attendees


The upgrade project was successful, a big thank you to everyone who worked hard on this!

What

We are several versions behind on Proxmox on Coherence, as of writing the current version of Proxmox is 7.1-7 while we are still running 5.4-13(!). This means that we can't make any new LXC containers with newer distributions because they are simply not supported by Proxmox 5. Several attempts to upgrade Coherence have been made in the past, but I hope this time we can actually do it! It would also be nice if we could use this as an opportunity to clear old unused VMs and containers off Coherence. Done

Upgrading Proxmox path

A upgrade from 5 to 6 is needed, and from there on we can upgrade from 6 to 7. - [1] - [2] Done \o/

Upgrade stuff (can be done already when backups are possible)

Please make backups first before you upgrade a VM or container

There should be a backup system available on backup-nfs before we start.

Debian containers should be Debian 10 at least before we can upgrade Proxmox, and Ubuntu 18.04 it seems.

Thinking steps required

* Backups - if we mess up, how do we bring things back? - Need at least 4Tb to do this.

  • Redundancy - Can we upgrade some other host and migrate? Would this be a good plan going forwards?

- Probably not required - Proxmox upgrade scripts should check containers before upgrades.

  • Upgrades - All VkMs/Containers below need to be ticked off

Done

Backup

A system called `notnotapp.lan.nurd.space` has been setup and has 4x 1TB in raid 5 storage, meaning we have 3TB available for backup with redundancy. Although, roughly 1.8TB has been used for the mp3 share and the user data. Which should leave enough for VMs however, it is advised to remove your backup once the VM has been successfully upgraded to leave enough room.

A update of the user directory and mp3 share has finished at 10 January, rsync should be ran from time to time to keep notnotapp in sync with notapp. There is 966GB left free for VM backups.

Restore

Make sure you don't check unprivileged checkbox if you plan to do NFS inside the container! Screen-2022-01-30-12-30-01.png

List of LXCs/VMs

LXCs/VMs in need of OS upgrade

VMID type Name OS Done Remarks Last backup
144 VM nurdservices.nurdspace.lan Debian 9 - Critical - but this is a VM, so not blocking for proxmox upgrade Feb 24th 2022

LXCs/VMs that are upgraded or are new

Some of these have been done previously

VMID type Name OS Done Remarks Last backup
102 LXC parts.vm.nurd.space Debian 10 OK A partkeepr frontend. Has php7.1 from https://deb.sury.org/ now. Feb 23th 2022
103 LXC postgresql.vm.nurd.space Debian 10 Dennis New container with Debian 10, will need some work later (newer postgres, auth stuff) Feb 23th 2022
106 LXC harkbot Debian 11 New r3boot, replacement for NURDbot No need
108 LXC gitea.vm.nurd.space Debian 11 New Niz No need
110 LXC IPFIXer Debian 11 New Flok Feb 24th 2022
133 LXC egg.vm.nurd.space Debian 10 - This is now Debian 10, but is a bit wacky. Might be a good idea to rebuild this one from scratch someday. Feb 24th 2022
101 LXC notapp.vm.nurd.space Debian 10 - Critical - this is the main container that provides storage mounts to everywhere (egg, mpd). Update this one early. Feb 24th 2022
125 LXC ipam.vm.nurd.space Debian 10 - Petraea Feb 24th 2022
140 LXC nurdbot.vm.nurd.space Ubuntu 16.04 LTS Update to Ubuntu 20.04.3 LTS Critical Feb 24th 2022
100 LXC spacesound.vm.nurd.space Ubuntu 20.04.3 LTS OK Used to be VM 105 but now it's LXC 100 Feb 23rd 2022
107 LXC nurdbotdev.vm.nurd.space Ubuntu 20.04 LTS OK Feb 23rd 2022
109 LXC asterisk Debian 10 Previously Feb 24th 2022
113 LXC auth.vm.nurd.space / auth.nurd.space Debian 10 New Replaces saml.nurd.space - using Keycloak Feb 24th 2022
114 LXC irctvserver Debian 11 New Flok Feb 24th 2022
123 VM jarvis.vm.nurd.space Hass OS 6.6 - Feb 24th 2022
124 LXC sdr.nurd.space Ubuntu 20.04 LTS OK Feb 23rd 2022
128 LXC revproxy01.nurd.space Debian 10 Previously Feb 24th 2022
105 LXC influx.vm.nurd.space Debian 10 Previously Feb 23th 2022
112 VM portal.nurd.space Debian 10 OK CAREFULNEXTTIME: see that the http://pkg.switch.ch/switchaai/debian repo also follows along well!! Feb 24th 2022

LXCs/VMs that are not running

These can possibly be deleted or archived.

VMID type Name OS Done Remarks Last backup
104 LXC status.vm.nurd.space Debian 10 Previously Feb 23rd 2022
106 VM jarvis-ng.vm.nurd.space ? Deleted Not needed
108 VM docker.vm.nurd.space ? Deleted Not needed
110 LXC bridger.lan.nurd.space ? Deleted Petraea - An old routing box for prototyping different network layers. Could be folded. Not needed
111 LXC Flunk ? - Feb 23rd 2022
113 LXC panopticon.vm.nurd.space ? Deleted Petraea - An old monitoring setup (zabbix). Could be folded. Not needed
114 LXC vinculum.vm.nurd.space Debian 8 Deleted Petraea - An old IoT processing system. Could be folded. Not needed
115 LXC wopr.vm.nurd.space ? Deleted Gaming VM. Noone knows whose it is so fuck it :p Not used since 2015 Not needed
116 LXC music.nurd.space ? Deleted Not needed
117 LXC snor.vm.nurd.space ? - Feb 23rd 2022
118 LXC gopher Debian 10 OK Feb 23rd 2022
119 LXC gallery.nurd.space Debian 10 Previously Feb 23rd 2022
120 LXC mailng.vm.nurd.space Debian 9 Deleted Not needed
121 LXC mercator.vm.nurd.space ? - Petraea Feb 23rd 2022
122 LXC saml.nurd.space CentOS 6.6 stopped Zarya has replaced this with 113 Feb 23rd 2022
126 LXC soundfx.dhcp.nurd.space stopped Feb 23rd 2022
127 LXC mail.vm.nurd.space ? - Feb 23rd 2022
131 oldinflux.vm.nurd.space Deleted New one is 105 Not needed
135 LXC lightsjam.vm.nurd.space ? Deleted backup has been transferred to Petraeas homedir Not needed
137 LXC wol-server.vm.nurd.space Debian 7 Deleted Petraea - Some central means of coordinating WoL handling with an API. Could be folded. Not needed
141 LXC pretender.dhcp.nurd.space ? - Petraea Feb 23rd 2022
143 LXC neural.vm.nurd.space ? Deleted Petraea - an old artistic neural network processor. Could be folded. Not needed