Upgrading Ubuntu 24.04 to 26.04: Broken Python venvs, a MySQL Upgrade Blocker, and a Cleanup That Took the Database Down
Linux01 Nov 2026
1. Context & The Problem
I run a handful of small VPS servers hosting Python (Flask) applications. Each app lives in /opt/<app>, has its own virtualenv in /opt/<app>/venv, runs under a dedicated OS user, and is managed by a systemd service. One of the servers also runs MySQL. All of them were on Ubuntu 24.04 LTS (kernel 6.8, Python 3.12, MySQL 8.0), and I decided to move them to Ubuntu 26.04 LTS.
The reason was support lifetime and a newer toolchain, not speed. The kernel moves from 6.8 to 7.0, but across those releases the changes are mostly hardware support and incremental scheduler, memory, and I/O tuning. My apps spend their time waiting on the network and disk, so I did not expect a measurable speed-up, and I did not measure one.
The upgrade itself went through, but it produced four separate problems, three of them predictable and one self-inflicted:
- Python services failed with
flasknot found, andpipwas gone from the venv. - The sudo configuration printed parser warnings on every call.
- The MySQL package upgrade refused to proceed.
- A post-upgrade cleanup command stopped MySQL and left it unable to start.
This post walks through each one: the symptoms, the root cause, and the exact fix.
2. Diagnostics & Root Cause
Problem 1: Python services can't find Flask, and the venv has no pip
After the upgrade, a service whose unit points at its venv reported that Flask didn't exist. Trying to repair it in place made the problem clear:
$ sudo -u appuser /opt/my-app/venv/bin/pip install --upgrade yt-dlp flask
Traceback (most recent call last):
File "/opt/my-app/venv/bin/pip", line 5, in <module>
from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'
Root cause: a venv is not a self-contained Python. Its bin/python is a symlink to the system interpreter, and its site-packages directory is tied to that interpreter's minor version (3.12 here). A distro upgrade replaces the system Python (3.13 is the 26.04 default), so the venv ends up pointing at a missing or different interpreter that can't see the packages installed for the old one. You can confirm this in seconds:
ls -l /opt/my-app/venv/bin/python*
python3 --version
cat /opt/my-app/venv/pyvenv.cfg
If the symlink target is missing, or the version in pyvenv.cfg differs from python3 --version, the venv is stale. Repairing it in place isn't worth the effort. It has to be rebuilt.
Problem 2: sudoers warnings after the upgrade
Every sudo call started printing errors from a file shipped by the cloud provider's agent:
/etc/sudoers.d/100-oracle-cloud-agent-users:8:32: wildcards are not allowed in command arguments
Cmnd_Alias MULTIPATH_SYSTEMD = /bin/systemctl * multipathd.service, \
26.04 ships a stricter, Rust-based sudo (sudo-rs) that rejects wildcards in command arguments. I checked systemctl --failed and it listed 0 failed units, so this was noise rather than breakage. The only lost permissions were the cloud agent's passwordless restart rules for its own services. If you don't use that agent's plugins you can ignore it, otherwise replace the wildcard lines with explicit commands, always editing with visudo -f.
Problem 3: MySQL upgrade blocked by the root account
On the database server, do-release-upgrade stopped at a safety check:
Unable to upgrade MySQL
The following MySQL accounts use the mysql_native_password
authentication plugin, which is disabled by default in the version of
MySQL shipped with this release:
root@localhost
These accounts would be locked out after the upgrade.
Root cause: Ubuntu 26.04 ships MySQL 8.4, which disables mysql_native_password by default. Any account still using it would be locked out after the upgrade. The installer was protecting me, and nothing had been changed yet. I confirmed the scope with one query:
SELECT user, host, plugin FROM mysql.user WHERE plugin='mysql_native_password';
+------+-----------+-----------------------+
| user | host | plugin |
+------+-----------+-----------------------+
| root | localhost | mysql_native_password |
+------+-----------+-----------------------+
Only root@localhost was affected. The application users were already on a modern plugin.
Problem 4: my own cleanup took MySQL down
With the upgrade finished, I cleaned up old kernels and leftover packages. The next day I couldn't connect to MySQL at all:
$ mysql -u root -p
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
That error is not a password problem. It means the server isn't running. The service status said inactive (dead) and disabled, with a clean shutdown logged at 05:08. The apt history showed what had happened:
Start-Date: 2026-11-01 05:08:49
Commandline: apt purge -y keyboxd linux-image-6.8.0-86-generic ... linux-modules-6.8.0-142-generic mysql-server-8.0
Purge: linux-image-6.8.0-117-generic:amd64 (), ... mysql-server-8.0:amd64 (), ... keyboxd:amd64 ()
End-Date: 2026-11-01 05:09:00
Buried in a long list of old kernels was mysql-server-8.0, the leftover 24.04 package. Purging it stopped the running service. The new 8.4 server packages and my data were still on disk:
$ dpkg -l | grep -i mysql-server
ii mysql-server 8.4.11-0ubuntu0.26.04.1 amd64
ii mysql-server-core 8.4.11-0ubuntu0.26.04.1 amd64
$ sudo ls /var/lib/mysql
(all application database directories present)
But starting the service failed. Because the error log is written by mysqld itself, the journal was the place to look:
mysqld[11944]: [ERROR] [MY-010187] [Server] Could not open file '/var/log/mysql/error.log' for error logging: No such file or directory
mysqld[11944]: [ERROR] [MY-010119] [Server] Aborting
Root cause: the config under /etc/mysql was intact and still told MySQL to log to /var/log/mysql/error.log, but the /var/log/mysql directory had been removed along with the purged package. With no place to open its log, mysqld aborts at startup. The purge also left the service disabled, so it would not have survived a reboot either.
3. Step-by-Step Resolution
Rebuilding the venvs
The manual fix for one app is five commands. I move the old venv aside rather than deleting it, so there's something to roll back to:
sudo systemctl stop my-app
sudo mv /opt/my-app/venv /opt/my-app/venv.old
sudo apt install -y python3-venv python3-pip
sudo python3 -m venv /opt/my-app/venv
sudo chown -R appuser:appuser /opt/my-app/venv
sudo -u appuser /opt/my-app/venv/bin/python -m pip install --upgrade pip
sudo -u appuser /opt/my-app/venv/bin/python -m pip install -r /opt/my-app/requirements.txt
sudo systemctl start my-app
The python3-venv package matters. Without it, python3 -m venv on Ubuntu creates an environment with no pip. Since I have several apps and more servers to come, I wrapped this in a script driven by a small table of folder|os-user|service entries. The core is a loop:
APPS=(
"app-one|appone|app-one"
"app-two|apptwo|app-two"
)
for entry in "${APPS[@]}"; do
IFS='|' read -r dir user svc <<< "$entry"
app="/opt/$dir"
systemctl stop "$svc"
mv "$app/venv" "$app/venv.old"
python3 -m venv "$app/venv"
chown -R "$user:$(id -gn "$user")" "$app/venv"
sudo -u "$user" "$app/venv/bin/python" -m pip install --no-cache-dir --upgrade pip
sudo -u "$user" "$app/venv/bin/python" -m pip install --no-cache-dir -r "$app/requirements.txt"
systemctl start "$svc"
done
The version I actually run adds a dry-run flag, preflight checks (folder, user, service, and requirements.txt must exist before anything is touched), and an automatic rollback to venv.old if the build fails or the service isn't active a few seconds after starting. Adding an app to a new server is just one more line in the table.
Unblocking the MySQL upgrade
First, a backup, since this server holds real data:
sudo mysqldump --all-databases --routines --events --triggers > ~/mysql-all-$(date +%F).sql
Then change the root account's authentication plugin. There are two reasonable choices, depending on how root is used:
-- Option A: only you use root, via "sudo mysql" on the server (no password to forget)
ALTER USER 'root'@'localhost' IDENTIFIED WITH auth_socket;
-- Option B: scripts or tools log in as root with a password
ALTER USER 'root'@'localhost' IDENTIFIED WITH caching_sha2_password BY 'a-strong-password';
I verified the change before re-running the release upgrade, and tested a login from a second terminal while the first session was still open:
mysql -u root -p -e "SELECT user, host, plugin FROM mysql.user WHERE user='root';"
sudo do-release-upgrade
If you use the PyMySQL driver with caching_sha2_password, remember it needs the cryptography package installed.
Recovering MySQL after the bad purge
The fix was to recreate the missing log directory with the ownership MySQL expects, clear systemd's failed state, and start the service:
sudo mkdir -p /var/log/mysql
sudo chown mysql:adm /var/log/mysql
sudo chmod 750 /var/log/mysql
sudo systemctl reset-failed mysql
sudo systemctl start mysql
sudo systemctl enable mysql
I then confirmed the databases were intact with SHOW DATABASES;, read the tail of the new error log to make sure startup was clean, and ran systemctl --failed to find any application services that gave up while the database was down.
I deliberately did not run mysqld --initialize, delete anything in /var/lib/mysql, or reinstall with a purge. The data directory was untouched the whole time, and any of those would have put it at risk.
Limiting journal retention to 30 days
While cleaning up I also noticed journalctl -u showing history from the beginning of time. I capped it with a drop-in file instead of editing the main config:
sudo mkdir -p /etc/systemd/journald.conf.d
sudo nano /etc/systemd/journald.conf.d/retention.conf
[Journal]
MaxRetentionSec=30day
MaxFileSec=1day
SystemMaxUse=1G
sudo systemctl restart systemd-journald
sudo journalctl --vacuum-time=30d
journalctl --disk-usage
MaxFileSec=1day forces daily rotation, because retention is applied to whole files, and without it entries older than 30 days can linger until a large file rotates. SystemMaxUse is a size cap in case a service gets noisy.
A safer post-upgrade cleanup
This is the order I use now. Every destructive step is preceded by a simulation:
# 1. Reboot into the new kernel first
[ -f /var/run/reboot-required ] && echo "reboot needed"
uname -r
# 2. Simulate, then read the list line by line
sudo apt autoremove --purge -s
# 3. Only then do it for real, then clear the package cache
sudo apt autoremove --purge -y
sudo apt clean
# 4. Review (don't bulk-remove) packages that are no longer in the repos
apt list '?obsolete' 2>/dev/null
# 5. Look for config copies the upgrade left behind
sudo find /etc \( -name '*.dpkg-old' -o -name '*.dpkg-dist' -o -name '*.dpkg-new' \)
4. Results & Key Takeaways
Before vs. after
- OS: Ubuntu 24.04 LTS → Ubuntu 26.04 LTS (standard support now runs to April 2031).
- Kernel: 6.8 → 7.0, with the old 6.8 kernels removed.
- Python: 3.12 → 3.13 system default, with every app's venv rebuilt against it.
- MySQL: 8.0 → 8.4, with
root@localhostmoved offmysql_native_password. - Services:
systemctl --failedreports no failed units, and MySQL is enabled to start on boot. - Logs: journald now keeps 30 days and is size-capped.
- Performance: I did not measure a meaningful change, and I wouldn't expect one for I/O-bound web apps.
Key takeaways
- A venv is tied to its interpreter. Expect to rebuild every virtualenv after a distro upgrade. Keep a current
requirements.txtper app, generated while things still work withpip freeze. Using a tool likeuvto manage the Python interpreter itself, or Docker, removes this dependency on the OS Python. - Read the upgrade blockers rather than forcing past them. The MySQL check saved me from locking myself out of the database, and fixing the root account took two minutes.
- Never put a database server package in a purge list without simulating it.
apt purge -sshows exactly what will be removed. I would have noticedmysql-server-8.0in that list had I read it line by line. - Packages own more than binaries. Purging one removed a directory that a different, still-installed package depended on. A service that fails to start with "No such file or directory" is often a missing log or runtime directory, so check the journal before suspecting your data.
- "Can't connect through socket" means the server is down, not that your password is wrong. Check
systemctl statusand the journal before attempting any password reset. - Back up before touching a database server, and enable your services. A service that is
disabledworks fine until the next reboot, and then it doesn't.
None of this was dramatic, and the data was never at risk, but each issue cost time that a few minutes of simulation and checking would have saved. The rebuild script and the simulate-first cleanup order are now part of my standard upgrade routine for the remaining servers.