I'm installing packages via apt-get on a cluster on Databricks on Azure. Every time I spin up the cluster I need to reinstall the packages. (I specify Databricks only because it's relevant to mention that this is not happening on the same physical machine each time – and thus my question about how I can check what could be changing between runs.)
Sometimes a particular package successfully installs; other times it cannot be found.
Questions (Three attempts to get to the same place)
- What could be changing between runs that could be causing this, and how can I check them?
- What would cause
apt-get updateto not specify the most recent version? - What would prevent
apt-get installfrom finding the most recent version, when the version is specified?
Context
My initial problem is that I'm trying to install gdal:
sudo apt-get update -y
sudo apt-get install gdal -y
Err:1 http://security.ubuntu.com/ubuntu focal-updates/main amd64 libmysqlclient21 amd64 8.0.28-0ubuntu0.20.04.3
# 404 Not Found [IP: 91.189.91.39 80]
# ...
# Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-8.0/libmysqlclient21_8.0.28-0ubuntu0.20.04.3_amd64.deb 404 Not Found [IP: 91.189.91.39 80]
To see what's going on, I tried installing libmysqlclient21 alone and looking at what apt-get is trying to install when it installs libmysqlclient21.
sudo apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists...
sudo apt-get install -y libmysqlclient21
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
libmysqlclient21
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,292 kB of archives.
After this operation, 7,652 kB of additional disk space will be used.
Ign:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmysqlclient21 amd64 8.0.28-0ubuntu0.20.04.3
Err:1 http://security.ubuntu.com/ubuntu focal-updates/main amd64 libmysqlclient21 amd64 8.0.28-0ubuntu0.20.04.3
404 Not Found [IP: 91.189.91.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-8.0/libmysqlclient21_8.0.28-0ubuntu0.20.04.3_amd64.deb 404 Not Found [IP: 91.189.91.39 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
(Running apt-get update with --fix-missing has not changed the output.)
If the install is going to succeed, the version is 8.0.29-0ubuntu0.20.04.3
apt-cache policy libmysqlclient21
libmysqlclient21:
Installed: (none)
Candidate: 8.0.29-0ubuntu0.20.04.3
Version table:
8.0.29-0ubuntu0.20.04.3 500
500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
8.0.19-0ubuntu5 500
500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
When it fails, it is 8.0.28-0ubuntu0.20.04.3
apt-cache policy libmysqlclient21
libmysqlclient21:
Installed: (none)
Candidate: 8.0.28-0ubuntu0.20.04.3
Version table:
8.0.28-0ubuntu0.20.04.3 500
500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
8.0.19-0ubuntu5 500
500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
If I go the Ubuntu repository (I think that's what this is?), I find that the 8.0.29 version exists but the 8.0.28 version does not. (security.ubuntu.com/ubuntu/pool/main/m/mysql-8.0/)
Occasionally when I specify to install the 8.0.29 version, even when the candidate version is 8.0.28, the install will succeed but not typically:
sudo apt-get update -y
sudo apt-get install -y libmysqlclient21=8.0.29-0ubuntu0.20.04.3
# E: Version '8.0.29-0ubuntu0.20.04.3' for 'libmysqlclient21' was not found
One time, using apt-get clean before updating and installing also worked. I've not been able to recreate this yet.
sudo apt-get clean
sudo apt-get update -y
sudo apt-get install libmysqlclient21
Databricks-specific, of questionable site relevance
Finally, I am running this on both an interactive Databricks cluster and on a jobs Databricks cluster. All of the successes have been on the interactive cluster; the jobs cluster has always failed to install the package. I only include this in the off-chance it adds to understanding.
Other SE questions
I've read a number of other stackexchange questions on apt-get, including these:
- inconsistent
apt-get updatebehaviour on official Ubuntu AWS AMI; I feel this may be close to the issue. The particular solution doesn't work for me and I don't know what the equivalent of/var/lib/cloud/instance/boot-finishedwould be for me. - Why apt-get doesn't install the newest software; My issue, though, is not that I'm not getting the newest version, but that I can only inconsistently get the version hosted by Ubuntu
- apt-get not getting latest version on some servers
- An answer I can no longer find that says
focal-updatesandfocal-securitydon't offer the full package, only updates to the package and that I'd need to find the package elsewhere. That said, this doesn't change the fact that I've had inconsistent success. Why would apt-get sometimes know the correct origin, and other times not?
Edit
I do not believe this is a duplicate of Why don't the Ubuntu repositories have the latest versions of software? because the more recent version that I'm specifying is in the ubuntu repository.
Also, I should emphasize that I am not committed to any particular package version. I simply need to install gdal, and its failing because apt-get cannot find libmysqlclient21.
When I go to the ubuntu repository, I see that a version 8.0.29-0ubuntu0.20.04.3 exists, but there is no version 8.0.28-0ubuntu0.20.04.3. I also notice that when the candidate version under apt-cache policy libmysqlclient21 is 8.0.29… it successfully installs, and when it is 8.0.28…, it does not.
