pull-debian-source with apt-get or "how to download a Debian package from Ubuntu the hard but fun way"

Have you ever wanted to download a Debian package from an Ubuntu system? Well you are not alone: hidden away in the wonderful "ubuntu-dev-tools" package is the tool for you - the excellent pull-debian-source, written by tumbleweed. This is a really useful tool which is also fast! 

$ sudo apt-get install -y ubuntu-dev-tools
$ pull-debian-source hello
pull-debian-source: Downloading hello version 2.8-1
pull-debian-source: Downloading hello_2.8.orig.tar.gz from ftp.debian.org (0.665 MiB)
pull-debian-source: Downloading hello_2.8-1.debian.tar.gz from ftp.debian.org (0.006 MiB)
dpkg-source: info: extracting hello in hello-2.8                   
dpkg-source: info: unpacking hello_2.8.orig.tar.gz
dpkg-source: info: unpacking hello_2.8-1.debian.tar.gz
Or, to download the at package from the sid release: 
$ pull-debian-source at sid
pull-debian-source: Downloading at version 3.1.13-1
pull-debian-source: Downloading at_3.1.13.orig.tar.gz from ftp.debian.org (0.117 MiB)
pull-debian-source: Downloading at_3.1.13-1.diff.gz from ftp.debian.org (0.009 MiB)
dpkg-source: info: extracting at in at-3.1.13                      
dpkg-source: info: unpacking at_3.1.13.orig.tar.gz
dpkg-source: info: applying at_3.1.13-1.diff.gz
$
However, I had cause to pore over the apt.conf(5) man page recently, and wondered if it would be possible to do the same thing, but using only apt-get(8). Turns out it is...

The first thing to do is create a few directories: 
$ dir=~/.cache/pull-debian-source.sh
$ mkdir -p $dir
$ for d in etc/trusted.gpg.d etc/preferences.d var log; do mkdir -p $dir/$d; done
Now, we need a sources.list file so apt-get knows where to download index and package files from. I'm creating one for Debian sid: 
$ export conf=$dir/etc/sources.list
$ cat >>$conf<<EOT
deb http://ftp.uk.debian.org/debian/ sid main
deb-src http://ftp.uk.debian.org/debian/ sid main
deb http://ftp.uk.debian.org/debian/ experimental main
deb-src http://ftp.uk.debian.org/debian/ experimental main
EOT
Next, we need to grab a copy of the package cache containing the index files. This is where the magic comes in as we use some of the amazing array of apt options: 
$ export options="\
  -o Dir=\"$dir\" \
  -o Dir::Etc=\"$dir/etc\" \
  -o Dir::State=\"$dir/var\" \
  -o Dir::Cache=\"$dir/var\" \
  -o Dir::Log=\"$dir/log\" \
  -o APT::Get::AllowUnauthenticated=yes \
  -o Debug::NoLocking=true"
As you can see, we've specified alternative locations for the directories apt cares about. We've also turned off locking and authentication. The locking option isn't a problem unless you happen to be running multiple copies of apt-get as yourself. The authentication option needs might be a problem for you - check the man page for details. Now we've set the options, lets create the cache: 
$ eval "apt-get $options update"
Get:1 http://ftp.uk.debian.org sid InRelease [234 kB]
Get:2 http://ftp.uk.debian.org experimental InRelease [162 kB]
Ign http://ftp.uk.debian.org sid InRelease                    
Ign http://ftp.uk.debian.org experimental InRelease           
Get:3 http://ftp.uk.debian.org sid/main Sources/DiffIndex [7876 B]
Get:4 http://ftp.uk.debian.org sid/main i386 Packages/DiffIndex [7876 B]
Hit http://ftp.uk.debian.org sid/main TranslationIndex
Get:5 http://ftp.uk.debian.org experimental/main Sources/DiffIndex [7819 B]
Get:6 http://ftp.uk.debian.org experimental/main i386 Packages/DiffIndex [7819 B]
Hit http://ftp.uk.debian.org experimental/main TranslationIndex
Hit http://ftp.uk.debian.org sid/main Translation-en
Hit http://ftp.uk.debian.org experimental/main Translation-en
Fetched 407 kB in 2s (194 kB/s)
Reading package lists... Done
W: GPG error: http://ftp.uk.debian.org sid InRelease: No keyring installed in /home/james/.cache/pull-debian-source.sh/etc/trusted.gpg.d/.
W: GPG error: http://ftp.uk.debian.org experimental InRelease: No keyring installed in /home/james/.cache/pull-debian-source.sh/etc/trusted.gpg.d/.
$
We got a couple of warnings ("W:" entries above), but they are not fatal. Now we've got a local cache of package meta-data, let's download a package from Debian: 
$ eval "apt-get $options source cron"
Reading package lists... Done
Building dependency tree... Done
NOTICE: 'cron' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/git/pkg-cron/pkg-cron.git
Need to get 158 kB of source archives.
Get:1 http://ftp.uk.debian.org/debian/ sid/main cron 3.0pl1-121 (dsc) [1267 B]
Get:2 http://ftp.uk.debian.org/debian/ sid/main cron 3.0pl1-121 (tar) [59.2 kB]
Get:3 http://ftp.uk.debian.org/debian/ sid/main cron 3.0pl1-121 (diff) [97.2 kB]
Fetched 158 kB in 0s (206 kB/s)
dpkg-source: info: extracting cron in cron-3.0pl1
dpkg-source: info: unpacking cron_3.0pl1.orig.tar.gz
dpkg-source: info: applying cron_3.0pl1-121.diff.gz
dpkg-source: info: upstream files that have been modified: 
 cron-3.0pl1/Makefile
 cron-3.0pl1/README
 cron-3.0pl1/compat.c
 cron-3.0pl1/compat.h
 cron-3.0pl1/config.h
 cron-3.0pl1/cron.8
 cron-3.0pl1/cron.c
 cron-3.0pl1/cron.h
 cron-3.0pl1/crontab.1
 cron-3.0pl1/crontab.5
 cron-3.0pl1/crontab.c
 cron-3.0pl1/database.c
 cron-3.0pl1/do_command.c
 cron-3.0pl1/entry.c
 cron-3.0pl1/env.c
 cron-3.0pl1/externs.h
 cron-3.0pl1/job.c
 cron-3.0pl1/misc.c
 cron-3.0pl1/pathnames.h
 cron-3.0pl1/popen.c
 cron-3.0pl1/user.c
$
By the way, if you want to see all the options available to apt, run the following: 
$ apt-config dump
APT "";
APT::Architecture "i386";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "1";
APT::Install-Suggests "0";
APT::Authentication "";
APT::Authentication::TrustCDROM "true";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image.*";
APT::NeverAutoRemove:: "^kfreebsd-image.*";
APT::NeverAutoRemove:: "^linux-restricted-modules.*";
APT::NeverAutoRemove:: "^linux-ubuntu-modules-.*";
APT::NeverAutoRemove:: "^gnumach$";
APT::NeverAutoRemove:: "^gnumach-image.*";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Periodic "";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";
APT::Update "";
APT::Update::Post-Invoke-Success "";
APT::Update::Post-Invoke-Success:: "touch /var/lib/apt/periodic/update-success-stamp 2>/dev/null || true";
APT::Update::Post-Invoke-Success:: "[ ! -f /var/run/dbus/system_bus_socket ] || /usr/bin/dbus-send --system --dest=org.debian.apt --type=signal /org/debian/apt org.debian.apt.CacheChanged || true";
APT::Archives "";
APT::Archives::MaxAge "30";
APT::Archives::MinAge "2";
APT::Archives::MaxSize "500";
APT::Changelogs "";
APT::Changelogs::Server "http://changelogs.ubuntu.com/changelogs";
APT::Architectures "";
APT::Architectures:: "i386";
APT::Compressor "";
APT::Compressor::. "";
APT::Compressor::.::Name ".";
APT::Compressor::.::Extension "";
APT::Compressor::.::Binary "";
APT::Compressor::.::Cost "1";
APT::Compressor::.::CompressArg "";
APT::Compressor::.::CompressArg:: "";
APT::Compressor::.::UncompressArg "";
APT::Compressor::.::UncompressArg:: "";
APT::Compressor::gzip "";
APT::Compressor::gzip::Name "gzip";
APT::Compressor::gzip::Extension ".gz";
APT::Compressor::gzip::Binary "gzip";
APT::Compressor::gzip::Cost "2";
APT::Compressor::gzip::CompressArg "";
APT::Compressor::gzip::CompressArg:: "-9n";
APT::Compressor::gzip::UncompressArg "";
APT::Compressor::gzip::UncompressArg:: "-d";
APT::Compressor::bzip2 "";
APT::Compressor::bzip2::Name "bzip2";
APT::Compressor::bzip2::Extension ".bz2";
APT::Compressor::bzip2::Binary "bzip2";
APT::Compressor::bzip2::Cost "3";
APT::Compressor::bzip2::CompressArg "";
APT::Compressor::bzip2::CompressArg:: "-9";
APT::Compressor::bzip2::UncompressArg "";
APT::Compressor::bzip2::UncompressArg:: "-d";
APT::Compressor::lzma "";
APT::Compressor::lzma::Name "lzma";
APT::Compressor::lzma::Extension ".lzma";
APT::Compressor::lzma::Binary "lzma";
APT::Compressor::lzma::Cost "4";
APT::Compressor::lzma::CompressArg "";
APT::Compressor::lzma::CompressArg:: "-9";
APT::Compressor::lzma::UncompressArg "";
APT::Compressor::lzma::UncompressArg:: "-d";
APT::Compressor::xz "";
APT::Compressor::xz::Name "xz";
APT::Compressor::xz::Extension ".xz";
APT::Compressor::xz::Binary "xz";
APT::Compressor::xz::Cost "5";
APT::Compressor::xz::CompressArg "";
APT::Compressor::xz::CompressArg:: "-6";
APT::Compressor::xz::UncompressArg "";
APT::Compressor::xz::UncompressArg:: "-d";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::solvers "";
Dir::Bin::solvers:: "/usr/lib/apt/solvers";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Bin::lzma "/usr/bin/lzma";
Dir::Bin::xz "/usr/bin/xz";
Dir::Bin::bzip2 "/bin/bzip2";
Dir::Media "";
Dir::Media::MountPath "/media/apt";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Dir::Ignore-Files-Silently:: "\.save$";
Dir::Ignore-Files-Silently:: "\.orig$";
Dir::Ignore-Files-Silently:: "\.distUpgrade$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom/";
Acquire::Languages "";
Acquire::Languages:: "en";
Acquire::Languages:: "none";
Acquire::Languages:: "fr";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "if [ -d /var/lib/update-notifier ]; then touch /var/lib/update-notifier/dpkg-run-stamp; fi; if [ -e /var/lib/update-notifier/updates-available ]; then echo > /var/lib/update-notifier/updates-available; fi ";
Unattended-Upgrade "";
Unattended-Upgrade::Allowed-Origins "";
Unattended-Upgrade::Allowed-Origins:: "${distro_id}:${distro_codename}-security";
CommandLine "";
CommandLine::AsString "apt-config dump";
$

Admittedly, this technique is rather slower than pull-debian-source (atleast for the initial download of the index files), but it is an instructive example of what is possible when you RTFM ;-)

I've bundled the above into a pull-debian-source.sh script. Run it like this: 
$ chmod 755 ./pull-debian-source.sh
$ ./pull-debian-source.sh update
$ ./pull-debian-source.sh source hello


Comments

Post a Comment

Popular posts from this blog

Byobu Bling with Unicode Custom Indicators

Image
Introduction In these heady times of almost daily advances in UI pizzazz, you could be forgiven for thinking the terminal is somewhat of a dead-zone when it comes to excitement. Oh how wrong you would be though...! If you've never used  Byobu ,  Dustin's  awesome text-based window manage, sudo apt-get install it without delay to be bathed in pure terminal goodness. As shown below, byobu comes with a whole host of standard indicators, allowing the display of useful snippets of information. The indicators are minimal out of terminal-width real estate necessity, but that's also their strength - they provide just the details you really care about in a tight ASCII format. And they're cute of course :-) However, byobu has a well-kept secret: when running with the tmux backend, it supports Unicode indicators. So with the proviso that you are running " byobu-tmux " rather than " byobu-screen ", you can squeeze down those indicators even further
Read more

Procenv 0.46 - now with more platform goodness

I have just released procenv version 0.46 . Although this is a very minor release for the existing platforms (essentially 1 bug fix), this release now introduces support for a new platform... Darwin Yup - OS X now joins the ranks of supported platforms. Although adding support for Darwin was made significantly easier as a result of the recent internal restructure of the procenv code, it did present a challenge: I don't own any Apple hardware. I could have borrowed a Macbook, but instead I decided to see this as a challenge: Could I port procenv to Darwin without actually having a local Apple system?  Well, you've just read the answer, but how did I do this? Stage 1: Docker Whilst surfing around I came across this interesting docker image: https://hub.docker.com/r/andrewd/osxcross/ It provides a Darwin toolchain that I could run under Linux. It didn't take very long to follow my own instructions on porting procenv to a new platform . But although I
Read more

Upstart User Sessions in Ubuntu Raring

Image
Overview Ubuntu Raring now includes Upstart 1.8 . Upstart 1.7 and 1.8 combined mark a major milestone since they bring the proven power of Upstart to the user as never before: not only is Upstart now managing the system, but it is also capable of managing the default Ubuntu user's desktop sessions too. Why? Question:  Why reuse a system facility at the user level in this way? The modern Linux desktop environment is a very dynamic one: users start and stop applications, switch workspaces, search the dash, adjust personal settings in the panel, connect to different networks, hot-plug USB devices and so on. All of these activities can be represented by "events". Stepping back a second, recall that Upstart was written  from the outset  to take advantage of  the dynamic nature of a modern Linux system . Long gone are the days when a system booted serially. A modern Linux system abounds with "events" from all sorts of different sources: The user plugs or
Read more