Docker Based Development Environment for Packaging

I started a new repo called sid-builder which provides a docker based development environment for debian packaging. This project follows all of the best practices that were outlined in the Guide for New Maintainers. The benefit of this approach is that you should be able to easily reproduce your development environment on any machine that supports docker. It also allows you to easily run sid without worrying about breaking your system.

I have not actually built any packages using this setup just yet, but I am going to give it a try over the next few days and report back here.

Becoming a Debian Developer

I’ve been using debian for as long as I remember. I’ve always wanted to play a more active role in debian development but for whatever reason I never got around to it. Now that I am 30, older, wiser, I am starting a new push to become a debian developer.

So far I have gotten involved in the debian-qa team. Specifically I have been working on fixing some of the newcomer bugs on the distro-tracker project. It’s actually been really fun. The code base is django which I am pretty comfortable with. For the first time in many years, I have been rushing home after work so that I could keep hacking on the bugs that I am working on.

Code contributions are one thing. The thing that I am going to need more practice on is debian packaging itself. This is pretty complex process and I think that becoming someone who can debug and perform packaging issues would bring a lot of value to the project.

I am going to start working on some of the RC bugs (which typically involve at least re-packaging software) to get more comfortable with how other folks have been doing packaging.

In the next few months I would love to bring a new package through it’s full lifecycle.

I hope to be able to look back on this blog post in the future and see how far I have come.

Install Netbeans on Debian Stable

Netbeans is a great open source Java IDE. For some reason it is missing from the current stable repository on debian. In order to get it installed as a regular desktop application in Debian Jessie (using GNOME) you should do the following:

  1. JDK 8 is required in order to use netbeans. The default-jdkpackage on Jessie installs jdk7. First you must enable debian backportsand then you You can install it with sudo apt install -t jessie-backports openjdk-8-jdk
  2. Download the latest version from the releases page. There are a couple different flavors. I usually choose the one that contains everything. This will download a bash installer script.
  3. Open up a terminal and navigate to wherever you downloaded the script from Step 2. Execute the script with sh netbeans*.sh
  4. This will run some pre-flight checks and then fire up an installation wizard that will guide you through the rest of the process.
  5. Once Netbeans has been installed you can launch it by clicking on the icon that should now be on your desktop.

Using gtk-doc with Anjuta on Debian Stable

gtk-doc is a library that helps extract code documentation. When you create a new project with Anjuta it asks if you wish to include gkt-doc. Unfortunately, on Debian stable there seems to be a bug because the autoconf configuration is looking for the wrong version of gtk-doc.

/home/levlaz/git/librefocus/configure: line 13072: syntax error near unexpected token `1.0'
/home/levlaz/git/librefocus/configure: line 13072: `GTK_DOC_CHECK(1.0)'

On Debian stable, the version of GTK doc that comes with thegtk-doc-tools package is 1.21. In order to resolve this error you need to update configure.ac to use the newer version of gtk-doc as shown below:

GTK_DOC_CHECK([1.21])

Then you need to regenerate the entire project and everything should work as expected.

Terminal Reader Mode with Pandoc and Less

The other day Aosheng send me an article to read from the verge. When I tried to read it, it took about 5 minutes to load because of the 15 various JavaScript things that were running in addition to ads loading in the background. Firefox was unhappy, and even when I tried to turn on “Reader View” (which strips out all of the junk) it took another minute to load. I’ve been on a UNIX binge lately so I figured there had to be a clever hack to make my own reader view in a terminal. This is where pandoc comes to the rescue. I’ve written about this tool in the past discussing how to easily convert Markdown to PDF. It turns out that pandoc also supports arbitrary URL arguments which means that you can convert HTML files on the fly without having to download them first. This means that we can take an arbitrary URL, pass it into pandoc, and spit out plain text. Furthermore, we can pipe this into less to get a nice pager for longer documents. The full string is shown below:

pandoc -f html -t plain
https://www.theverge.com/2017/5/4/15547314/edward-snowden-cory-doctorow-nypl-talk-walkaway
| less

In the example above, -f specifies the input filetype, in this case HTML. -t specifies the conversion filetype, in this case plain text. Pandoc supports a ton of different formats, you can read the man pagefor more info.

The next logical step is to make a script like my wordpress mutt posterto make this even easier. You could make a simple program called readerand put it in /usr/local/bin/reader. The contents of this script are:

1
2
3
4
5
6
#!/bin/bash
# Terminal Reader Mode using Pandoc and Less

url="$1"

pandoc -f html -t plain $url | less

You can then use this  by typing reader $URL.

Reading gz files with zcat

The Debian Policy Manual dictates that all packages should come with documentation. In order to save space in the debian archive these documents need to be compressed with gzip. There are a ton of these files floating around in the /usr/share/doc directory. Recently I wanted to read some of the documentation. If you try to open the file with cat it spits out binary gibberish. You can of course unzip the file as you normally would and open it up that way, but it turns out there is an easier way. Using zcat you can read the contents of compressed files just like you would with cat.

zcat is identical to gunzip -c. (On some systems, zcat may be installed as gzcat to preserve the original link to compress.) zcat uncompresses either a list of files on the command line or its standard input and writes the uncompressed data on standard output. zcat will uncompress files that have the correct magic number whether they have a .gz suffix or not. GZIP(1) man page.

By default, this will put all of the output into your terminal window, which is fine for most files. The other place where this can come in handy is when you are trying to look through compressed log files. In this case, having to scroll around the terminal may not be a great option. You can pipe the output of zcat into other programs such asless in order to be able to page through long files. For example, if I wanted to read the first 10 lines of a compressed log file, I could do so with the following command:

levlaz@debvm:/var/log$ sudo zcat syslog.2.gz | head -n 10

The output of this command would look like this:

May  2 22:27:43 debvm rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="585" x-info="http://www.rsyslog.com"] start
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1a] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1b] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1c] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1d] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1e] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1f] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x20] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x21] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x22] high edge lint[0x1])

Help Out With Packages You Use in Debian

Many new and existing Debian users want to help make the distribution  better but do not quite know where to begin. Debian comes with a very handy package called how-can-i-help which tells you after each aptinvocation the current bugs that are associated with packages on your system. The “Work-Needing and Perspective Packages” (WNPP) listing is a bit overwhelming for new contributors. What better way to figure out what packages need your help than by seeing a list of them each time you use apt.

The first time you run apt after installing this package it will likely spit out a long list of packages that need your help. Each subsequent time it will only show new packages or changes. In order to see the master list again you can use the how-can-i-help --old command to see all packages that need your help. I think this is a great way to get engaged with the software that you rely on each day.

Although getting started with Debian development is not trivial, this lowers the barrier a bit and provides some clear direction on what to work on since the list includes packages that you are using every day.

Using Owncloud Client for Nextcloud Server on Debian Stable

There is no official debian package for the nextcloud client. There have been a handful of RFP bugs reported but it looks like no one has taken this on yet. I want to get more involved with debian packaging so this might be a great first package to maintain. For the time being, the owncloud client is still backwards compatible with nextcloud. Unfortunately, the version that ships with Debian stable (8, jessie at the time of writing) is a bit old. When I tried to connect to my nextcloud instance it complained that my password was incorrect. Luckily, there is a slightly newer version available in jessie-backports  which has no trouble connecting to nextcloud. The steps to get a working version of owncloud-client to work with the latest stable version of Nextcloud are as follows:

  1. If you have not already, enable jessie-backports
    1. Open up /etc/apt/sources.list
    2. Append deb http://ftp.debian.org/debian jessie-backports mainto that file.
  2. Run sudo apt-get update
  3. Install the latest version of owncloud-client with sudo apt-get install -t jessie-backports owncloud-client

You should now be able to connect to nextcloud without any issues.

Testing Syntax Errors in Apache Config

If you spend any time mucking around config files in Linux you are likely to run into some syntax errors sooner or later. Recently I was setting up cgit on Debian 8 and was banging my head against the wall for a few minutes trying to figure out why apache was so unhappy.

Symptoms

The key issue was when I restarted apache2 like I normally would after adding a new configuration it spat out an angry message at me.

root@nuc:/etc/apache2# sudo service apache2 restart
Job for apache2.service failed. See 'systemctl status apache2.service' and 'journalctl -xn' for details.

Troubleshooting

The first place that I would look is the error logs. However, in this particular case they were not very helpful.

root@nuc:/etc/apache2# tail -f /var/log/apache2/error.log
[Mon May 01 21:00:11.922943 2017] [mpm_prefork:notice] [pid 20454] AH00169: caught SIGTERM, shutting down

Next, I read the error message per the suggestion from the restart command. This was also not very helpful.

root@nuc:/etc/apache2# systemctl status apache2.service
 apache2.service - LSB: Apache2 web server
 Loaded: loaded (/etc/init.d/apache2)
 Drop-In: /lib/systemd/system/apache2.service.d
 └─forking.conf
 Active: failed (Result: exit-code) since Mon 2017-05-01 21:05:58 PDT; 1min 45s ago
 Process: 20746 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)
 Process: 20697 ExecReload=/etc/init.d/apache2 reload (code=exited, status=1/FAILURE)
 Process: 20920 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)

May 01 21:05:58 nuc apache2[20920]: Starting web server: apache2 failed!
May 01 21:05:58 nuc apache2[20920]: The apache2 configtest failed. ... (warning).
May 01 21:05:58 nuc apache2[20920]: Output of config test was:
May 01 21:05:58 nuc apache2[20920]: apache2: Syntax error on line 219 of /etc/apache2/apache2.conf: Syntax error on line 22 of /etc/a... section
May 01 21:05:58 nuc apache2[20920]: Action 'configtest' failed.
May 01 21:05:58 nuc apache2[20920]: The Apache error log may have more information.
May 01 21:05:58 nuc systemd[1]: apache2.service: control process exited, code=exited status=1
May 01 21:05:58 nuc systemd[1]: Failed to start LSB: Apache2 web server.
May 01 21:05:58 nuc systemd[1]: Unit apache2.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.

Inspecting the error message, we see that it is unhappy with line 219 of the main /etc/apache2/apache2.conf file. Looking at that line we can see that it is simply loading all of the other config files in sites-enabled which means that before it even gets to load my new cgit config file it fails.

Help

So now that we have done some basic troubleshooting. It’s time to dig into the manual for further information. I know that the config file is failing to load, and knowing my fat fingers it is very likely a config error on my part. Before reading 200 pages of documentation on the apache website we should take a look at the built in help to see if we can find something of value.

root@nuc:/etc/apache2# apache2 -help
Usage: apache2 [-D name] [-d directory] [-f file]
 [-C "directive"] [-c "directive"]
 [-k start|restart|graceful|graceful-stop|stop]
 [-v] [-V] [-h] [-l] [-L] [-t] [-T] [-S] [-X]
Options:
 -D name : define a name for use in <IfDefine name> directives
 -d directory : specify an alternate initial ServerRoot
 -f file : specify an alternate ServerConfigFile
 -C "directive" : process directive before reading config files
 -c "directive" : process directive after reading config files
 -e level : show startup errors of level (see LogLevel)
 -E file : log startup errors to file
 -v : show version number
 -V : show compile settings
 -h : list available command line options (this page)
 -l : list compiled in modules
 -L : list available configuration directives
 -t -D DUMP_VHOSTS : show parsed vhost settings
 -t -D DUMP_RUN_CFG : show parsed run settings
 -S : a synonym for -t -D DUMP_VHOSTS -D DUMP_RUN_CFG
 -t -D DUMP_MODULES : show all loaded modules
 -M : a synonym for -t -D DUMP_MODULES
 -t : run syntax check for config files
 -T : start without DocumentRoot(s) check
 -X : debug mode (only one worker, do not detach)

Success! It turns out we can run a linter on a specific config file using the -t flag.

Solution

root@nuc:/etc/apache2# apache2 -t -f sites-available/git.levlaz.org.conf
apache2: Syntax error on line 22 of /etc/apache2/sites-available/git.levlaz.org.conf: </VirtualHost> without matching <VirtualHost> section

Doh! Such a silly mistake with a missing </VirtualHost> closing bracket. Fixing this syntax error resolved the issue. The main takeaway for me is that the best part about most Linux tools is that they usually give you everything you need in order to succeed. We were able to troubleshoot and resolve this issue without resorting to google and running random commands that stranger posted on the internet 5 years ago.

Change the Default Terminal Editor in Debian

Debian comes with a very handy utility called update-alternatives that helps to set default tools for various tasks.

It is possible for several programs fulfilling the same or similar functions to be installed on a single system at the same time. For example, many systems have several text editors installed at once. This gives choice to the users of a system, allowing each to use a different editor, if desired, but makes it difficult for a program to make a good choice for an editor to invoke if the user has not specified a particular preference.

On Linode, it seems that the default editor is nano, I prefer to use vim for editing git commits, visudo, and other things that use the default editor which is symbolically linked through /usr/bin/editor. The update-alternatives package basically changes the symbolic links for you. In order to change your default editor, you simply need to run the following command:

sudo update-alternatives --config editor

The output of this command is shown below. You will see a list of all of your editors that you currently have installed and will be asked to make a choice.

There are 3 choices for the alternative editor (providing /usr/bin/editor).

Selection Path Priority Status
------------------------------------------------------------
 0 /bin/nano 40 auto mode
 1 /bin/nano 40 manual mode
 2 /usr/bin/vim.basic 30 manual mode
* 3 /usr/bin/vim.tiny 10 manual mode

Press enter to keep the current choice[*], or type selection number:

Behind the scenes you can see that all this does it updates the symbolic links.

levlaz@dev:~$ ls -al /usr/bin/editor
lrwxrwxrwx 1 root root 24 Feb 10 20:49 /usr/bin/editor -> /etc/alternatives/editor
levlaz@dev:~$ ls -al /etc/alternatives/editor
lrwxrwxrwx 1 root root 17 Apr 28 18:56 /etc/alternatives/editor -> /usr/bin/vim.tiny

There are many other things that can be configured this way. For more information reading the man page for update-alternatives is worthwhile.