programming

Spring Security, Webjars, and MIME type error

I volunteered to be JLO (Java Language Owner) at CircleCI and I am currently working on getting a sample Spring Framework project running on CircleCI 2.0. I made a simple app bootstrapped with the Spring Initializer. I included Spring Security for the first time and I decided to try out WebJars for static Javascript libraries such as bootstrap. I am using Thymeleaf for templating.The app does not actually do anything yet but I ran into a pretty strange issue today that I wanted to write up here. My home page is pretty straightforward.

<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org">
<head>
    <title>CircleCI Spring Demo</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

    <link rel="stylesheet" th:href="@{/webjars/bootstrap/3.3.7/css/bootstrap.min.css}" />

    <link rel="stylesheet" th:href="@{/css/style.css}" href="../static/css/style.css" />
</head>
<body>

    <nav class="navbar">
        <div class="container">
            <div class="navbar-header">
                <a class="navbar-brand" href="#">CircleCI Demo Spring</a>
            </div>
            <div id="navbar" class="collapse navbar-collapse">
                <ul class="nav navbar-nav">
                    <li class="active"><a href="#">Home</a></li>
                    <li><a href="#">About</a></li>
                </ul>
            </div>
        </div>
    </nav>

    <div class="container">
        <h1> CircleCI Spring Demo </h1>
    </div>

    <script th:src="@{/webjars/bootstrap/3.3.7/js/bootstrap.min.js}"></script>
</body>
</html>

However, when I tried to load up the app with mvn spring-boot:run none of the styles showed up and console showed the following error message:

Resource interpreted as Stylesheet but transferred with MIME type text/html

It turns out, that a default spring-security config will basically block any request unless you whitelist it. The MIME type is a red herring since what is actually happening is that my spring-security config is redirecting all unauthenticated users to my login page (which is login.html) instead of serving up the stylesheet from the/webjars directory. The solution is to update my security configuration to whitelist anything that comes from /webjars

package com.circleci.demojavaspring;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;

@Configuration
@EnableWebSecurity
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .authorizeRequests()
                .antMatchers("/", "/home", "/webjars/**").permitAll()
                .anyRequest().authenticated()
                .and()
            .formLogin()
                .loginPage("/login")
                .permitAll()
                .and()
            .logout()
                .permitAll();
    }
}

Now, the styles load as expected.

Standard
debian, linux

Install Netbeans on Debian Stable

Netbeans is a great open source Java IDE. For some reason it is missing from the current stable repository on debian. In order to get it installed as a regular desktop application in Debian Jessie (using GNOME) you should do the following:

  1. JDK 8 is required in order to use netbeans. The default-jdkpackage on Jessie installs jdk7. First you must enable debian backportsand then you You can install it with sudo apt install -t jessie-backports openjdk-8-jdk
  2. Download the latest version from the releases page. There are a couple different flavors. I usually choose the one that contains everything. This will download a bash installer script.
  3. Open up a terminal and navigate to wherever you downloaded the script from Step 2. Execute the script with sh netbeans*.sh
  4. This will run some pre-flight checks and then fire up an installation wizard that will guide you through the rest of the process.
  5. Once Netbeans has been installed you can launch it by clicking on the icon that should now be on your desktop.
Standard
debian, programming

Using gtk-doc with Anjuta on Debian Stable

gtk-doc is a library that helps extract code documentation. When you create a new project with Anjuta it asks if you wish to include gkt-doc. Unfortunately, on Debian stable there seems to be a bug because the autoconf configuration is looking for the wrong version of gtk-doc.

/home/levlaz/git/librefocus/configure: line 13072: syntax error near unexpected token `1.0'
/home/levlaz/git/librefocus/configure: line 13072: `GTK_DOC_CHECK(1.0)'

On Debian stable, the version of GTK doc that comes with thegtk-doc-tools package is 1.21. In order to resolve this error you need to update configure.ac to use the newer version of gtk-doc as shown below:

GTK_DOC_CHECK([1.21])

Then you need to regenerate the entire project and everything should work as expected.

Standard
debian, software

Terminal Reader Mode with Pandoc and Less

The other day Aosheng send me an article to read from the verge. When I tried to read it, it took about 5 minutes to load because of the 15 various JavaScript things that were running in addition to ads loading in the background. Firefox was unhappy, and even when I tried to turn on “Reader View” (which strips out all of the junk) it took another minute to load. I’ve been on a UNIX binge lately so I figured there had to be a clever hack to make my own reader view in a terminal. This is where pandoc comes to the rescue. I’ve written about this tool in the past discussing how to easily convert Markdown to PDF. It turns out that pandoc also supports arbitrary URL arguments which means that you can convert HTML files on the fly without having to download them first. This means that we can take an arbitrary URL, pass it into pandoc, and spit out plain text. Furthermore, we can pipe this into less to get a nice pager for longer documents. The full string is shown below:

pandoc -f html -t plain
https://www.theverge.com/2017/5/4/15547314/edward-snowden-cory-doctorow-nypl-talk-walkaway
| less

In the example above, -f specifies the input filetype, in this case HTML. -t specifies the conversion filetype, in this case plain text. Pandoc supports a ton of different formats, you can read the man pagefor more info.

The next logical step is to make a script like my wordpress mutt posterto make this even easier. You could make a simple program called readerand put it in /usr/local/bin/reader. The contents of this script are:

1
2
3
4
5
6
#!/bin/bash
# Terminal Reader Mode using Pandoc and Less

url="$1"

pandoc -f html -t plain $url | less

You can then use this  by typing reader $URL.

Standard
software

Posting to WordPress via Email with Mutt

Soemtimes you are hanging out in your terminal and you just want to be able to post something to your blog quickly. I was pretty inspired by Derek Siver’s OpenBSD post [1] where he really embraces the unix philosophy of having one tool to do a job correctly and putting together various small tools like this to come up with a solution for the problem that you are trying to solve with your computer. WordPress with Jetpack makes it dead simple to post to your blog via email [2], even if you do not have a mail server configured. I was able to write a three line bash script to “automate” creating a new post from my command line.’

#!/bin/bash
# Bash Utility to Post to WordPress using Mutt

subject="$1"
WP_ADDRESS=

mutt -s "${subject}" $WP_ADDRESS

I saved this file in /usr/local/bin/wp and whenever I am inspired to fire off some quick thoughts to this blog I can run wp "Blog Post Title" which dumps me into a vim buffer that once I complete is sent off via mutt to wordpress. [1] https://sivers.org/openbsd [2]https://jetpack.com/support/post-by-email/#examples

Standard
debian, linux

Reading gz files with zcat

The Debian Policy Manual dictates that all packages should come with documentation. In order to save space in the debian archive these documents need to be compressed with gzip. There are a ton of these files floating around in the /usr/share/doc directory. Recently I wanted to read some of the documentation. If you try to open the file with cat it spits out binary gibberish. You can of course unzip the file as you normally would and open it up that way, but it turns out there is an easier way. Using zcat you can read the contents of compressed files just like you would with cat.

zcat is identical to gunzip -c. (On some systems, zcat may be installed as gzcat to preserve the original link to compress.) zcat uncompresses either a list of files on the command line or its standard input and writes the uncompressed data on standard output. zcat will uncompress files that have the correct magic number whether they have a .gz suffix or not. GZIP(1) man page.

By default, this will put all of the output into your terminal window, which is fine for most files. The other place where this can come in handy is when you are trying to look through compressed log files. In this case, having to scroll around the terminal may not be a great option. You can pipe the output of zcat into other programs such asless in order to be able to page through long files. For example, if I wanted to read the first 10 lines of a compressed log file, I could do so with the following command:

levlaz@debvm:/var/log$ sudo zcat syslog.2.gz | head -n 10

The output of this command would look like this:

May  2 22:27:43 debvm rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="585" x-info="http://www.rsyslog.com"] start
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1a] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1b] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1c] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1d] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1e] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x1f] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x20] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x21] high edge lint[0x1])
May  2 22:27:43 debvm kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0x22] high edge lint[0x1])
Standard
debian, linux

Help Out With Packages You Use in Debian

Many new and existing Debian users want to help make the distribution  better but do not quite know where to begin. Debian comes with a very handy package called how-can-i-help which tells you after each aptinvocation the current bugs that are associated with packages on your system. The “Work-Needing and Perspective Packages” (WNPP) listing is a bit overwhelming for new contributors. What better way to figure out what packages need your help than by seeing a list of them each time you use apt.

The first time you run apt after installing this package it will likely spit out a long list of packages that need your help. Each subsequent time it will only show new packages or changes. In order to see the master list again you can use the how-can-i-help --old command to see all packages that need your help. I think this is a great way to get engaged with the software that you rely on each day.

Although getting started with Debian development is not trivial, this lowers the barrier a bit and provides some clear direction on what to work on since the list includes packages that you are using every day.

Standard
debian, software

Using Owncloud Client for Nextcloud Server on Debian Stable

There is no official debian package for the nextcloud client. There have been a handful of RFP bugs reported but it looks like no one has taken this on yet. I want to get more involved with debian packaging so this might be a great first package to maintain. For the time being, the owncloud client is still backwards compatible with nextcloud. Unfortunately, the version that ships with Debian stable (8, jessie at the time of writing) is a bit old. When I tried to connect to my nextcloud instance it complained that my password was incorrect. Luckily, there is a slightly newer version available in jessie-backports  which has no trouble connecting to nextcloud. The steps to get a working version of owncloud-client to work with the latest stable version of Nextcloud are as follows:

  1. If you have not already, enable jessie-backports
    1. Open up /etc/apt/sources.list
    2. Append deb http://ftp.debian.org/debian jessie-backports mainto that file.
  2. Run sudo apt-get update
  3. Install the latest version of owncloud-client with sudo apt-get install -t jessie-backports owncloud-client

You should now be able to connect to nextcloud without any issues.

Standard
debian, software

Testing Syntax Errors in Apache Config

If you spend any time mucking around config files in Linux you are likely to run into some syntax errors sooner or later. Recently I was setting up cgit on Debian 8 and was banging my head against the wall for a few minutes trying to figure out why apache was so unhappy.

Symptoms

The key issue was when I restarted apache2 like I normally would after adding a new configuration it spat out an angry message at me.

root@nuc:/etc/apache2# sudo service apache2 restart
Job for apache2.service failed. See 'systemctl status apache2.service' and 'journalctl -xn' for details.

Troubleshooting

The first place that I would look is the error logs. However, in this particular case they were not very helpful.

root@nuc:/etc/apache2# tail -f /var/log/apache2/error.log
[Mon May 01 21:00:11.922943 2017] [mpm_prefork:notice] [pid 20454] AH00169: caught SIGTERM, shutting down

Next, I read the error message per the suggestion from the restart command. This was also not very helpful.

root@nuc:/etc/apache2# systemctl status apache2.service
 apache2.service - LSB: Apache2 web server
 Loaded: loaded (/etc/init.d/apache2)
 Drop-In: /lib/systemd/system/apache2.service.d
 └─forking.conf
 Active: failed (Result: exit-code) since Mon 2017-05-01 21:05:58 PDT; 1min 45s ago
 Process: 20746 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)
 Process: 20697 ExecReload=/etc/init.d/apache2 reload (code=exited, status=1/FAILURE)
 Process: 20920 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)

May 01 21:05:58 nuc apache2[20920]: Starting web server: apache2 failed!
May 01 21:05:58 nuc apache2[20920]: The apache2 configtest failed. ... (warning).
May 01 21:05:58 nuc apache2[20920]: Output of config test was:
May 01 21:05:58 nuc apache2[20920]: apache2: Syntax error on line 219 of /etc/apache2/apache2.conf: Syntax error on line 22 of /etc/a... section
May 01 21:05:58 nuc apache2[20920]: Action 'configtest' failed.
May 01 21:05:58 nuc apache2[20920]: The Apache error log may have more information.
May 01 21:05:58 nuc systemd[1]: apache2.service: control process exited, code=exited status=1
May 01 21:05:58 nuc systemd[1]: Failed to start LSB: Apache2 web server.
May 01 21:05:58 nuc systemd[1]: Unit apache2.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.

Inspecting the error message, we see that it is unhappy with line 219 of the main /etc/apache2/apache2.conf file. Looking at that line we can see that it is simply loading all of the other config files in sites-enabled which means that before it even gets to load my new cgit config file it fails.

Help

So now that we have done some basic troubleshooting. It’s time to dig into the manual for further information. I know that the config file is failing to load, and knowing my fat fingers it is very likely a config error on my part. Before reading 200 pages of documentation on the apache website we should take a look at the built in help to see if we can find something of value.

root@nuc:/etc/apache2# apache2 -help
Usage: apache2 [-D name] [-d directory] [-f file]
 [-C "directive"] [-c "directive"]
 [-k start|restart|graceful|graceful-stop|stop]
 [-v] [-V] [-h] [-l] [-L] [-t] [-T] [-S] [-X]
Options:
 -D name : define a name for use in <IfDefine name> directives
 -d directory : specify an alternate initial ServerRoot
 -f file : specify an alternate ServerConfigFile
 -C "directive" : process directive before reading config files
 -c "directive" : process directive after reading config files
 -e level : show startup errors of level (see LogLevel)
 -E file : log startup errors to file
 -v : show version number
 -V : show compile settings
 -h : list available command line options (this page)
 -l : list compiled in modules
 -L : list available configuration directives
 -t -D DUMP_VHOSTS : show parsed vhost settings
 -t -D DUMP_RUN_CFG : show parsed run settings
 -S : a synonym for -t -D DUMP_VHOSTS -D DUMP_RUN_CFG
 -t -D DUMP_MODULES : show all loaded modules
 -M : a synonym for -t -D DUMP_MODULES
 -t : run syntax check for config files
 -T : start without DocumentRoot(s) check
 -X : debug mode (only one worker, do not detach)

Success! It turns out we can run a linter on a specific config file using the -t flag.

Solution

root@nuc:/etc/apache2# apache2 -t -f sites-available/git.levlaz.org.conf
apache2: Syntax error on line 22 of /etc/apache2/sites-available/git.levlaz.org.conf: </VirtualHost> without matching <VirtualHost> section

Doh! Such a silly mistake with a missing </VirtualHost> closing bracket. Fixing this syntax error resolved the issue. The main takeaway for me is that the best part about most Linux tools is that they usually give you everything you need in order to succeed. We were able to troubleshoot and resolve this issue without resorting to google and running random commands that stranger posted on the internet 5 years ago.

Standard