If you spend any time mucking around config files in Linux you are likely to run into some syntax errors sooner or later. Recently I was setting up cgit on Debian 8 and was banging my head against the wall for a few minutes trying to figure out why apache was so unhappy.
Symptoms
The key issue was when I restarted apache2 like I normally would after adding a new configuration it spat out an angry message at me.
root@nuc:/etc/apache2# sudo service apache2 restart
Job for apache2.service failed. See 'systemctl status apache2.service' and 'journalctl -xn' for details.
Troubleshooting
The first place that I would look is the error logs. However, in this particular case they were not very helpful.
root@nuc:/etc/apache2# tail -f /var/log/apache2/error.log
[Mon May 01 21:00:11.922943 2017] [mpm_prefork:notice] [pid 20454] AH00169: caught SIGTERM, shutting down
Next, I read the error message per the suggestion from the restart command. This was also not very helpful.
root@nuc:/etc/apache2# systemctl status apache2.service
● apache2.service - LSB: Apache2 web server
Loaded: loaded (/etc/init.d/apache2)
Drop-In: /lib/systemd/system/apache2.service.d
└─forking.conf
Active: failed (Result: exit-code) since Mon 2017-05-01 21:05:58 PDT; 1min 45s ago
Process: 20746 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)
Process: 20697 ExecReload=/etc/init.d/apache2 reload (code=exited, status=1/FAILURE)
Process: 20920 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)
May 01 21:05:58 nuc apache2[20920]: Starting web server: apache2 failed!
May 01 21:05:58 nuc apache2[20920]: The apache2 configtest failed. ... (warning).
May 01 21:05:58 nuc apache2[20920]: Output of config test was:
May 01 21:05:58 nuc apache2[20920]: apache2: Syntax error on line 219 of /etc/apache2/apache2.conf: Syntax error on line 22 of /etc/a... section
May 01 21:05:58 nuc apache2[20920]: Action 'configtest' failed.
May 01 21:05:58 nuc apache2[20920]: The Apache error log may have more information.
May 01 21:05:58 nuc systemd[1]: apache2.service: control process exited, code=exited status=1
May 01 21:05:58 nuc systemd[1]: Failed to start LSB: Apache2 web server.
May 01 21:05:58 nuc systemd[1]: Unit apache2.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.
Inspecting the error message, we see that it is unhappy with line 219 of the main /etc/apache2/apache2.conf file. Looking at that line we can see that it is simply loading all of the other config files in sites-enabled which means that before it even gets to load my new cgit config file it fails.
Help
So now that we have done some basic troubleshooting. It’s time to dig into the manual for further information. I know that the config file is failing to load, and knowing my fat fingers it is very likely a config error on my part. Before reading 200 pages of documentation on the apache website we should take a look at the built in help to see if we can find something of value.
root@nuc:/etc/apache2# apache2 -help
Usage: apache2 [-D name] [-d directory] [-f file]
[-C "directive"] [-c "directive"]
[-k start|restart|graceful|graceful-stop|stop]
[-v] [-V] [-h] [-l] [-L] [-t] [-T] [-S] [-X]
Options:
-D name : define a name for use in <IfDefine name> directives
-d directory : specify an alternate initial ServerRoot
-f file : specify an alternate ServerConfigFile
-C "directive" : process directive before reading config files
-c "directive" : process directive after reading config files
-e level : show startup errors of level (see LogLevel)
-E file : log startup errors to file
-v : show version number
-V : show compile settings
-h : list available command line options (this page)
-l : list compiled in modules
-L : list available configuration directives
-t -D DUMP_VHOSTS : show parsed vhost settings
-t -D DUMP_RUN_CFG : show parsed run settings
-S : a synonym for -t -D DUMP_VHOSTS -D DUMP_RUN_CFG
-t -D DUMP_MODULES : show all loaded modules
-M : a synonym for -t -D DUMP_MODULES
-t : run syntax check for config files
-T : start without DocumentRoot(s) check
-X : debug mode (only one worker, do not detach)
Success! It turns out we can run a linter on a specific config file using the -t flag.
Solution
root@nuc:/etc/apache2# apache2 -t -f sites-available/git.levlaz.org.conf
apache2: Syntax error on line 22 of /etc/apache2/sites-available/git.levlaz.org.conf: </VirtualHost> without matching <VirtualHost> section
Doh! Such a silly mistake with a missing </VirtualHost> closing bracket. Fixing this syntax error resolved the issue. The main takeaway for me is that the best part about most Linux tools is that they usually give you everything you need in order to succeed. We were able to troubleshoot and resolve this issue without resorting to google and running random commands that stranger posted on the internet 5 years ago.
I see this now as a less of a problem, this model works, its clear, use it and focus on building your product instead of re-inveting the wheel of what product pages are supposed to look like.