software

Upgrading and Restarting Salt on OS X

I have been working a lot with salt this month on OS X. I have done some work in the past with Linux and it is a much more pleasant experience. Recently I have salted the installation of DataDog on a fleet of Mac Minis. My initial attempts did not work for several reasons. First, I was getting an error message from salt complaining that list indicies must be integers, not str. This turned out to be a python bug that was resolved in the latest version of salt. After upgrading salt-master and salt-minion to the latest versions I was getting yet another error stating that global name '__salt_system_encoding__' is not defined. It turns out that this error can be resolved by restarting salt-minion. This turned out to be an issue on OS X because if you attempt to simply restart the process with salt-minion restart it hangs up and does nothing. If you try to run kill, or killall it is not able to match the name salt-minion for some reason. At first I was logging into each box manually and kill 9ing, but this kind of defeated the entire purpose of salt so I was determined to find a better way. This is where pkill comes in to save the day. pkill is able to match on salt-minion and is a working solution to restart salt-minion. On OS X salt-minion appears to run as a super daemon and immediately restarts if you try to kill it. This is not so bad since all we need to do to resolve the errors above is kill all of the old salt-minion processes. The complete process to upgrade and restart salt-minion on OS X is outlined below:

  1. Upgrade Salt (Assuming you installed salt with pip)
    `salt '*' cmd.run '/usr/local/bin/pip install salt --upgrade'`
    
  2. Restart Salt Minion (Be extra careful with this command)
    `salt '*' cmd.run 'sudo pkill -f salt-minion'`
    
  3. Verify that you are on the latest version (salt-minion 2015.5.5 (Lithium) as of this post)
    `salt '*' cmd.run '/usr/local/bin/salt-minion --version'`
    

After following these steps, everything is now working! So, now I have the latest version of salt-minion, DataDog chugging along, and I was able to do it all without resorting to writing some jenky bash script or logging in manually to each box. Salt on OS X has been a headache so far, but hopefully with everything being on the latest version things will be a bit smoother going forward.

Standard
debian, linux

Using cgit

This is the last post in a three part series about cgit. Previously we saw how to install and configure cgit and then we saw how to make cgit pretty. This last post has to do with actually using cgit. There are a couple of tools out there to make the process of adding, removing, and modifying git repositories within cgit much simpler but we will focus here on how to do it manually and I will leave it up to you to decide which tools works best for you. cgit does not impose any special requirements for adding, removing, or modifying repositories so the entire process is based around core git functionality. Below is a workflow that works well for me.

Creating a Git User

Creating a git user that has access to /srv/git just makes good sense. To add a new user in Debian you will need to run the following commands and follow the on screen instructions to complete this process. adduser git

Adding your public key

In order to authenticate to the server and work with git it is good to add your public key to avoid working with passwords. If you do not already have a key, go ahead and create one on your local machine. ssh-keygen -b 8192 Follow the on screen instructions, and be sure to add a password to this key! Once you have create this key, copy the public key over to your git server under the git users .ssh/authorized_keysfile. scp $USER/.ssh/id_rsa.pub git@git.example.com:/home/git/.ssh/authorized_keys

Creating a new repository

Log into the git server as the git user and create a bare repository

cd /srv/git 
mkdir myrepo.git 
cd myrepo.git 
git init --bare

Push to the new repository

On your local computer

mkdir myrepo 
cd myrepo git init echo "My Awesome Repo" >> README.md 
git add README.md 
git commit -m 'initial commit' 
git remote add origin git@git.example.com:/srv/git/myrepo.git 
git push origin master

Add the new repository to cgit

Now that your repository is on your server, you can add it cgit by editing /etc/cgitrc and adding it as we did before.

repo.url = myrepo 
repo.path = /srv/git/myrepo.git 
repo.desc = My Awesome Repo 
repo.owner = Your Name

You are all set! Now you will see the new repo and any future changes inside of cgit as well as being able to hack away with git as you normally would.

Standard
debian, linux

Making cgit Pretty

Previously I wrote about how to install and configure cgit on Debian. If you have followed that short guide you now have a kick ass cgit install running on your server. So whats next? A bare cgit installation is pretty dull and there are some neat built in features that should be enabled for an optimal user experience. Open up /etc/cgitrc and made the following additions.

Configure Default README

You can set the default README for all repos by adding the following line. readme=master:README.md This will tell cgit to look for the README.md file in your master branch for the default README for a given project.

Add an about filter

You can add an about filter which will add the about tab to your projects and display the README file configured in the previous step.about-filter=/usr/lib/cgit/filters/about-formatting.sh Assuming that you have a README.md file in your master branch you will now see it in the about tab of your project.

Enable Syntax Highlighting

This is a no brainer. Adding syntax highlighting makes your code easier to look at on the website. This takes a few steps. First install the highlight package with sudo apt-get install highlight . Next, open up/usr/lib/cgit/filters/syntax-highlighting.sh and uncomment the last line that reads exec highlight --force -f -I -O xhtml -S "$EXTENSION" 2>/dev/null Next, open up your stylesheet which should be found inusr/share/cgit/cgit.css and add the following sections.

# Style definition file generated by highlight 2.4.8, http://www.andre-simon.de/ 
table.blob .num { color:#2928ff; } 
table.blob .esc { color:#ff00ff; } 
table.blob .str { color:#ff0000; } 
table.blob .dstr { color:#818100; } 
table.blob .slc { color:#838183; font-style:italic; } 
table.blob .com { color:#838183; font-style:italic; } 
table.blob .dir { color:#008200; } 
table.blob .sym { color:#000000; } 
table.blob .kwa { color:#000000; font-weight:bold; } 
table.blob .kwb { color:#830000; } 
table.blob .kwc { color:#000000; font-weight:bold; } 
table.blob .kwd { color:#010181; }  

# Style definition file generated by highlight 3.13, http://www.andre-simon.de/  
body.hl { background-color:#e0eaee; } 
pre.hl { color:#000000; background-color:#e0eaee; 
font-size:10pt; 
font-family:'Courier New',monospace;} 
.hl.num { color:#b07e00; } 
.hl.esc { color:#ff00ff; } 
.hl.str { color:#bf0303; } 
.hl.pps { color:#818100; } 
.hl.slc { color:#838183; font-style:italic; } 
.hl.com { color:#838183; font-style:italic; } 
.hl.ppc { color:#008200; } 
.hl.opt { color:#000000; } 
.hl.ipl { color:#0057ae; } 
.hl.lin { color:#555555; } 
.hl.kwa { color:#000000; font-weight:bold; } 
.hl.kwb { color:#0057ae; } 
.hl.kwc { color:#000000; font-weight:bold; } 
.hl.kwd { color:#010181; }

Lastly, add this to your configuration file to enable syntax highlighting.

source-filter=/usr/lib/cgit/filters/syntax-highlighting.sh

Since most of my repos look the same I configured these settings globally. You can also configure this at a per-repo level if you simply add any of theses sections to your specific repo settings. For example if you have a different README format such as README.rst for a repo called “my-repo” Then your configuration for that specfic repo would look something like this:

repo.url=my-repo 
repo.path=/srv/git/my-repo.git 
repo.readme=master:README.rst 
repo.about-filter=/usr/lib/cgit/filters/about-formatting.sh 
repo.desc=The best repo ever 
repo.owner=Your Name

Now you have a super fast git repo web viewer and it has a couple of bells and whistles as well.

Standard
debian, linux

Installing cgit + nginx on Debian Jessie

cgit is a hyperfast web frontend for git repositories written in C. Coupled with nginx, this makes a super quick git repository viewer for your web server. I was not able to find a good tutorial on how to get this installed with nginx on a Debian server. The latest version of Debian Stable (Jessie) comes with cgit already in the repositories so a lot of previous tutorials that had you compile cgit are no longer necessary unless you really need the latest and greatest version of cgit.

Install some software

In order to run cgit with nginx, you will need to install cgit, nginx,
and fcgiwrap.

sudo apt-get update sudo apt-get install cgit nginx fcgiwrap

Configure your nginx.conf to work with cgit

The contents of /etc/nginx/nginx.conf file should look something like this:

worker_processes 1; 

events { 
  worker_connections 1024; 
} 

http { 
  include mime.types; 
  default_type application/octet-stream; 
  sendfile on;
  keepalive_timeout 65; 
  gzip on;

  server { 
     listen 80; 
     server_name git.example.com; # update to be your own domain
     root /usr/share/cgit;
     try_files $uri @cgit;

     location @cgit { 
      include fastcgi_params; 
      fastcgi_param SCRIPT_FILENAME /usr/lib/cgit/cgit.cgi; 
      fastcgi_param PATH_INFO $uri; 
      fastcgi_param QUERY_STRING $args; 
      fastcgi_param HTTP_HOST $server_name; 
      fastcgi_pass unix:/run/fcgiwrap.socket; 
      } 
   }
}

Configure cgit Edit /etc/cgitrc

Your file should look like this:

#cgit config
#===========
#see cgitrc(5) for details
#=========================

css=/cgit.css 
logo=/cgit.png 
virtual-root=/

Fire it up!

Now you are ready to see cgit. Restart nginx and navigate to your web server.

/etc/init.d/nginx restart

You should see the cgit home screen at your domain name or IP address
now.

Add some repos

You can add repos to your `/etc/cgitrc` file and they will show up in
cgit. The repo section of this file should look something like this:

# List of repositories. 
# This list could be kept in a different file (e.g. '/etc/cgitrepos') 
# and included like this: 
# include=/etc/cgitrepos 

repo.url=MyRepo 
repo.path=/srv/git/MyRepo.git 
repo.desc=This is my git repository 
repo.url=MyOtherRepo 
repo.path=/srv/git/MyOtherRepo.git 
repo.desc=That's my other git repository

Since cgit is packaged in Debian now, getting up and running is easier than ever. If you ran into any issues following this guide let me know in the comments.

Credits: I got a lot of inspiration from this guide via the awesome Arch Wiki. There were some minor changes that had to be made due to some configuration differences between Arch and Debian.

Standard
debian, linux

Using Virt-Manager on Debian

Virt-Manager is an awesome front end for running QEMU VMs on top of KVM. This is a great alternative to VirutalBox and since the latest version of Virtualbox has been moved to the “contrib” section in Debian (due to a non-free compiler that is required to builds the BIOS) it is one of the only 100% free software GUIs for managing virtual machines. To get virt-manager working debian you will need the following:

apt-get install qemu-kvm libvirt-bin virt-manager

Even though the [Debian Doc] states that adding your user to the kvm and libvirt groups will allow you to create Virtual Machines, when you try to do so you may be confrtonted with an error message that reads:

Could not access KVM kernel module: Permission denied failed to initialize KVM: Permission denied

This makes this application fairly useless so lets get this fixed. The problem is that libvirtd starts qemu with whichever user:group is defined in /etc/libvirtr/qemu.conf 1 by default this is nobody so this is why the error message occurs. In order to fix this, edit this file and add your username and group to this file. The file should end up looking something like this:

# The user for QEMU processes run by the system instance. It can be 
215 # specified as a user name or as a user id. The qemu driver will try to 
216 # parse this value first as a name and then, if the name doesn't exist, 
217 # as a user id. 
218 # 
219 # Since a sequence of digits is a valid user name, a leading plus sign 220 # can be used to ensure that a user id will not be interpreted as a user 
221 # name. 
222 # 
223 # Some examples of valid values are: 
224 # 
225 #       user = "qemu"   # A user named "qemu" 
226 #       user = "+0"     # Super user (uid=0) 
227 #       user = "100"    # A user named "100" or a user with uid=100 228 # 
229 #user = "root" 
230 user = "username" 
231 # The group for QEMU processes run by the system instance. It can be 232 # specified in a similar way to user. 
233 #group = "root" 
234 group = "groupname"

After this, restart the libvirtd service and you will be able to create virtual machines with virt-manager.

systemctl restart libvirtd.service

References: https://github.com/voidlinux/void-packages/issues/1095

Standard