Roughly 2 weeks ago I started Plug which aims to create a package format for Python daemons. The project started after seeing how Supervisor handles 150+ processes.
A current project at work can easily have many daemon processes with differing number of running instances that may need to be adjusted frequently. Deploying with Supervisor can be a problem given the amount of time Supervisor would take to start/stop processes.
Plug installs each package into a virtualenv then uses runit to manage each daemon instance.
I have a prototype version working now with a packaged version to come in the next few weeks after giving it more testing.
The biggest issue is after watching To Package or Not to Package I am falling in more of the "to package" crowd and despite Plug being a packaging solution smells a bit too much of NIH.
I've kept my VIM/dot files online for a while in my Github but I recently spent some time to update my .vimrc file.
One of the changes that bugged me on OSX is that it ships with VIM 7.2 which doesn't have ColorColumn support. I like highlighting the 80th column in Python. As I discovered, the code to do this is :
{% highlight vim %} if version >= 730 autocmd FileType python set cc=80 hi ColorColumn ctermbg=darkgrey guibg=darkgrey endif {% endhighlight %}
Another change was to add Gundo support. This adds a window to navigate your undo tree, an incredibly useful feature.
I've pushed the first version of Plug. You can download it with pip
pip install plug
This is mostly to help me manually test Plug by using it in a few of my own projects and getting together a list of glaring issues.
Release v0.1.1 adds an uninstall command to Plug that takes a --plug=
option and removes the virtualenv and all runit links.
You can get Plug on PyPi and try it out. As always, report any issues.
A few weeks ago at work we improved our code review process by using Git more effectively. Previously a code review happened after the topic branch was merged into master. This obviously was not very effective as changes could have broken master without a proper review and there was less incentive to perform as careful of a review since the code was "working" already. This was carried over from when we were using SVN until we realized we were no longer forced to work in the dark ages.
Since we were already using Git we could easily change our workflow for a better review process. Once a topic branch was ready for review we could push a remote branch. Our remote branches take the form /review/{initials}/{topic}
To push a new remote branch:
{% highlight bash %} git push origin {branch name}:/review/{initials}/{branch name} {% endhighlight %}
And then we would move the Kanban card to the review column and find someone to review our changes.
When the code has been reviewed and any necessary changes made the reviewer will merge them into master.
{% highlight bash %} git checkout master git merge --no-ff --no-commit /review/{initials}/{branch name} git commit -s {% endhighlight %}
The merge command turns off fast-forward merging and commiting so when we commit with a -s we can sign-off on the changes. This shows who reviewed the code. We don't permit anyone to push their own changes without review although Git doesn't prevent you from changing the committer or the sign-off.
Then finally remove the remote branch by pushing an empty branch over it and delete your local copy of the branch.
{% highlight bash %} git push origin :/review/{initials}/{branch name} git branch -d /review/{initials}/{branch name} {% endhighlight %}
I've pushed a new PyPi package TestHTTPServer based on some work at AWeber where I needed to test processes that make web requests.
This probably shouldn't be used right away as it was from memory that I created the package. It will be expanded to record more of the request data and handle more requests.
I've been using Google Music quite a bit lately. It was rather painless to setup although the upload process can take a while. The service works out well with my Macbook Air since I only have a 64GB SSD and don't want to fill my drive with music I may or may not be listening to very frequently.
The only issue for me is that the Flash performance can frequently cause studdering if I am doing too much in Chrome at the same time.
I've added categories to posts although you can't really see them on each page due to an annoying quirk with Jekyll where you can't get the category for a post unless you are listing all of the posts.
Github also doesn't seem to load the pluging properly...
Release v0.1.2 includes the ability to set custom response headers and the __version__ package attribute! This release should be "complete" for my own needs and for common uses. If you see anything you need open an issue and I can probably get it updated and packaged in a day or so.
After I use the package a bit more I will bump the version and switch it to production status.
This release fixes a bug cause by linking all plug service instances to the installed plug. Runit uses a ./supervise in the service's directory to maintain state which would be clobbered when multiple services link to the same plug.
Now the virtualenv is copied to /srv/plug/plug_instances/ and linked into runit.
There is also a fix for uninstalling plugs leaving orphaned processes. Now Plug will stop the processes before removing them to prevent this.
I've push v0.1.1 of TestHTTPServer. This release adds the ability to handle any method as well as storing request headers and content for all methods.
For v0.1.2 I should be adding more to the server reponse options.
Since I heard about the addition of branch coverage tracking for Coverage I've wanted to give it a try. Originally it required a beta release which somehow I never got working.
Once it was in a normal release I somehow forgot about it. There is still no commandline argument to turn it on when using Nose. You can however use the .coveragerc file to enable it.
In .coveragerc simply put
{% highlight bash %} [run] branch = True {% endhighlight %}
And next time you run Nose with coverage you'll have branch coverage too! I was finally reminded about this when coming across the Test Coverage Analysis post by Kai Lautaportti.
Part of a current project at work involves writing many stand-alone processes to handle events from RabbitMQ. The worker processes are managed by Supervisor but should gracefully handle SIGTERM and generally follow a common pattern.
The main design is splitting out how worker's should run from what they are supposed to be doing. Extending the BaseWorker class will give a main classmethod used to start new workers.
The BaseWorker class is split from the WorkerRunner class giving a common interface for all workers and Unixy interaction.
You can take a look at what I have so far on the projects page.