x | thought

For those who haven't done so already, now is the right time to read git in five minutes, and maybe take a look at the stack question git for beginners. Github itself also has a nice help page

We will be using github to host our repository, at https://github.com/ashwoods/hospex. Linking our code with the github account is easy with git-remote add, then we commit our code to the local repository and push to github :

git init  
git remote add origin git@github.com:ashwoods/hospex.git
git add *
git commit -m "initial commit"
git push origin master

Actually, 'git push' would suffice in this case, but you can have more than one remote linked to your local repository, and this is how you specify the remote and branch you want to commit to.

Voila! The project is now available on github: https://github.com/ashwoods/hospex for you to fork and modify as you wish.

So, now lets tweak or git settings:

First, we tag our code with a human readable tag:

git tag -a -m "Version 0.1" v0.1 HEAD

The command 'git describe' now outputs v0.1, and we will use this command for a pre-commit hook, that will print out a nice version number each time someone commits (read more about it here)[http://stackoverflow.com/questions/514188/how-would-you-include-the-current-commit-id-in-a-git-projects-files/514668#514668]:

 0.1-5-g0c9d-mod
(version tag)-(commits since tag)-( g+5 sha1 id)-(if tracked files are modified)

The pre-commit that we place under .git/hooks/pre-commit looks like this:

We can use this to include the VERSION in our python code. As a start, we will use this as the VERSION in our setup.py file by stripping the version from our VERSION file like this:

VERSION = open('VERSION').read().lstrip('version: ').rstrip('\n')

and we use it instead of hardcoding a version.

Our last step for now, is adding a .gitignore file, that automatically ignores file that we, well, want to ignore. I use VIM and pycharm as editors, and python creates the *.pyc files automatically.

Later on we will come back to git, discussing possible workflows. Git is very flexible and can probably adapt to the wildest workflow of your dreams, but there seem to be two popular workflows (that I am aware of). One is git-flow, and then there is the the "always commit & deploy to master and deploy often", (can't find the blog post describing it where I read it right now), that is mabye better if you deploy often. But we will get back to git workflows later, right now, we want to start writing some django code that works. Coming up next: modelling our django (models).

A simple django project layout consists of not much more than a few files:

• urls.py
• manage.py
  
• settings.py
  

and maybe an app that consists of the following files:

• models.py
  
• views.py

Although there is a little bit of magic in django file naming and structure, it's all python, and as long as you follow the few conventions that django needs to function, you can order them in standard python modules and libraries than can live anywhere in your filesystem as long as the directories holding them are defined in your PYTHON_PATH.

Actually, you want to create a project that consists of small self-contained, generic and reusable components, so trying to keep your actual django project as small, lean (and empty) as possible is one way of achieving this. Anything that can be used generically is spinned off as it's own resuable app and lives outside of the project structure. We will have a folder called apps for our project specific apps, and a templates folder for our basic html templates. The folder static will hold our static media files, and I will be splitting the settings file in parts and placing them in the settings folder.

Furthermore, during development and for deployment, we want our project to be a python package, that we can install using pip, a tool for installing and managing Python packages. We achieve this by structuring our project in a certain way, and adding a setup.py file. We will add a bin folder for custom scripts, so if the root folder of our project is called django-hospex, our initial result will look like this:

django-hospex/
- hospex/
 -- apps/
 -- bin/
 -- -- manage.py
 -- settings/
 -- static/
 -- tempates/
 -- urls.py
 -- settings.py
 --  __init__.py
 - setup.py
 - fabfile.py
 - __init__.py

In our case our setup.py file does little more than define some metadata and install the scripts under bin. You can do more in the setup.py file, like configure the metadata that is needed when you want to upload a python package to pypi, the python package repository.

The fabfile will hold our development and deployment fabric commands, used to automate common tasks.

Notice we moved the manage.py file under bin/. I learned this trick from lincolnloop, a cool django shop that have been opensourcing a lot of there know-how. Once we install our project using pip, the manage.py file will be placed on the system PATH and you will be able to call it from anywhere. We will be using virtualenvs to contain python environments, so having several installed won't be a problem, but more on that later.

Let's take a look at the setup.py file:

from setuptools import setup, find_packages

setup(name='django-hospex',
     version = "0.1",
     packages=find_packages(),
     exclude_package_data={'rh2': ['bin/*.pyc']},
     setup_requires = ["setuptools_git >= 0.3",],
     scripts=['hospex/bin/manage.py'])

That's all what is needed to turn a set of python modules into a package. We call setup from setuptools with our project metadata, and tell it where to find the files it needs to include using find_packages. We include setuptools_git, as it helps find_packages find what it needs when you are using git.

We also modify the manage.py script that comes with django out of the box, enabling it to find settings (as manage.py now resides in ./bin (if you have skipped the django tutorials, don't worry, you will see what we use manage.py for later):

##!/usr/bin/env python
import os
import sys
from django import get_version
from django.core.management import execute_from_command_line, LaxOptionParser
from django.core.management.base import BaseCommand

# Work out the project module name and root directory, assuming that this file
# is located at [project]/bin/manage.py
PROJECT_DIR, PROJECT_MODULE_NAME = os.path.split(
            os.path.dirname(os.path.dirname(os.path.realpath(__file__))))

# Check that the project module can be imported.
try:
    __import__(PROJECT_MODULE_NAME)
except ImportError:
    # Couldn't import the project, place it on the Python path and try again.
    sys.path.append(PROJECT_DIR)
    try:
        __import__(PROJECT_MODULE_NAME)
    except ImportError:
        sys.stderr.write("Error: Can't import the \"%s\" project module." %
                         PROJECT_MODULE_NAME)
        sys.exit(1)

def has_settings_option():
    parser = LaxOptionParser(usage="%prog subcommand [options] [args]",
                             version=get_version(),
                             option_list=BaseCommand.option_list)
    try:
        options = parser.parse_args(sys.argv[:])[0]
    except:
        return False # Ignore any option errors at this point.
    return bool(options.settings)

if not has_settings_option() and not 'DJANGO_SETTINGS_MODULE' in os.environ:
    settings_module = '%s.settings' % PROJECT_MODULE_NAME
    os.environ['DJANGO_SETTINGS_MODULE'] = settings_module

execute_from_command_line()

Good! Now that we have a simple folder structure, let's go on to creating a git repository.

Where to start?

Django is a webframework for python. Obviously, some knowledge of python is an advantage, but python is easy enough to understand if you already know another programming language. For developers in other languanges I recommend reading Mark Pilgrms excellent dive into python, it's freely available as pdf. Basic *nix command line skills are a must, but you can pick them up as we go along as well. HTML, some javascript, and basic http knowledge is also a must.

The first stop for anybody wanting to learn django, is the official 4 part django tutorial. I will assume you already have python installed and I will begin this tutorial from where the official one ends. Have your editor of choice ready (I use vim and pycharm), and let's dive in!

Tutorial structure: Our next steps...

We will start by creating a sane project layout for larger projects, and setup a basic development environment. We will be using git as our version control tool and the project will be hosted on github. In part 2 we will get our hands a little bit dirty with django and define our first models for the project. In part three we will checkout 3rd party apps that will want to add to our project. In part 4 we will introduce vagrant and puppet as development tools.

Where to find help

Django has excellent documentation and a vibrant community that is very supportive. Your best tools in learning django will be the django documentation, stackoverflow, and of course the web is full of tutorials, screencasts and blogs that you can find with google. If you get stuck with a problem, just hop on the django IRC channel where you normally find a lot of helpful ears (btw: I am kyrix on #django). Please do try to take your time to read the docs before asking!

Knowing your tools

So, in a nutshell: Programming is a craftsmanship, and spending time to learn to use your tools well will pay off:

• editor: no matter what editor/IDE you use, spending a day to learn how to use it well will boost your productivity.
• vcs: I will be using git. It's better to know how to use more than one. take a look at git in five minutes and the git resources mentioned on this stack question
• finding help: docs, stackoverflow, google, mailinglists, irc.
• python: I'll be addressing python resources later.

Enough babling, lets get started with part 1: setting up the project layout.

I was a vivid couchsurfer. I had excellent experiences and has changed the way I travel for ever. But while the community was growing, the site had been lagging behind. I thought about contributing to the code base, but the site is closed sourced and the official position of the organization was against opensource.

Years have passed by and now couchsurfing has changed it's status from a non-profit to a B-corporation. While this move in itself might not be as bad as it sounds, it seems that couchsurfing has a long history of in-transparency and mismanagement, enraging many volunteers and people who have donated to the non-profit.

Many have fled to bewelcome.org, a registered non-profit developing the site with opensource tools and open to everybody.

It currently runs on rox/php and there is a drupal/php branch in the works. Momentum has picked up since couchsurfing went non-profit, and I decided to chip in, in a way.

Drupal is a great CMS with framework capabilities, but I considered that a site like bw should be built with a framework, with several services that are as decoupled as posible, as my experience with CMS is that in the long run you spend more time tuning and bending the CMS as wrtiing productive code.

My goal is to write a generic hospex application based on django/python with a tutorial documenting django best coding practices. Some of technologies I will use:

• lang, framework: python/(geo)django
• webservers: gunicorn + nginx for static
• databases: postgresql + redis
• cache: memcached
• css: compass, but not sure what css framework yet
• javascript: jquery, backbonejs (probably)
• additional dev tools: vagrant+puppet+teamcity+selenium+cucumber
• search: elasticsearch

The project will not be bw specific, and could be easily used by any hospex site as a code basis. I will talk about every aspect of modern web programming that I use daily: test automation, version control, caching, common development and deployments tools like vagrant, fabric, puppet and chef.

So head on the first post if your ready.

5:30 in the morning and I just finished a django project in under 3 hours from scratch*, all the way to deployment on webfaction (referal link). I've been using agilezen for a week now and I'm thrilled. I wanted to document from beginning to end how to deploy on webfaction, and here goes the first part.

Preface: Webfaction rocks!

Webfaction combines the power of a virtual hosting solution and the convenience of a traditional hosted solution. You have all the tools you need, and it is easy to install your own if there is something missing, but at the same time they take care of keeping the system secure and up to date. They have really competitive prices and plans for all the goodies they offer, and their support rocks. But I have no need to ramble about how good they are, the web is already full of good reviews, and surprisingly few negative ones.

Chapter 1: Setting up your python environment and installing deployment tools.

Webfaction has excellent python support, with versions 2.4, 2.5, 2.6, 3.0, 3.1 installed by default, and if you need anything else it is easy to install it yourself.

We will start by setting python 2.6 as system default, and install the tools we will need to deploy our app: virtualenv, virtualenvwrapper, pip, subversion, git and mercurial.

Python

Set python 2.6 as system default, and install virtualenv, virtualenvwrapper, pip and mercurial:

a) Create python 2.6 lib directory:

mkdir /home/{{accountname}}/lib/python2.6

b) add the following lines to your bashrc file:

alias python=python2.6 export LD_LIBRARY_PATH=$HOME/lib

c) install pip, making sure you use easy_install 2.6, and install virtualenv.

$ easy_install-2.6 pip $ pip install virtualenv virtualenvwrapper mercurial $ mkdir $HOME/virtualenvs

d) add the following lines to your bashrc file:

VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python2.6 export WORKON_HOME=$HOME/virtualenvs source /home/{{accountname}}/bin/virtualenvwrapper.sh

Version Control Applications

Subversion is installed by default, and to install git you can either install it from webfaction’s app installer or manually:

$ wget http://www.kernel.org/pub/software/scm/git/git-1.7.x.x.tar.gz $ tar zxf git-1.7.x.x.tar.gz $ cd git-1.7.x.x $ ./configure --prefix=$HOME $ make $ make install

Installing mercurial using pip is easy too: pip install mercurial

So now you have everything you need to start deploying your django apps on webfaction. In the next part I'll show how to setup django using two setup methods: the traditional mod_wsgi+apache way, and the cutting edge gunicorn+nginx way :) Time to go to bed.

Resources: http://docs.webfaction.com/software/python.html http://docs.webfaction.com/software/git.html

I've worked for several NGO's for the last 10 years. I was very active with environmental groups as an activist and I've had the honor of visiting several jails all over europe for it. And even though I still think it is important and necessary, I have had a shift in thought in the last 6 months and was looking for new ways of being more productive, and I came to the conclusion that one of the best ways of helping is out is by doing what I can do best. In my case, that's in the field of computer engineering. Of course I'd love to spend some time with a cool company like Inveneo out in the field, but sometimes it isn't possible. And not everybody can up with something as cool as Samasource. I had already been donating time to opensource projects, why not donate 5% of my time helping out in different NGO or "opensocial" projects?

This February I am leaving for Morocco and I will be spending some time in the Atlas mountains with a friend of mine who teaches at a village school near Amizmiz. Besides climbing some beautiful mountains and learning how to cook Moroccan food I'd like to spend some time teaching the kids how to play ultimate.

That's why I need some help from the Vienna (and surroundings) disc community to help out with old discs. I know most of you have more than 30 disks laying around :) You don't have to sacrifice any of the discs that are hanging on the walls. They can be used but should still be playable. Just imagine the berber children playing with the discs in the wind with a big smile on their faces.

>

Why frisbee?

Discs are cheap, and nearly indestructible. I know, they do wear down a bit and street discs can be a little hard on the hands, but they are still playable for a very long time. From discgolf to ultimate frisbee, there are dozen of games that can be played from solo to large teams, suitable for almost any age, and that are playable without the need for extra equipment.

Why Ultimate?

Besides being one of the coolest team sports on the planet, that has a great community around it, a sport that is played on grass, sand, rock, pavement... and in the wind, sun, rain, snow... where you run like hell chasing rotating plastic thingies, it's because of the Spirit. For those who play know exactly what I am talking about, and for those who don't:

Ultimate relies upon a spirit of sportsmanship that places the responsibility for fair play on the player. Highly competitive play is encouraged, but never at the expense of mutual respect among competitors, adherence to the agreed upon rules, or the basic joy of play. Protection of these vital elements serves to eliminate unsportsmanlike conduct from the Ultimate field. Such actions as taunting opposing players, dangerous aggression, belligerent intimidation, intentional infractions, or other 'win-at-all-costs' behavior are contrary to the Spirit of the Game and must be avoided by all players."

Spirit of the Game sets Ultimate apart from other competitive team sports. For over 30 years, Ultimate has flourished, reaching a highly competitive level, without the use of referees.

And it really works. Ultimate rules promote the resolution of conflict through dialogue and in a civil manner. It allows you to learn to be competitive and respectful during play. The ultimate community is like a big family, we only play against each other on the fields, but as soon as the game is over, it's time to party with the other team, and it's always fun.

And if all goes well, in a couple of years we might be visiting the 1st ultimate tournament in the Atlas Mountains :)

cheers!

Django-mingus is more or less a collection of reusable apps nicely put together, but in itself django-mingus is not a reusable-app. In this sense django-mingus is more like a project template than an app. I don't really like modifying 3rd party code if I don't have to, so this is how I installed django-mingus using virtualenv and virtualenvwrapper on webfaction and set up a project using django-mingus as an app. I must say that I started around 3'oclock AM and I just wanted to get it working somehow. I assume you already know how to setup a wsgi app on webfaction and used virtualenv and virtualenvwrapper before.

I found a very useful post on how to use sites with virtualenv on webfaction. Just follow the instructions there on setting up virtualenv and virtualenvwrapper, and take a look at its apache config file later if you don't know how to setup django wsgi on webfaction.

1) Install Django-mingus

mkvirtualenv mingus —no-site-packages 
workon mingus
cd mingus
easy_install pip
git clone git://github.com/montylounge/django-mingus.git

2) Setup project in webfaction control panel

Create a django-wsgi app, create the domains, and link the apps.

3) Setup project for using mingus

a) Copy the init.py from mingus to your project and comment out the mingus one. This is the only change I had to do to mingus code and probably solvable by changing setting environment variables.

b) In your settings file add the line:

from mingus.settings import *

Then for everything that is specific for your project, just override it in the settings file like this:

#settings.py

INSTALLED_APPS += (
    'xthought.apps.blog',
    'xthought.apps.code',
)

ROOT_URLCONF = '%s.urls' % (PROJECTNAME)

I usually have a settings.py file where all the project specific constants are defined, and then i have a config.py file that is deployment specific, some prefer to use the local_settings.py trick. Either way, you have to override mingus.local_settings.py definitions too with your prefered method. Now you are ready to sync the database with syncdb :)

Now its time to create a wsgi file:

import os, site, sys
# add the virtual environment path
site.addsitedir('/home/USERNAME/.virtualenvs/YOURPROJECT/lib/python2.5/site-packages')
# fix markdown.py (and potentially others) using stdout
sys.stdout = sys.stderr
#Calculate the path based on the location of the WSGI script.
project = os.path.dirname(__file__)
workspace = os.path.dirname(project)
sys.path.append(workspace)
os.environ['DJANGO_SETTINGS_MODULE'] = 'YOURPROJECT.settings'
from django.core.handlers.wsgi import WSGIHandler
application = WSGIHandler()

and modify the apache config as usual to load the wsgi file with a WSGIScriptAlias command.

Setting it up this way lets you have several projects use the same virtualenv environment (if that is desired), and also makes it easier to upgrade django-mingus code without having to apply patches everywhere :)

A long time ago, when I had too much time, I had a blog where I kept family and friends that were far away up 2 date. Now that facebook, twitter and Co. do that much better, I just kept the "blog" as my web playground to try out new stuff. I'm working on a django powered e-campaigning platform for a friend of mine and decided to try out a couple of django blogs. Currently this baby is running django-mingus and is hosted on one the best providers out there and of course comes with ponies with magical powers.

I will mostly be posting geeky stuff here, so for those not interested in django/python and arduinos will be better served in other places, at least until i develop the other parts of the site that cover some other hobbies: photoblog and photojournalism projects, environmental work and ultimate frisbee. If I ever decide to post any of the other stuff I write again it will also appear here someday.

As pointed out on baratrion.org and like many developers out there I have spent more time working on blog software as writing content, and that is probably not going to change that much. What I do want to do is start documenting what I do for work and I will start out tomorrow with how I set this up on webfaction using virtualenv/virtualenvwrapper but without having to alter the mingus code so that I can have more than one instance using it.

I know I should link to some famous jazz musician as django tradition dictates, but I am hooked on this song:

>

Cheers and happy new years!