Python Version Management with Pythonbrew

pythonbrew is a Python version manager that allows for local Python installations. This is necessary because 1and1 gives us quite an old (2.6.6) version of Python, meaning that we either have to have the old version on a local machine to do local development, or we need to be able to switch into newer versions of Python on the server. Knowing how to use pythonbrew solves both of these problems.

Deprecation Warning

At the time of writing, PyPi lists pythonbrew as no longer under active maintenance. Users are encouraged to look into pyenv as an alternative. However, at present this project does not look very mature, and pythonbrew continues to work. It is probably worth changing over in another year or so.

Installing Pythonbrew

Installation of pythonbrew is simple:

1. Grab and install it with the following command:
   

             curl -kL http://xrl.us/pythonbrewinstall | bash
         

   This installs pythonbrew in ~/.pythonbrew

2. Add the following line to one of the bash startup scripts - preferably .bashrc
   

           [[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc
         

   This initializes pythonbrew on ssh login. On the current server, this line is actually in ~/.bash_profile; the effect is the same.

using Pythonbrew

Detailed information on using pythonbrew can be found on the project page.

Some basics include the following:

1. pythonbrew list -k (list all Python versions available for install)
2. pythonbrew list (list all Python versions currently installed)
3. pythonbrew install 2.6.6 (install Python 2.6.6)
4. pythonbrew use 2.6.6 (switch Python to version 2.6.6 - assuming it is installed - for this shell session)
5. pythonbrew switch 2.6.6 (mask system Python with version 2.6.6 "permanently" - aka until you tell pythonbrew to use something else)

Virutal Environemnts

It is possible to use pythonbrew to set up virtual environments for specific projects. More detailed instructions are available here, but here are some basics:

1. pythonbrew venv init (activate virtual environment support)
2. pythonbrew venv create newvirtualenv (creates a virtual environment with the current Python callednewvirtualenv)
3. pythonbrew venv use newvirtualenv (switch into newvirtualenv)
4. deactivate (exit newvirtualenv)
5. pythonbrew venv delete newvirtualenv (delete newvirtualenv)
6. pythonbrew venv list (list all currently-existing virtualenvs)

Using installed Pythons Outside of the Virtualenv

While pythonbrew is quite a powerful tool, there is one issue it doesn't address directly: how do we get scripts that are run by the webserver to execute in a given environment? After all, if we want to run a cgi program in Python, Apache will call it using the system Python - 1and1's hopelessly out of date version 2.6.6 - which just puts us back to square one.

This isn't a problem if you understand that all that pythonbrew really does is change the system environment very slightly so that the system first checks our chosen locations for commands before looking in the usual places. So, if you know where the installed Pythons are and you know how to get the system to look there first, everything should work as though you had set up the virtual environment.

The following two steps must be taken:

1. Make sure that the entry point of your cgi program gets executed with the userspace virtualenv Python you want rather than the system one. You can do this with a shebang line. At the time of writing, the way to use Python 2.6.8, for example, instead of the system 2.6.6 (admittedly a trivial differencet) is:
   

       #!/kunden/homepages/43/d109612362/htdocs/.pythonbrew/pythons/Python-2.6.8/bin/python
       

   Just put that line at the top (yes, it absolutely MUST be the first line!) of your cgi script source file, and Apache should execute it with the proper python.

2. Somewhere near the start of your script, modify Python's internal search path to ensure that the installed packages in the virtual environment are found before any identical system packages. At the time of writing, the way to do this for Python 2.6.8 "global" installs is:

     import sys
     sys.path.insert(0, '/kunden/homepages/43/d109612362/htdocs/.pythonbrew/pythons/Python-2.6.8/lib/python2.6/site-packages')
     

   As you can see, the search path is included as a Python list as the path member variable of the sys module. So, you first import this module, then add the path to the relevant site-packages folder as the first item in the list (to ensure that it's found first in any search Python does for modules).

pkg_resources Installation Guidelines

Within a pythonbrew venv you may occasionally get the following error:

NameError: name 'install' is not defined

This is caused by a mismatch between the pythonbrew and system Python setup environments. Basically, pythonbrew can't find a module called pkg_resources on account of our having clobbered the system Python.

Fortunately, there's a quick workaround. As per this StackOverflow page, you can do the following:

curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py

That should take care of it. Note that this has to be done separately for each version of Python you use.