Josh Bohde

Self Hosting Git

Posted by Josh Bohde

For awhile I've been pretty unhappy with hosting my code at Github. They have enourmous cultural capital, but use it irresponsibly, as evidenced by "harcore forking action", the meritocracy rug, and drinkups. The recent news about their sexist culture and treatement of Julie Ann Horvath made me look into alternative hosting options.

I looked into a few prepackaged options, such as GitLab or Gitorious, but settled on a combination of gitolite, gitweb, and git-daemon.

Throughout all the following examples, I'll be using git.joshbohde.com as my server, but you should change that to your own server.

Gitolite

Install gitolite was straight forward, I just followed the instructions from their docs.

Configuring gitolite took awhile, I ended up setting up wild repos. This let me create new repos by just setting the origin, and pushing.

After this is setup, you can run commands to interact with your repositories through ssh, e.g. ssh git@git.joshbohde.com help will show all the commands.

One thing that I spent a bit of time on was figuring out how to delete a repo. For this I had to uncomment the "D" command in gitolite.rc in the git users home directory. From there ssh <your_git_host> D -h will show how to delete a repo.

Gitweb

I installed gitweb directly from apt. From there I added the following config options:

$projectroot = "/home/git/repositories";
$projects_list = "/home/git/projects.list";
$strict_export = "true";
$export_ok = "git-daemon-export-ok";
@git_base_url_list = ('git://git.joshbohde.com');
$feature{'highlight'}{'default'} = [1];

This will display any repo in gitweb that the gilolite user gitweb can read. You can add these permissions via ssh <your_git_host> perms <repo_name> + READERS gitweb

There is a more advanced config for authorizing users, but I only care about public repos being shown.

Gitweb uses cgi, so you'll need a web server to actually serve it. I use nginx, with this config.

For the theme, I accidently stumbled upon gitweb-theme, and ran the install process

Git-Daemon

Installing from apt didn't work, so I wrote an upstart script based upon the docs.

Like gitweb, you can make a repo publicly cloneable by allowing the daemon user to read, e.g: ssh <your_git_host> perms <repo_name> + READERS daemon

Migrating

I manually migrated a handful of private repos, and used a script for all my public stuff.

I wrote a quick python script to parse Github's API output

import json
import sys
import pipes
 
for repo in json.loads(''.join(sys.stdin)):
print pipes.quote(repo['name']),
print pipes.quote(repo['git_url']),
print pipes.quote(repo['description'] or '')

And used it to generate a list of all my repos.

curl "https://api.github.com/users/joshbohde/repos?type=owner" | python convert.py >> repos

I had two pages of repos, so I had to do it again with the url "https://api.github.com/users/joshbohde/repos?type=owner&page=2".

I used the following script to clone a repo:

set -e
git clone --bare https://github.com/joshbohde/$1.git
cd $1.git
git push --mirror ssh://git@git.joshbohde.com/joshbohde/$1.git
cd ..

To clone them all, I ran cd /tmp; cat repos | xargs -L 1 bash clone.sh

To setup descriptions, and public access, I wrote the following script:

ssh <your_git_host> desc joshbohde/$1 "$3"
ssh <your_git_host> perms joshbohde/$1 + READERS daemon
ssh <your_git_host> perms joshbohde/$1 + READERS gitweb

To run this over them all, I ran cat repos | xargs -L 1 bash access.sh

Thoughts

I'm still looking for a good way to setup issues and pull requests, but I get so few of those that it isn't a high priority.

For backups, Pete Keen has written how to mirror to s3.

An immediate benefit I've noticed is that I'm no longer constrained by the low number of private repos, so I'm more likely to put something I don't want to be public in git now. I also was less affected by the recent Github outage.

The total size of my repos was around 60 MB. This would be an awesome use for a spare Raspberry Pi.