Candidate technologies
Some technologies to consider for the new task system
Job system + RabbitMQ
Extend the current job system to be a daemon that listens to RabbitMQ for notification of pending jobs.
Job system + ampoule
Extend the current job system to be a daemon that listens for AmpQ messages for notification of pending jobs.
Celery
Gearman
Beanstalk
http://kr.github.com/beanstalkd/
Kestrel
http://wiki.github.com/robey/kestrel/
UEC
We could use UEC to manage our worker machines, but this does not support non-virtualizable architectures. However, it could allow us to take advantage of our internal UEC cloud, and might make it feasible for individuals to run their own buildfarms as a test.
Landscape
We could use Landscape to manage our worker machines.