Magnum is an HTTP server that I've been working on and decided to create an open-source project for it. The aim of the project is correct the misguided direction of modern web serving software. By focusing on efficient resource management down to the operating system level, Magnum can run faster than most servers written in compiled code (such as apache), can handle more simultaneous open keep-alive connections, and works well with slow clients. Because it's written in python, it's short and sweet, extensible, and integrates well with python web frameworks such as Django. The easiest way to explain its architecture is with this diagram:
A similar python-based webserver called Tornado was recently released by Facebook, and has many similar features such as epoll. I decided to release mine anyway since it has numerous advantages. For one, Tornado is a single-process web server, and they recommend using nginx as a reverse proxy on top of several instances of Tornado to use all the available resources on a machine. Magnum's multi-process approach is fully integrated and there are no unnecessary middle-man sockets tying up kernel resources.
Secondly, since Tornado only has one process, it cannot support blocking web app interfaces such as WSGI without turning off the epoll features that make it interesting. Magnum's non-blocking master process simply offloads every request it gets onto a shared memory queue, the worker processes can take however long they want to process each request, and the master process just picks up the response and sends it off when the worker is done. Magnum supports WSGI out of the box, and in fact mattgattis.com is now proudly serving this page with Magnum + Django over WSGI.
Anyone interested is welcome to contribute. I'll follow up with benchmarks.
Posts
WSGI support out of the box is great thing - I'll try Magnum on my Django projects.
I was just about to rewrite the entire project since it was just a proof of concept however this code looks a lot cleaner. I will connect with you to try and contribute.
Or at the very least it should be a lot less efficient than the model used by Tornado.
Also these days the biggest challenge to my mind is handling long-running connections for Comet, websocket etc... And a separate process per connection is bad news when you get into the thousands or more!
Also, long-running connections are a specialty for magnum. Magnum doesn't launch a separate process per connection. There is simply a queue in shared memory and a fixed number of worker processes pull requests off the queue. This is one of the main purposes.