[Solved]-Realtime server push with Socket IO (or Strophe.js), XMPP and Django

I think once you get your hands dirty with some node you’ll find that straying from Node for socket.io is going to be much harder. There are very easy to use xmpp modules in node ready to go (see https://github.com/astro/node-xmpp). Remember, node is all javascript, so you’re probably familiar with programming in it already.

Personally, I’ve had some memory leak issues using node 0.6 or higher. Node 0.4 worked without those issues. If you are new to github (as I was before playing with Node) here is how you would get going with a node server.

Getting Node

1. Login to your linux box and favorite directory (I’ll assume /)
2. git clone https://github.com/joyent/node.git
3. cd /node
4. git tag -l (this will list all available version of node)
5. git checkout v0.6.16 (this will checkout 0.6.16 version of node, you could replace that with v0.4.12 for example if you have memory issues)
6. ./configure
7. make
8. make install

You’ll need certain development tools to build it such as g++, but at this point you’ll have a working node command.

Installing Node Modules like xmpp

Node has a nice amount of modules where most things have already been written for you. There is a search facility at http://search.npmjs.org or you can access all modules directly from your shell by using the npm command. NPM is nodes tool for installing and managing node modules. You can type npm search xmpp to search for all xmpp modules, for instance. To install a basic xmpp library for node you would do npm install node-xmpp. By the way, most github node module pages will include instructions on the front page readme file.

Keeping Node Running in Production

This threw me when I first started out. If you have any errors that are not caught node will simply die. So, you can either
1. Make sure there are no errors whatsoever or they are all caught (unlikely because even Node itself will error)
2. Use the uncaughtException handler to catch these problems. You would use code like this in your program

process.addListener("uncaughtException", function (err) {
    util.log("Uncaught exception: " + err);
    console.log(err.stack);
    console.log(typeof(this));
    // maybe email me?

});

Be Extra Safe and Use Forever

Even with the uncaughtException issue your program in production might die. Memory running out, segfaults, who knows what. That’s where it pays to use something like the wonderful Node module called “Forever” (see https://github.com/nodejitsu/forever). You can type npm install forever -g to install forever. Note the -g option which puts forever in the GLOBAL node module directory. Without -g it puts the node module in the current working directory. You’ll then be able to type something like (assuming your node program was called my_program.js) forever start my_program.js and then the Forever program will make sure that if it dies it gets restarted.

First of all, full disclosure: I work for a company called PubNub, which I’m going to mention shortly.

There are a whole range of hosted bidirectional messaging services (sometimes called IaaS – Infrastructure as a Service) that I think are worth considering. They are Pusher, Firebase, Flotype, PubNub, and others. I’m reasonably confident you could use any of them for what you want to accomplish. Firebase has a built-in database that ties right into their service, which is a pretty cool feature, but probably not useful for your particular use case (I assume you already have a database on your backend).

I can’t speak too heavily about our competitors, but as far as you wanting a JavaScript library on the frontend that communicates with your Python backend, we (PubNub) provide a very similar api in both languages and that communicate on the same databus in the cloud. So you can send messages with Python and catch them with JavaScript, or vice-versa. We even wrote a PubNub-hosted version of socket.io, which you could use instead of our vanilla JavaScript api, and would still tie into your Django backend in about 10 lines of code.

Finally, the nice thing about using an IaaS (or at least us; again I’m not certain about the others) is that we handle that tricky scaling problem for you. If you reach the point of a million simultaneous users and need to push something to them in real-time, you’ll find that’s no problem.

We are using real-time push as well with Django and Celery. When I first created the architecture, I also researched my options. Eventually, I decided that I’d rather focus on getting the app just right rather then on fiddling around with devops work. There are several services out there that offer hosted real-time push technology that can be easily integrated with any app.

I chose PubNub and I couldn’t be happier. They support socket.io for the client side and have a Python lib I use from Django and Celery workers. They also have SDKs you could use from native mobile apps.

I know, you already have a working setup in place. But I’m betting that the time it will take you to replace your current setup with such a hosted solution would be less than the time it will take you to find a good solution for what you’re looking for and implement it. Also keep in mind maintenance costs down the road (esp if you opt for a lib which is not well maintained).

True, you will be paying for the service, but they price is very reasonable and you will be getting a solid service with nice perks like colocation.

I’m not affiliated with that company, just a happy customer. There are other similar services out there.