I have been wondering lately about the state of the Indian tech industry and why it doesn't receive much coverage on mainstream tech sites. I actually raised the question in an online discussion recently, with no response, so it was interesting to read Datawocky's post on an Indian application comparable with Twitter.
SMS GupShup is an Indian startup providing a similar service to Twitter, with much higher usage numbers and no downtime. Here are some stats comparing the two services:
Users: Twitter (1+ million), SMS GupShup (7 million)
Messages per day: Twitter (3 million); SMS GupShup (10+ million)
As you can see, SMS GupShup has 7 times the number of users and more then 3 times the number of daily messages, yet it reportedly doesn't suffer from the same downtime problems that have plagued Twitter lately.
Both applications run on Linux hardware, and both use MySQL as the database, so why does one scale so well, while the other falls over?
Beerud Sheth, one of the co-founders says the difference lies in the architecture, with SMS GupShup using a three-tier architecture, with JBoss app servers sitting between the webservers and the database, while Twitter uses a two-tier architecture.
"GupShup also uses an object architecture (called the "objectpool") which allows each task to be componentized and run separately - this helps immensely with reliability (can automatically handle machine failure) and scalability (can scale dynamically to handle increased load). The objectpool model allows each module to be run as multiple parallel instances - each of them doing a part of the work. They can be run on different machines, can be started/stopped independently, without affecting each other. So the "receiver", the "sender", and the "ad server" all run as multiple instances. As traffic scales, they can just add more hardware — no re-architecting. If one machine fails, the instance is restarted on a different machine.
No amount of caching can help a 2-tier read/write application scale. The middle-tier enables the database to be sharded, and that's what gets you the scalability. I believe Twitter has recently started using message queues as a middle-tier to accomplish the same thing, but they haven't partitioned the database yet — which is the key step here."
Could be an avenue worth investigating for Twitter.