June 06, 2007

Kelsey update

Where do I begin?

Many of you no doubt have already heard that Kelsey Smith was found murdered today. My wife, Laura, was at the Smith home when the detectives came with the news that a body had been found and it was likely Kelsey. The identification was later confirmed. Laura and my daughter, Beth, are attending a prayer service at Hillcrest Covenant Church as I write this. Prayers are flooding in from across the country and around the globe. Please add your prayers to them. I will have much more to say about this in the coming days.

Quite aside from the unspeakable human tragedy, my technical team and I learned a great deal through this. The traffic to www.findkelsey.com (see mirror of original HTML site) was 160,000 page views yesterday and grew exponentially this morning. By 10:00 am we had serious performance impact, essentially taking down Vine Hosting, the ISP where our web servers are located. Searches for findkelsey.com reached #5 on Google as reported by Google Trends. You read that right. Our little hand-built quickie site was the #5 search on Google today. (Right now "Kelsey Smith" is #3, "find Kelsey" is # 7, and "findkelsey.com" is #9.) The flood of legitimate traffic was so great, Glenn - the director of operations for Vine Hosting - at first thought it was a DDOS. In reality, it was all legit traffic. Too much traffic. Glenn is still dealing with the aftermath.

The only people with enough resources to absorb this kind of traffic is, that's right, Google. Realizing the disaster we were causing for Vine, I made the command decision to totally take down all traffic related to findkelsey.com at Vine, even the DNS. Then, as rapidly as possible, we created a new site on Blogger (which is owned by Google), changed to a high-volume DNS host, and pointed the DNS at Blogger. As soon as the DNS change propagated, we started getting comments on our posts. In fact, my first indication that we finally had things working was when a comment appeared in my inbox. It took almost another hour to figure out how to get most traffic coming to us as it should. In the following 2.5 hours, 754 comments have come in from around the country, directed to Kelsey's family. That's one comment every 10-15 seconds. Unbelievable.

Today I had the experience of building one of the highest traffic sites on the Internet. I was building it WHILE it was being hit.

5 comments:

David S. said...

That's pretty amazing Clif! That sounds like one heck of an experience. Sorry it had to be under such circumstances :-(

Dean Lisenby said...

Clif...sorry to hear about Kelsey. I certainly feel for this family. I can't imagine the pain they're experiencing. Don't underestimate the comfort your family will bring them.

As for the technical stuff, I know you've learned alot. I hope you'll share more about how you would change it if you had to do it again. Great effort!

Jimmy Williams said...

Cliff,

Bless you brother for your hard work! I know that today is hard for you, but remember that the family will be able to experience many tributes of love shown from all over the world! You did a great thing using your technical expertise to help the way you knew how! We are certainly praying for this family and for you guys as you show the love of Christ! May the Lord overwhelm you with His love and compassion and may you draw near to HIM! Bless you!

Anonymous said...

I don't know Kelsey, but am very saddened by this. I have a 19 year old daughter who was involved in a horrific car accident last year which claimed the lives of two of her friends from college who were in the car. She somehow, thru God's grace, survived with only bruises and a slight problem with her leg. I thank God every day that her life was spared. I know how I would have felt if the news had been otherwise. It's just a shame that our children can't even go anyplace anymore without being in danger. My thoughts and prayers are with the family of Kelsey. May God Bless and Keep you every day.

Mike Mayfield said...

Clif,

You guys did great work. My prayers are with Kelsey's family as the mourn her loss and with you and your family as you try to provide comfort to them. I think it is a great testament to your team that they were so committed to making a difference in a bleak situation.