Staff Blog / What's taking so long?

Update: Things seem to be back to normal for now, there may still be some delays later. For a video recap of this post, be sure to check out this video by Carleton Torpin.

If you've uploaded a video in the past 48 hours or so you've probably noticed the extremely long wait times when it comes to your video getting converted to Flash. First off, we're sorry! Around here minimizing the amount of time it takes from upload to viewing has always been a priority, so having to wait around for hours to see your video is just unacceptable.

Here's a little background on why this is happening:

Over the past several months we've been planning to beef up our Flash encoding infrastructure to be able to encode a lot more videos at the same time. These improvements are still a few weeks a way. Earlier this week, Stage6 (a well-known and large video sharing site) announced they were shutting down completely, and users only had until the end of this week to move their videos off. For us, this is a good problem. A lot of great people from Stage6 have migrated here (hi guys!), brought some awesome videos with them, and we're very excited to have them. Unfortunately, this sudden and extreme influx crossed a tipping point where people were uploading faster than we could encode given our current setup. Normally, a backup like this clears up quickly when traffic dies down. However, people were uploading at a furious pace right through the night, and our encoding servers just haven't had a chance to catch up.

Today we've taken some immediate steps to fix the problem:

- Added extra encoding machines (including borrowing some from our office buddies at CollegeHumor)
- Rolled out additional upload servers so transfer times stay speedy
- Temporarily reduced the number of thumbnails you can choose to 3 (you can still upload custom thumbnails)
- Optimized our internal file shuffling techniques so things get where they need to be faster
- Sacrificed a goat

Just to give you a sense of the level of traffic we're dealing with right now, on Monday we had more uploads in one hour that we did in an average day a year ago. At peak hours yesterday we were getting 50-75% more uploads than we were last week. Thanks to Peter, we've managed to speed up the encoding process significantly and it looks like we've reached a point where we can stay ahead of the flood of videos coming our way.

Unfortunately, it's still going to take awhile to work though the videos that are currently queued up. We may disable uploading for a few hours tonight (after 12 am EST) to allow our machines to catch up.

Again, sorry for the inconvenience. In the mean time, grab a drink and relax. We'll automatically email you when your video has finished encoding.