ine@mit software and mathematics

Downloading the xCode and iOS SDK on a slow internet connection

At home, our internet connection tops out around 60KB/sec - which is really hard to readjust to after experiencing the 20+MB/sec awesomeness which the MIT networks deliver. So when I wanted to download the latest iOS SDK, a 3.5Gb file, things weren't as easy as they seemed.

First I tried letting our iMac download the file overnight. It failed at around 1.5Gb. The next night it failed around 2.2Gb. Also, asking Safari or Chrome to restart the download would restart the download from the beginning, clearly not desirable failure. Then I looked into using a download manager (like DownThemAll) for Firefox to run the download (supports chunking the file and multithreading the download) but I got 403 Forbidden errors because Apple runs the download over HTTP and requires cookies. Grrr. I would have switched to a torrent, but our ISP here blocks torrent ports.

Then, inspired by this StackOverflow question, I remembered I had my XVM. I turned it on,  exported my cookies from Firefox , and ran:

wget --cookies=on --load-cookies=cookies.txt --keep-session-cookies --save-cookies=cookies.txt https://developer.apple.com/devcenter/download.action?path=/ios/ios_sdk_4.2__final/xcode_3.2.5_and_ios_sdk_4.2_final.dmg

wget was kind enough to follow the redirect and finish the download in a cool 3 minutes.

...
Moved Temporarily Location: http://adcdownload.apple.com/ios/ios_sdk_4.2__final/xcode_3.2.5_and_ios_sdk_4.2_final.dmg [following] --13:28:40--  http://adcdownload.apple.com/ios/ios_sdk_4.2__final/xcode_3.2.5_and_ios_sdk_4.2_final.dmg            => `download.action?path=%2Fios%2Fios_sdk_4.2__final%2Fxcode_3.2.5_and_ios_sdk_4.2_final.dmg' Resolving adcdownload.apple.com... xx.xx.xx.xx, xx.xx.xx.xx Connecting to adcdownload.apple.com|xx.xx.xx.xx|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 3,782,241,266 (3.5G) [application/octet-stream]  69% [=======================>           ] 2,629,919,501   18.27M/s    ETA 00:56

Then, after installing lighttpd on my server, I was downloading in five segments at a blazing 30Kb/s. Pausing and resuming the download even worked!

Edit: When I md5'd the downloaded file (a day later!) the checksum didn't match the server checksum. So I ran the unix split utility and divided the .dmg into 500MB chunks - incredibly, these chunks also were failing checksum. I ran split again, dividing into 100MB chunks. The md5s started matching, everything downloaded...I checked all the individual md5s, ran cat to stitch the files back together, checksummed the final .dmg...and it was finally there. That night I uploaded the update to my iPhone app to Apple. Yay!