Project Euler download.py crashes on large batch

[jayvdb]

When doing a large batch download, e.g. starting from 1, download.py crashes with URLError due to Connection timed out.

Problem 232
Downloading...
Traceback (most recent call last):
 File "download.py", line 98, in <module>
 main()
 File "download.py", line 92, in main
 downloaded = save_problem(problem_num)
 File "download.py", line 68, in save_problem
 soup = download(url)
 File "download.py", line 57, in download
 data = browser.open(url).get_data()
 File "/usr/lib/python2.7/site-packages/mechanize/_mechanize.py", line 203, in open
 return self._mech_open(url, data, timeout=timeout)
 File "/usr/lib/python2.7/site-packages/mechanize/_mechanize.py", line 230, in _mech_open
 response = UserAgentBase.open(self, request, data)
 File "/usr/lib/python2.7/site-packages/mechanize/_opener.py", line 193, in open
 response = urlopen(self, req, data)
 File "/usr/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 344, in _open
 '_open', req)
 File "/usr/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 332, in _call_chain
 result = func(*args)
 File "/usr/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 1142, in http_open
 return self.do_open(httplib.HTTPConnection, req)
 File "/usr/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 1118, in do_open
 raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

[jontsai]

Yeah, probably not much you can do here. Do a smaller batch or increase the sleep timeout between each download.

If we make a fix here, PE can always change settings on their end. Maybe the best fix for this script is to add some comments about what a user can do to mitigate issues like this happening.