After scanning the urllib2
source, it seems that connections are automatically closed even if you do specify keep-alive.
Why is this?
As it is now I just use httplib
for my persistent connections... but wonder why this is disabled (or maybe just ambiguous) in urllib2.
It's a well-known limit of urllib2 (and urllib as well). IMHO the best attempt so far to fix it and make it right is Garry Bodsworth's coda_network for Python 2.6 or 2.7 -- replacement, patched versions of urllib2 (and some other modules) to support keep-alive (and a bunch of other smaller but quite welcome fixes).