I have the following simple code:

import urllib2
import sys
sys.path.append('../BeautifulSoup/BeautifulSoup-3.1.0.1')
from BeautifulSoup import *
page='http://en.wikipedia.org/wiki/Main_Page'
c=urllib2.urlopen(page)

This code generates the following error messages:

    c=urllib2.urlopen(page)File "/usr/lib64/python2.4/urllib2.py", line 130, in urlopenreturn _opener.open(url, data)File "/usr/lib64/python2.4/urllib2.py", line 364, in openresponse = meth(req, response)File "/usr/lib64/python2.4/urllib2.py", line 471, in http_responseresponse = self.parent.error(File "/usr/lib64/python2.4/urllib2.py", line 402, in errorreturn self._call_chain(*args)File "/usr/lib64/python2.4/urllib2.py", line 337, in _call_chainresult = func(*args)File "/usr/lib64/python2.4/urllib2.py", line 480, in http_error_defaultraise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden

But if I replace urllib2 by urllib, I get no error messages. Can anybody explain this behavior?

The original urllib simply does not raise an exception on a 403 code. If you add print c.getcode() to the last line of your program, urllib will reach it and still print out 403.

Then if you do print c.read() at the end, you will see that you did indeed get an error page from Wikipedia. It's just a matter of urllib2 deciding to treat an error 403 as a runtime exception, versus urllib allowing you to still get an error 403 and then do something with the page.