Python Selenium. How to use driver.set_page_load_timeout() properly?

2024/10/13 11:19:49
from selenium import webdriverdriver = webdriver.Chrome()
driver.set_page_load_timeout(7)def urlOpen(url):try:driver.get(url)print driver.current_urlexcept:return

Then I have URL lists and call above methods.

if __name__ == "__main__":urls = ['http://motahari.ir/', 'http://facebook.com', 'http://google.com']# It doesn't print anything# urls = ['http://facebook.com', 'http://google.com', 'http://motahari.ir/'] # This prints https://www.facebook.com/ https://www.google.co.kr/?gfe_rd=cr&dcr=0&ei=3bfdWdzWAYvR8geelrqQAw&gws_rd=sslfor url in urls:urlOpen(url)

The problem is when website 'http://motahari.ir/' throws Timeout Exception, websites 'http://facebook.com' and 'http://google.com' always throw Timeout Exception.

Browser keeps waiting for 'motahari.ir/' to load. But the loop just goes on (It doesn't open 'facebook.com' but wait for 'motahari.ir/') and keep throwing timeout exception

Initializing a webdriver instance takes long, so I pulled that out of the method and I think that caused the problem. Then, should I always reinitialize webdriver instance whenever there's a timeout exception? And How? (Since I initialized driver outside of the function, I can't reinitialize it in except)

Answer

You will just need to clear the browser's cookies before continuing. (Sorry, I missed seeing this in your previous code)

from selenium import webdriverdriver = webdriver.Chrome()
driver.set_page_load_timeout(7)def urlOpen(url):try:driver.get(url)print(driver.current_url)except:driver.delete_all_cookies()print("Failed")returnurls = ['http://motahari.ir/', 'https://facebook.com', 'https://google.com']for url in urls:urlOpen(url)

Output:

Failed
https://www.facebook.com/
https://www.google.com/?gfe_rd=cr&dcr=0&ei=o73dWfnsO-vs8wfc5pZI

P.S. It is not very wise to do try...except... without a clear Exception type, is this might mask different unexpected errors.

https://en.xdnf.cn/q/69539.html

Related Q&A

Editing both sides of M2M in Admin Page

First Ill lay out what Im trying to achieve in case theres a different way to go about it!I want to be able to edit both sides of an M2M relationship (preferably on the admin page although if needs be …

unstacking shift data (start and end time) into hourly data

I have a df as follows which shows when a person started a shift, ended a shift, the amount of hours and the date worked. Business_Date Number PayTimeStart PayTimeEnd Hours 0 2019-05-24 1…

Tensorflow model prediction is slow

I have a TensorFlow model with a single Dense layer: model = tf.keras.Sequential([tf.keras.layers.Dense(2)]) model.build(input_shape=(None, None, 25))I construct a single input vector in float32: np_ve…

Pandas Sqlite query using variable

With sqlite3 in Python if I want to make a db query using a variable instead of a fixed command I can do something like this :name = MSFTc.execute(INSERT INTO Symbol VALUES (?) , (name,))And when I tr…

How to remove ^M from a text file and replace it with the next line

So suppose I have a text file of the following contents:Hello what is up. ^M ^M What are you doing?I want to remove the ^M and replace it with the line that follows. So my output would look like:Hello…

Cython: size attribute of memoryviews

Im using a lot of 3D memoryviews in Cython, e.g.cython.declare(a=double[:, :, ::1]) a = np.empty((10, 20, 30), dtype=double)I often want to loop over all elements of a. I can do this using a triple loo…

python asynchronous httprequest

I am trying to use twitter search web service in python. I want to call a web service like:http://search.twitter.com/search.json?q=blue%20angels&rpp=5&include_entities=true&result_type=mix…

What are response codes for 256 and 512 for os.system in python scripting

When i ping servers with os.system in python i get multiple response codes. Command used - os.system("ping -q -c 30 -s SERVERANME")0 - Online 256 - Offline 512 - what does 512 mean ?

Sphinx floating point formatting

Im using Sphinx to generate documentation from code. Does anyone know if there is a way to control the formatting of floating point numbers generated from default arguments. For example if I have the f…

Truncating column width in pandas

Im reading in large csv files into pandas some of them with String columns in the thousands of characters. Is there any quick way to limit the width of a column, i.e. only keep the first 100 characters…