How to catch network failures while invoking get() method through Selenium and Python?

2024/10/13 4:24:06

I am using Chrome with selenium and the test run well, until suddenly internet/proxy connection is down, then browser.get(url) get me this: enter image description here

If I reload the page 99% it will load fine, what is the proper way to handle this ?

MY CODE:

def web_adress_navigator(browser, link):
"""Checks and compares current URL of web page and the URL to be navigated and if it is different, it does navigate"""try:current_url = browser.current_url
except WebDriverException:try:current_url = browser.execute_script("return window.location.href")except WebDriverException:current_url = Noneif current_url is None or current_url != link:retries = 5while retries > 0:try:browser.get(link)breakexcept TimeoutException:logger.warning('TimeoutException when tring to reach page')retries -= 1while not is_connected():sleep(60)logger.warning('there is no valid connection')

I am not getting into TIMEOUT EXCEPTION but to the break part.

Answer

As per your question and your code trials as you are trying to access the url passed through the argument link you can adapt a strategy where:

  • Your program will make pre-defined number of trials to invoke the desired url, which you can pass through range().
  • Once you invoke get(link) your program will invoke WebDriverWait for a predefined interval for the url to contain a pre-defined partialURL from the url.
  • You can handle this code within a try{} block with expected_conditions method title_contains() and in case of TimeoutException invoke browser.get(link) again within the catch{} block.
  • Your modified code block will be:

    #imports
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.common.exceptions import TimeoutException
    # other code works
    browser.get(link)
    for i in range(3):try:WebDriverWait(browser, 10).until(EC.title_contains(partialTitle))breakexcept TimeoutException:browser.get(link)
    logger.warning('there is no valid connection')
    
https://en.xdnf.cn/q/118121.html

Related Q&A

Pandas crosstab with own function

I have a function which takes two inputs and returns a float e.g. my_func(A, B) = 0.5. I have a list of possible inputs: x = [A, B, C, D, E, F].I want to produce a square matrix (in this case 6 by 6) …

Using a custom threshold value with tf.contrib.learn.DNNClassifier?

Im working on a binary classification problem and Im using the tf.contrib.learn.DNNClassifier class within TensorFlow. When invoking this estimator for only 2 classes, it uses a threshold value of 0.5 …

How to remove certain characters from a variable? (Python)

Lets suppose I have a variable called data. This data variable has all this data and I need to remove certain parts of it while keeping most of it. Lets say I needed to remove all the , (commas) in thi…

criticism this python code (crawler with threadpool)

how good this python code ? need criticism) there is a error in this code, some times script do print "ALL WAIT - CAN FINISH!" and freeze (no more actions are happend..) but i cant find reas…

cx_Freeze executable not displaying matplotlib figures

I am using Python 3.5 and I was able to create an executable using cx_Freeze but whenever I try to run the executable it runs without error but it cannot display any matplotlib figure. I have used Tkin…

Saving variables in n Entry widgets Tkinter interface

Firstly apologises for the length of code but I wanted to show it all.I have an interface that looks like this:When I change the third Option Menu to "List" I will add in the option to have n…

Pulling the href from a link when web scraping using Python

I am scraping from this page: https://www.pro-football-reference.com/years/2018/week_1.htmIt is a list of game scores for American Football. I want to open the link to the stats for the first game. The…

Php: Running a python script using blender from a php project using cmd commands

I need to run in cmd a python script for blender from blender and print the result from a php project, but I dont get the all result. Here is my code:$script = "C:\Users\madalina\Desktop\workspace…

Pymysql when executing Union query with %s (Parameter Placeholder)

This is the code about UNION QUERY:smith =Smithsmithb=Smithsql="""SELECT Distinct Pnumber FROM PROJECT, DEPARTMENT, EMPLOYEE WHERE Dnum = Dnumber AND Mgr_ssn=Ssn AND Lname= %s UNION SELE…

Django - Calling list or dict item using a variable in template

Im trying to call a dictionary or list object in a template using a variable in that template with no results.What Im trying to is identical to this general python code:keylist=[firstkey,secondkey,thir…