Selenium, python dynamic table

2024/10/8 4:29:07

enter image description here

I'm creating a robot with selenium that get all info from agencies in Brasil, i've alredy done the permutation click between all States and counties, all i have to do now's click in all agencies and get infos like: "telefone","endereço","HORÁRIO DE ATENDIMENTO". How can I do that?? That's my code till now:

 # -*- coding: UTF-8 -*-
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.select import Select
import time     
import urllib
import urllib.request
from bs4 import BeautifulSoup
import requestdef make_soup(url):thepage = urllib.request.urlopen(url)soupdata = BeautifulSoup(thepage,'html.parser')return soupdatachrome_path = r"D:\Users\gbenites\Desktop\Inovação\arquivos py\WebDriver\chromedriver.exe"
url = 'http://www2.correios.com.br/sistemas/agencias/'
driver = webdriver.Chrome(chrome_path)
driver.get("http://www2.correios.com.br/sistemas/agencias/")
#WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, 'municipioAgencia')))
driver.maximize_window()time.sleep(3) 
estadoList = Select(driver.find_element_by_name('estadoAgencia'))
for index in range(3,len(estadoList.options)):select = Select(driver.find_element_by_name('estadoAgencia'))print("selecting Estado: ",select.options[index].text)    print("selecting Index: ",index)    select.select_by_index(index)time.sleep(2)municípioList = Select(driver.find_element_by_name('municipioAgencia'))for index_b in range(1,len(municípioList.options)):select_b = Select(driver.find_element_by_name('municipioAgencia'))print("...selecting Municipio",select_b.options[index_b].text.replace("'",""))print("...selecting Municipio Index: ",index_b)select_b.select_by_index(index_b)time.sleep(3)get_info_list =  WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//table[@class = 'dadosAgencia']//td")))info_list = driver.find_elements_by_xpath("//table[@class = 'dadosAgencia']//span")print(range(len(info_list)))for record in range(0,len(info_list)):print(record.get_attribute('innerText'))

And that's my output:

selecting Estado:  AMAPÁ
selecting Index:  3
...selecting Municipio AMAPA
...selecting Municipio Index:  1
range(0, 27)
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Answer

Try this xPath:

//table[@class = 'dadosAgencia']//tr

It would be like this:

elements = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//table[@class = 'dadosAgencia']//tr")))

it gives you a list of all elements located. To print the text of each element you can use this:

for element in elements:print(element.text)

Note: you have to add some imports:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
https://en.xdnf.cn/q/118740.html

Related Q&A

How to split string into column

I got a csv file with some data, and I want to split this data.My column one contains a title, my column 2 contains some dates, and my column 3 contains some text linked to the dates.I want to tran…

remove whitespaces with new line characters

I have a string that looks like that:"\n My name is John\n and I like to go.\n blahblahblah.\n \n\n ".Note - In this string example, there are 5 white-spaces after…

Python tkinter GUI freezing/crashing

from Tkinter import * import tkFileDialog import tkMessageBox import os import ttkimport serial import timeit import time################################################################################…

If/else in python list comprehension

I would like to return random word from file, based on passed argument. But if the argument doesnt match anythning I dont want to return anything. My method looks like:def word_from_score(self,score):p…

Conditional module importing in Python

Im just trying out Maya 2017 and seen that theyve gone over to PySide2, which is great but all of my tools have import PySide or from PySide import ... in them.The obvious solution would be to find/rep…

AttributeError: list object has no attribute lower : clustering

Im trying to do a clustering. Im doing with pandas and sklearn. import pandas import pprint import pandas as pd from sklearn.cluster import KMeans from sklearn.metrics import adjusted_rand_score from s…

I’m dealing with an error when I run server in Django

PS C:\Users\besho\OneDrive\Desktop\DjangoCrushcourse> python manage.py runserver C:\Users\besho\AppData\Local\Programs\Python\Python312\python.exe: cant open file C:\Users\besho\OneDrive\Desktop\Dja…

python threading with global variables

i encountered a problem when write python threading code, that i wrote some workers threading classes, they all import a global file like sharevar.py, i need a variable like regdevid to keep tracking t…

How to write nth value of list into csv file

i have the following list : sec_min=[37, 01, 37, 02, 37, 03, 37, 04, 37, 05,....]i want to store this list into CVS file in following format: 37,01 37,02 37,03 37,04 ... and so onthis is what i coded: …

Read R function output as columns

Im trying to come up with a way to solve this question I asked yesterday:rpy2 fails to import rgl R packageMy goal is to check if certain packages are installed inside R from within python.Following th…