I am scraping election data from a website and trying to store it in a dataframe
import pandas as pd
import bs4
import requestscolumns = ['Candidate','Party','Criminal Cases','Education','Age','Total Assets','Liabilities']df = pd.DataFrame(columns = columns)ind=1url = requests.get("http://myneta.info/up2007/index.php?action=show_candidates&constituency_id=341")
soup = bs4.BeautifulSoup(url.content)for content in soup.findAll("td")[16:]:df.iloc[ind//7,ind%7-1] = content.textind=ind+1
print(df)
Essentially, each iteration of content.text will provide me a value which I will populate in the table. The loop will populate values to df in the following sequence -
df[0,0]
df[0,1]
df[0,2]
.
.
.
df[1,0]
df[1,1]
.
.
and so on. Unfortunately the iloc is throwing an error - "single positional indexer is out-of-bounds". The funny part is when I try df.iloc[0,0] = content.text
outside the for loop (in a separate cell for testing purpose), the code works properly, but in the for loop it creates an error. I believe it might be something trivial but i am unable to understand.Please help