Python - Why is this data being written to file incorrectly?

2024/10/10 22:23:31

Only the first result is being written to a csv, with one letter of the url per row. This is instead of all urls being written, one per row.

What am I not doing right in the last section of this code that is causing the cvs to be written only with one of the results instead of all of them?

import requests
from bs4 import BeautifulSoup
import csvdef grab_listings():url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/2/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/3/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/4/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/5/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/6/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/7/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/8/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/page/9/")r = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class":"wlt_search_results"})for elem in l_area.findAll("a", {"class":"frame"}):return elem["href"]l = grab_listings()with open ("gyms.csv", "wb") as file:writer = csv.writer(file)for row in l:writer.writerow(row)
Answer

So I refactored your code a bit and i think it should work as you would expect it now:

import requests
from bs4 import BeautifulSoup
import csvdef grab_listings(page_idx):ret = []url = ("http://www.gym-directory.com/listing-category/gyms-fitness-centres/""page/{}/").format(page_idx) # the index of the page will be inserted herer = requests.get(url)soup = BeautifulSoup(r.text, 'html.parser')l_area = soup.find("div", {"class": "wlt_search_results"})for elem in l_area.findAll("a", {"class": "frame"}):# be sure to add all your results to a list and return it,# if you return here then you will only get the first resultret.append(elem["href"])return retdef main():l = [] # this will be a list of lists# call the function 9 times here with idx from 1 till 9for page_idx in range(1, 10):l.append(grab_listings(page_idx))print lwith open("gyms.csv", "wb") as f:writer = csv.writer(f)for row in l:# be sure that your row is a list here, if it is only# a string all characters will be seperated by a comma.writer.writerow(row)# for writing each URL in one line separated by commas at the end 
#    with open("gyms.csv", "wb") as f:
#        for row in l:
#            string_to_write = ',\n'.join(row)
#            f.write(string_to_write)if __name__ == '__main__':main()

I added some comments to the code and hope it is explanatory enough. If not just ask :)

https://en.xdnf.cn/q/118400.html

Related Q&A

How does Python interpreter look for types? [duplicate]

This question already has answers here:How does Python interpreter work in dynamic typing?(3 answers)Closed 10 months ago.If I write something like:>>> a = float()how does Python interpreter …

title() method in python writing functions when word like arent

using functiondef make_cap(sentence):return sentence.title()tryining outmake_cap("hello world") Hello World# it workd but when I have world like "arent" and isnt". how to write…

Creating a C++ Qt Gui for a Python logic

I was presented with a Python logic for which I need to create a GUI. I want to use Qt for that purpose and ideally I would like to program it in C++, without using the Qt Creator.What are recommended …

Pythons BaseHTTPServer returns junky responses

I use Pythons BaseHTTPServer and implement the following very simple BaseHTTPRequestHandler:class WorkerHandler(BaseHTTPRequestHandler):def do_GET(self):self.wfile.write({"status" : "rea…

Why is matplotlib failing on import matplotlib.pyplot as plt

I installed matplotlib using conda:conda install matplotlibThe following code failed:#!/usr/bin/env python import matplotlib import matplotlib.pyplot as pltWith this error message:"ImportError: N…

Setting cell color of matplotlib table and save as a figure?

Im following this code a link! to save a table as the image, and I have some feature like check value in a cell then set color for a cell, but I added some code stylemap, it doesnt workimport pandas a…

Errno 111 Connection refused - Python Mininet API hosts client/server no connection?

I am new to Mininet and I am trying to find a way to use a script in python to execute a few tests using Mininet. More precisely I want to build topology and send a few xmlrpc request from one host t…

Finding the Corners of the an array of coordinates

I have a 2D array of Coordinates in Numpy.My goal is to attempt to find the corners (as if it were a square). So the :Top left: smallest x, highest y Top right: largest x, largest y bottom left: smalle…

How to make a dictionary retain its sort order?

def positive(self):total = {}final = {}for word in envir:for i in self.lst:if word in i:if word in total:total[word] += 1else:total[word] = 1final = sorted(total, reverse = True)return totalThis return…

Counting line frequencies and producing output files

With a textfile like this:a;b b;a c;d d;c e;a f;g h;b b;f b;f c;g a;b d;fHow can one read it, and produce two output text files: one keeping only the lines representing the most often occurring couple …