Python URL Stepping Returns Only First Page Results

2024/11/15 9:24:59

Any help with the below code would be appreciated. I have checked the results of h and g using print to verify that they are incrementing the url properly, but the program seems to be only repeating the results from first page. Hope this makes sense and I have provided enough info. I know this code looks terrible.

edit** I am testing the code as I go in the Python2.7 shell. I have it print the link results to make sure they are working correctly, but it just repeats pg1.

Update** The problem with the code was due to the website using json to get pages. Python Link to File Iterator not Iterating

g = 'http://www.somesite.com/pg'
b = 'http://www.somesite.com/pg'
PageCount = 1while PageCount < 3:h = g + str(PageCount)c = b + str(PageCount)f = urllib2.urlopen(h)# variable a is for the second function that opens links for webpages# meeting criteria from variable fa = urllib2.urlopen(c)# res variable captures lines for items meeting criteria to be opened in a webpageres = []PageCount += 1        #check function checks for criteria current webpagecheck()#ReturnLine function opens webpages using data from variable resReturnLine()
Answer

I derived a minimal working example (no lengthy code possible in comment section..)

g = 'http://www.somesite.com/pg'
PageCount = 1while PageCount < 3:h = g + str(PageCount)print hPageCount += 1

which is working just fine. The output is

http://www.somesite.com/pg1
http://www.somesite.com/pg2

Is this what you get? If so, try calling urllib2.urlopen([URL]) with a fixed url to check proper function in a separate minimal working example and go from there. Otherwise I see no error (or sources of error) that may cause such a behaviour.

https://en.xdnf.cn/q/119677.html

Related Q&A

Text processing to find co-occurences of strings

I need to process a series of space separated strings i.e. text sentences. ‘Co-occurrence’ is when two tags (or words) appear on the same sentence. I need to list all the co-occurring words when they…

Flask doesnt render any image [duplicate]

This question already has answers here:How to serve static files in Flask(24 answers)Link to Flask static files with url_for(2 answers)Closed 6 years ago.I have a flask application where I need to rend…

Bug in python thread

I have some raspberry pi running some python code. Once and a while my devices will fail to check in. The rest of the python code continues to run perfectly but the code here quits. I am not sure wh…

how does a function changes the value of a variable outside its scope? Python

i was coding this code and noticed something weird, after my function has been called on the variable, the value of the variable gets changed although its outside of the functions scope, how exactly is…

Python extracting element using bs4, very basic thing I think I dont understand

So Im using Beautiful Soup to try to get an element off of a page using the tag and class. Here is my code: import requests from bs4 import BeautifulSoup# Send a GET request to the webpage url = "…

Why Isnt my Gmail Account Bruteforcer Working? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.Want to improve this question? Add details and clarify the problem by editing this post.Closed 4 years ago.Improve…

Python: Split Start and End Date into All Days Between Start and End Date

Ive got data called Planned Leave which includes Start Date, End Date, User ID and Leave Type.I want to be able to create a new data-frame which shows all days between Start and End Date, per User ID.S…

Python and java AES/ECB/PKCS5 encryption

JAVA VERSION:public class EncryptUtil {public static String AESEncode(String encodeRules, String content) {try {KeyGenerator keygen = KeyGenerator.getInstance("AES");keygen.init(128, new Secu…

How to find the center point of this rectangle

I am trying to find the center point of the green rectangle which is behind the fish, but my approach is not working. Here is my code:#Finding contours (almost always finds those 2 retangles + some noi…

Simple Battleships game implementation in Python

Okay Im not sure how to develop another board with hidden spaces for the computers ships per-se, and have it test for hits. Again Im not even sure how Im going to test for hits on the board I have now.…