scraping css values using scrapy framework

2024/11/16 22:28:57

Is there a way to scrap css values while scraping using python scrapy framework or by using php scraping. any help will be appreaciated

Answer

scrapy.Selector allows you to use xpath to extract properties of HTML elements including CSS.

e.g. https://github.com/okfde/odm-datenerfassung/blob/master/crawl/dirbot/spiders/data.py#L83

(look around that code for how it fits into an entire scrapy spider)

If you don't need web crawling and just html parsing, you can use xpath directly from lxml in python. Another example:

https://github.com/codeformunich/feinstaubbot/blob/master/feinstaubbot.py

Finally, to get at the css from xpath I only know how to do it via css=element.attrib['style'] - this gives you everything inside of the style attribute which you further split by e.g. css.split(';') and then each of those by ':'.

It wouldn't surprise me if someone has a better suggestion. A little knowledge is enough to do a lot of scraping and that's how I would approach it based on previous projects.

https://en.xdnf.cn/q/120276.html

Related Q&A

Access dict via dict.key

I created a dict source = {livemode: False}. I thought its possible to access the livemode value via source.livemode. But it doesnt work. Is there a way to access it that way?As a not source[livemode]…

Function not returning anything

My viewdef login(request):c = {}c.update(csrf(request))return render_to_response(request, login.html, c)def auth_view(request):username = request.POST.get (username, )password = request.POST.get (passw…

My entry box always returns PY_VAR1 value!!though Im using the .get function

please take a look at my code, its really simple I need to take the value from the entry box and use it in my program and when pressing the add button I print it ,it keeps giving me this value PY_VAR1…

Arthimatic Quiz Not Accepting Correct Answers

I am attempting to make an arithmetic quiz, but have run into this issue: Even if I input the correct answer, it seems to ignore the correct answer code and go straight to the incorrect answer code. Ba…

Linux - Check if python script is running in screen and run if not [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.Want to improve this question? Update the question so it focuses on one problem only by editing this post.Closed 1…

Making Phonebook in python : i want to get this screen by fixing my current code

I made my code like below....But as i input the data such as spam & number, previous data is deleted.So id like to make multiple value in one key... (i think using list is kinda good method)For exa…

Adding enemies to a pygame platformer

Im new to pygame and trying to make a platformer game thats based on this tutorial: http://programarcadegames.com/python_examples/show_file.php?file=platform_scroller.pyI cant quite figure out how to …

Tips for cleaning up a challenges answer? Weighted Sum of Digits [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.Want to improve this question? Add details and clarify the problem by editing this post.Closed 8 years ago.Improve…

Get a variable as filename from python script and use it in a batch script [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.Want to improve this question? Add details and clarify the problem by editing this post.Closed 5 years ago.Improve…

Python 3.x AttributeError: NoneType object has no attribute groupdict

Being a beginner in python I might be missing out on some kind of basics. But I was going through one of the codes from a project and happened to face this :AttributeError: NoneType object has no attri…