Since yesterday, I encountered an issue where my facebook marketplace scraper ceased to fetch data, I’m currently using scrapy due to his features, am I doing any mistakes?
from scrapy import Spider
import logging
from json import loads
class Facebook(Spider):
name="facebook"
start_urls = ["https://www.facebook.com/marketplace/112047398814697/search?query=funko&sortBy=creation_time_descend&radius=500"]
def parse(self, response):
flex_selector = response.xpath('//*[@id="facebook"]/body/script/text()')
ads = []
for script in flex_selector.getall():
try:
ads = loads(script)
ads = ads['require'][0][3][0]['__bbox']['require'][0][3][1]['__bbox']['result']['data']['marketplace_search']['feed_units']['edges']
break
except:
pass
# if(len(ads) == 0):
for item in ads:
try:
title = adJson['node']['listing']['marketplace_listing_title']
from pdb import set_trace; set_trace()
To run the code below, runs: scrapy runspider main.py
I’m currently using py 3.12.2 and scrapy 2.11.2
You need to sign in to view this answers
Leave feedback about this