Marketplace scraper suddenly stopped, they updated the hidden json?

Since yesterday, I encountered an issue where my facebook marketplace scraper ceased to fetch data, I’m currently using scrapy due to his features, am I doing any mistakes?

from scrapy import Spider
import logging
from json import loads


class Facebook(Spider):
    name="facebook"
    start_urls = ["https://www.facebook.com/marketplace/112047398814697/search?query=funko&sortBy=creation_time_descend&radius=500"]

    def parse(self, response):

        flex_selector = response.xpath('//*[@id="facebook"]/body/script/text()')

        ads = []

        for script in flex_selector.getall():
            try:
                ads = loads(script)
                ads = ads['require'][0][3][0]['__bbox']['require'][0][3][1]['__bbox']['result']['data']['marketplace_search']['feed_units']['edges']
                break
            except:
                pass
        # if(len(ads) == 0):

        for item in ads:
            try:
                title = adJson['node']['listing']['marketplace_listing_title']
                from pdb import set_trace; set_trace()

To run the code below, runs: scrapy runspider main.py

I’m currently using py 3.12.2 and scrapy 2.11.2

You need to sign in to view this answers

Related Post