October 25, 2024
Chicago 12, Melborne City, USA
python

How to download a very big zip file on the fly using Sanic?


I am working on a Python app which uses Sanic and I need to download a very big archive (10 GB).

I am using zipstream to create the archive on the fly and then stream it (Sanic has support for streaming responses).

Something like:

zs = ZipStream()
        for root, _, files in os.walk(file_path):
            for file in files:
                fpath = str(os.path.join(root, file))
                zs.add_path(fpath)

        response = await request.respond(
            headers={'Content-Disposition': f'attachment; filename="{dataset_id}.zip"'},
            content_type="application/zip"
        )

        for chunk in zs:
            await response.send(chunk)

The problem is that the download stops after a few hundreds of MB and I don’t know why.

The only way it works is by using:

return await sanic.response.file_stream(
            fpath,
            headers={'Content-Disposition': f'attachment; filename="file.zip"'},
        )

but the problem here is that file_stream() expects a real file as the first parameter (location: str | PurePath) which I cannot obtain from zipstream (I want to zip on the fly).

Does anybody know how to do this? Is there a way to emulate a file on the filesystem to make file_stream() work? Or is there another way?



You need to sign in to view this answers

Leave feedback about this

  • Quality
  • Price
  • Service

PROS

+
Add Field

CONS

+
Add Field
Choose Image
Choose Video