How to stream large binary data from RPC/CGI to browser in LuCI with JSapi?

Hello there,

I'm developing a LuCI app that needs to export Podman volumes as .tar.gz files for browser download. Currently my RPC method runs podman volume export | gzip, writes the result to a temp file in /tmp, base64-encodes it, and returns it as JSON. This works for small volumes but fails for large ones because /tmp is tmpfs (RAM-backed) and fills up quickly.

What's the recommended way to stream large binary data directly to the browser in LuCI without buffering the entire file in /tmp? I've seen cgi-io/cgi-download can serve files, but those also require the file to exist on disk first. Ideally I'd pipe podman volume export | gzip straight to the HTTP response.

cheers

redirect to a static file

Not sure if i understand it correctly, but i have a static file in the temp folder. Problem is for larger volumes for example with 3gb. To export them the only way, i think, is to stream the output because podman directly returns the file in the api.

However, a short (link) example or a few words more would help a lot :wink:

ln -s /temp/static.file /www/same.static.file

point to http://router.IP/same.static.file

1 Like

Yeah, nearly what i am doing for small files. /temp or /tmp can save a lot but not always 3gb. So create/save a 3gb(or more) file at the router is not acceptable - or if there is for example only 500mb space left and i want get a 1gb file, how should i statically link if i can not create it? The podman api returns the binary data so i could stream it directly to the browser instead of saving it at the router (to be honest if i have the file it does not matter where it is saved, with a rpc call i can get it from anywhere but that is not the problem itself :wink: )

I thought there must be a way because i also do not think that for example luci backup function is not possible if space is full.

I think i have found it...

Yes this works perfectly like i want, exported a 5gb volume without the need of saving the file before sending to the user :slight_smile:

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.