Nginx buffering even with proxy_request_buffering off;

Installing and Using OpenWrt
1

Hello,

I've been struggling for some days trying to upload big files using LuCi. I created a new view which allows update a file using ui.uploadFile

o = s.option(form.Button, 'fileupload', _('File Upload'));
o.inputstyle = 'action important';
o.inputtitle = _('Upload new file');
o.onclick = L.bind(this.handleFileUpload, this);

Being handleFileUpload

	handleFileUpload: function(ev) {
		return ui.uploadFile(DiskPath, ev.target.firstChild)
			.catch(function(e) { ui.addNotification(null, E('p', e.message)) })
			.finally(L.bind(function(btn) {
			}, this, ev.target));
	},

So far so good, view is fine and I can upload files.

The problem is that this file, which could be very, very large (>20GB) is always copied to /tmp by nginx, and once upload finish, pass it to uwsgi (no deleting it), which does the same, a copy on /tmp, so I need to have a /tmp more than 2*file_size, which I cannot have and furthermore I think it is pretty inefficient.

I tried all possible combinations I could figure out of nginx, but cannot find a proper solution. I would like to nginx directly passes the chunks of the file as they are being uploaded to uwsgi, not buffer at all or at least some small buffer (~MB) but proxy_request_buffering or fastcgi_request_buffering do nothing, nginx always copies the entire file to /tmp and then pass it to uwsgi which makes another copy.

This is my nginx configuration file, any help would be really appreciatted :slight_smile:

# Consider using UCI or creating files in /etc/nginx/conf.d/ for configuration.
# Parsing UCI configuration is skipped if uci set nginx.global.uci_enable=false
# For details see: https://openwrt.org/docs/guide-user/services/webserver/nginx

worker_processes auto;

user root;

events {
        
}

http {
    #access_log syslog:server=unix:/dev/log,tag=nginx,nohostname,severity=info combined;
    #error_log  syslog:server=unix:/dev/log,tag=nginx,nohostname,severity=info;

    include mime.types;
    default_type application/octet-stream;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    proxy_buffering off;

    server {
        listen 443 ssl;
        server_name 192.168.6.210;
        
        location /cgi-bin/luci {
            index  index.html;
            include uwsgi_params;
            uwsgi_param SERVER_ADDR $server_addr;
            uwsgi_modifier1 9;
            uwsgi_pass unix:////var/run/luci-webui.socket;
        }

        location ~ /cgi-bin/cgi-(backup|download|upload|exec) {
            proxy_pass http://localhost:3000;

            include uwsgi_params;
            uwsgi_pass unix:////var/run/luci-cgi_io.socket;
            uwsgi_param SERVER_ADDR $server_addr;
            uwsgi_modifier1 9;

            client_max_body_size 0;
            proxy_request_buffering off;
            fastcgi_request_buffering off;
        }

        location /luci-static {
                error_log stderr crit;
        }

        location /ubus {
                ubus_interpreter;
                ubus_socket_path /var/run/ubus/ubus.sock;
                ubus_parallel_req 2;
        }

    }

    gzip on;
    #gzip_vary on;
    #gzip_proxied any;

    root /www;

    #UCI_HTTP_CONFIG
    include conf.d/*.conf;
}

Where conf.d/ is an empty folder.

I had some kind of luck trying with

    client_body_buffer_size         0m;
    client_max_body_size            20000m;

but then file is stored in RAM, and I don't have either 20 GB of RAM...

Again, thank you so much