When I tried to upload a big csv file of size about 600MB in my project which is hosted in the digital ocean, it tries to upload but shows 502 Bad Gateway Error (Nginx).

The application is a data conversion application.

This works fine while working locally.

sudo tail -30 /var/log/nginx/error.log

shows

[error] 132235#132235: *239 upstream prematurely closed connection while reading response header from upstream, client: client's ip , server: ip, request: "POST /submit/ HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/submit/", host: "ip", referrer: "http://ip/"sudo nano /etc/nginx/sites-available/myproject

shows

server {listen 80;server_name ip;client_max_body_size 999M;location = /favicon.ico { access_log off; log_not_found off; }location /static/ {alias  /root/static/;}location / {include proxy_params;proxy_pass http://unix:/run/gunicorn.sock;}
}

nginx.conf

user root;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;events {worker_connections 768;# multi_accept on;
}http {### Basic Settings##sendfile on;tcp_nopush on;tcp_nodelay on;keepalive_timeout 65;types_hash_max_size 2048;# server_tokens off;# server_names_hash_bucket_size 64;# server_name_in_redirect off;include /etc/nginx/mime.types;default_type application/octet-stream;

I have also the javascript loader running while the conversion process takes place. How can I fix this?

If you are using django 3.1 or higher you can make your file processing asynchronous this way and return a response to the user while the file conversion takes place. Your view should look something like this...

    import asynciofrom django.http import JsonResponsefrom asgiref.sync import sync_to_async@sync_to_asyncdef crunching_stuff(my_file):# do your conversion hereasync def upload(request):json_payload = {"message": "Your file is being converted"}my_file = request.POST.get('file')asyncio.create_task(crunching_stuff(my_file))return JsonResponse(json_payload)

On the front end if you use Dropzone.js your user can see the file upload progress and will get a response quicker. this is a better user experience. https://www.dropzonejs.com/