How to fix 502 Bad Gateway Error in production(Nginx)?

2024/4/15 1:50:44

When I tried to upload a big csv file of size about 600MB in my project which is hosted in the digital ocean, it tries to upload but shows 502 Bad Gateway Error (Nginx).

The application is a data conversion application.

This works fine while working locally.

sudo tail -30 /var/log/nginx/error.log

shows

[error] 132235#132235: *239 upstream prematurely closed connection while reading response header from upstream, client: client's ip , server: ip, request: "POST /submit/ HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/submit/", host: "ip", referrer: "http://ip/"sudo nano /etc/nginx/sites-available/myproject

shows

server {listen 80;server_name ip;client_max_body_size 999M;location = /favicon.ico { access_log off; log_not_found off; }location /static/ {alias  /root/static/;}location / {include proxy_params;proxy_pass http://unix:/run/gunicorn.sock;}
}

nginx.conf

user root;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;events {worker_connections 768;# multi_accept on;
}http {### Basic Settings##sendfile on;tcp_nopush on;tcp_nodelay on;keepalive_timeout 65;types_hash_max_size 2048;# server_tokens off;# server_names_hash_bucket_size 64;# server_name_in_redirect off;include /etc/nginx/mime.types;default_type application/octet-stream;

I have also the javascript loader running while the conversion process takes place. How can I fix this?

Answer

If you are using django 3.1 or higher you can make your file processing asynchronous this way and return a response to the user while the file conversion takes place. Your view should look something like this...

    import asynciofrom django.http import JsonResponsefrom asgiref.sync import sync_to_async@sync_to_asyncdef crunching_stuff(my_file):# do your conversion hereasync def upload(request):json_payload = {"message": "Your file is being converted"}my_file = request.POST.get('file')asyncio.create_task(crunching_stuff(my_file))return JsonResponse(json_payload)

On the front end if you use Dropzone.js your user can see the file upload progress and will get a response quicker. this is a better user experience. https://www.dropzonejs.com/

https://en.xdnf.cn/q/73225.html

Related Q&A

Shift theorem in Discrete Fourier Transform

Im trying to solve a problem with python+numpy in which Ive some functions of type that I need to convolve with another function . In order to optimize code, I performed the fft of f and g, I multipli…

Running pudb inside docker container

I prefer pudb for python debugging. I am building python applications that run inside docker container. Does any one know how to make pudb available inside docker container?Thank you

Argparse: defaults from file

I have a Python script which takes a lot of arguments. I currently use a configuration.ini file (read using configparser), but would like to allow the user to override specific arguments using command …

How can access Uploaded File in Google colab

Im new in python and I use Google Colab . I uploaded a train_data.npy into google Colab and then I want to use it . According to this link How to import and read a shelve or Numpy file in Google Colabo…

__add__ to support addition of different types?

Would be very easy to solve had python been a static programming language that supported overloading. I am making a class called Complex which is a representation of complex numbers (I know python has …

How to open .ndjson file in Python?

I have .ndjson file that has 20GB that I want to open with Python. File is to big so I found a way to split it into 50 peaces with one online tool. This is the tool: https://pinetools.com/split-files N…

loading a dataset in python (numpy) when there are variable spaces delimiting columns

I have a big dataset contains numeric data and in some of its rows there are variable spaces delimiting columns, like:4 5 6 7 8 9 2 3 4When I use this line:dataset=numpy.loadtxt("dataset.txt&q…

how to organise files with python27 app engine webapp2 framework

Ive gone through the getting started tut for python27 and app engine: https://developers.google.com/appengine/docs/python/gettingstartedpython27/By the end of the tut, all the the classes are in the sa…

Keras MSE definition

I stumbled across the definition of mse in Keras and I cant seem to find an explanation.def mean_squared_error(y_true, y_pred):return K.mean(K.square(y_pred - y_true), axis=-1)I was expecting the mean …

How do I adjust the size and aspect ratio of matplotlib radio buttons?

Ive been trying for hours to get the size and aspect ratio of a simple list of radio buttons correct with no success. Initially, import the modules:import matplotlib.pyplot as plt from matplotlib.widge…