The problem you may face when you upload a big file to a Nginx + Django application — 413 Request Entity Too Large, RequestDataTooBig and TypeError: cannot serialize ‘_io.BufferedRandom’ object

Python 3.7 + Django 2 + Nginx 1.10.3

Nginx

If you want to upload a large file of which size is more than 1 MB to Django application (Django + uWSGI + Nginx), you may get ‘413 Request Entity Too Large problem’.

This is because in Nginx, the default maximum accepted body size of client request is 1 MB. If you want to increase the limit, you can set client_max_body_size in nginx.conf. According to documentation:

Syntax:client_max_body_size size;
Default:client_max_body_size 1m;
Context:http, server, location

The default maximum accepted body size is indicated by the line Content-Length in the header of request. Content-Length means the size of the compressed message body in “octets” (i.e. in units of 8 bits, which happen to be “bytes” for all modern computers). You can find this information in console:

content-length

Therefore, if you want to upload a file of which size is 10MB, you can add client_max_body_size in http, server,or location:

# nginx.conf
server {
    ...
    client_max_body_size 10M;
    ...
}

After adding client_max_body_size, if you use RHEL/CentOS/Debian/Ubuntu Linux, you can reload Nginx by the following command:

service nginx reload

Django

After increasing the limit in Nginx, if you upload a large file of which size is more than 2.5 MB to Django application, you may get 400 error with the following error message:

To fix this bug, you need to set DATA_UPLOAD_MAX_MEMORY_SIZE in settings.py. For instance, if you want to increase the limit from 2.5 MB to 5 MB:

# settings.py
DATA_UPLOAD_MAX_MEMORY_SIZE = 5242880 # 5 MB

Moreover, if you execute .copy in views, you may encounter the following error:

TypeError: cannot serialize '_io.BufferedRandom' object

According to the Django ticket, when uploaded file size is greater than FILE_UPLOAD_MAX_MEMORY_SIZE, Django uses TemporaryUploadedFile to represent the file object. It may make the application crash on Python 3 because of deepcopy().Therefore, we finally solve this problem by adding FILE_UPLOAD_MAX_MEMORY_SIZE in settings.py to increase the limit:

# settings.py
FILE_UPLOAD_MAX_MEMORY_SIZE = 5242880 # 5 MB

Hope this article can help you to solve your problem :).

Module ngx_http_core_module

What's the "Content-Length" field in HTTP header?

QueryDict.copy() returns closed files when the type of file is TemporaryUploadedFile