Django - Stream request from external site as received

2024/10/4 17:17:36

How can Django be used to fetch data from an external API, triggered by a user request, and stream it directly back in the request cycle without (or with progressive/minimal) memory usage?

Background

As a short-term solution to connect with externally hosted micro-services, there is a need to limit user accessibility (based off the Django application's authentication system) to a non-authenticated API. Previous developers exposed these external IPs in Javascript and we need a solution to get them out of the public eye.

Requirements

  • We are not bound to using the requests library and are open to using any others if it can help speed up the response time.
  • Responses from the external API may be somewhat large (5-10MB) and being able to shorten the request cycle (User request via Ajax > Django > External API > Django > User) is crucial.

Is this possible? If so, can you suggest a method?

from django.shortcuts import Http404, HttpResponse
import requestsdef api_gateway_portal(request, path=''):# Determine whether to grant access# If so, fetch and return datar = requests.get('http://some.ip.address/%s?api_key=12345678901234567890' % (path,))# Return as JSON response = HttpResponse(r.content, content_type='application/json')response['Content-Length'] = len(r.content)return response

Please note - I am fully aware this is a poor long-term solution, but is necessary short-term for demo purposes until a new external authentication system is completed.

Answer
import requestsfrom django.http import StreamingHttpResponsedef api_gateway_portal(request, path=''):url = 'http://some.ip.address/%s?api_key=12345678901234567890' % (path,)r = requests.get(url, stream=True)response = StreamingHttpResponse((chunk for chunk in r.iter_content(512 * 1024)),content_type='application/json')return response

Documentation:

  • Body content workflow (stream=True explained)
  • StreamingHttpResponse
  • iter_content()
https://en.xdnf.cn/q/70580.html

Related Q&A

django rest framework - always INSERTs, never UPDATES

I want to be able to UPDATE a user record by POST. However, the id is always NULL. Even if I pass the id it seems to be ignoredView Code:JSON POSTED:{"id": 1, "name": "Craig Ch…

Copy fields from one instance to another in Django

I have the following code which takes an existing instance and copies, or archives it, in another model and then deletes it replacing it with the draft copy. Current Codedef archive_calc(self, rev_num,…

Python selenium get Developer Tools →Network→Media logs

I am trying to programmatically do something that necessarily involves getting the "developer tools"→network→media logs. I will spare you the details, long story short, I need to visit thou…

deep copy nested iterable (or improved itertools.tee for iterable of iterables)

PrefaceI have a test where Im working with nested iterables (by nested iterable I mean iterable with only iterables as elements). As a test cascade considerfrom itertools import tee from typing import …

ImportError: No module named cv2.cv

python 3.5 and windows 10I installed open cv using this command :pip install opencv_python-3.1.0-cp35-cp35m-win_amd64.whlThis command in python works fine :import cv2But when i want to import cv2.cv :i…

Why arent persistent connections supported by URLLib2?

After scanning the urllib2 source, it seems that connections are automatically closed even if you do specify keep-alive. Why is this?As it is now I just use httplib for my persistent connections... bu…

How to find accented characters in a string in Python?

I have a file with sentences, some of which are in Spanish and contain accented letters (e.g. ) or special characters (e.g. ). I have to be able to search for these characters in the sentence so I can…

Import error with PyQt5 : undefined symbol:_ZNSt12out_of_rangeC1EPKc,versionQt_5

I had watched a video on YouTube about making your own web browser using PyQt5. Link to video: https://youtu.be/z-5bZ8EoKu4, I found it interesting and decided to try it out on my system. Please note t…

Python MySQL module

Im developing a web application that needs to interface with a MySQL database, and I cant seem to find any really good modules out there for Python.Im specifically looking for fast module, capable of h…

How to calculate correlation coefficients using sklearn CCA module?

I need to measure similarity between feature vectors using CCA module. I saw sklearn has a good CCA module available: https://scikit-learn.org/stable/modules/generated/sklearn.cross_decomposition.CCA.h…