How to select many to one to many without hundreds of queries using Django ORM?

2024/10/11 15:17:46

My database has the following schema:

class Product(models.Model):passclass Tag(models.Model):product = models.ForeignKey(Product)attr1 = models.CharField()attr2 = models.CharField()attr3 = models.CharField()class AlternatePartNumber(models.Model):product = models.ForeignKey(Product)

In other words, a Product has many Tags, and a Product has many AlternatePartNumbers. Tags are a collection of attributes of the Product.

Given the three attributes in a Tag, I want to select the associated Products that match (could be more than one), as well as all of the AlternatePartNumbers of each product.

Currently I do this:

# views.py
results = Tag.objects.filter(attr1=attr1).filter(attr2=attr2).filter(attr3=attr3)# a template
{% for result in results %}{% for alternate in result.product.alternatepartnumber_set.all %}{{ alternate.property }}{% endfor %}
{% endfor %}

This can run thousands of queries, depending on the number of matches. Is there a good way to optimize this? I tried using Tag.objects.select_related().filter... and that helped some, but it didn't help enough.

Answer

The relationship between Product and AlternatePartNumber is a reverse ForeignKey relationship, so select_related() won't work. You need prefetch_related(), which is a little less aggressive than select_related() but can handle many-to-one relationships.

I haven't used prefetch_related() myself before but if I'm reading the documentation correctly, you need something like Tag.objects.prefetch_related('product__alternatepartnumber_set').filter.... If that doesn't work, specify a related_name on the AlternatePartNumber model and use that instead of alternatepartnumber_set.

https://en.xdnf.cn/q/69759.html

Related Q&A

Quickly dumping a database in memory to file

I want to take advantage of the speed benefits of holding an SQLite database (via SQLAlchemy) in memory while I go through a one-time process of inserting content, and then dump it to file, stored to b…

QStatusBar message disappears on menu hover

I have a very basic QMainWindow application that contains a menubar and a statusbar. When I hover over the menu the status message disappears. More precisely, the status message is cleared. I have no i…

How to eliminate a python3 deprecation warning for the equality operator?

Although the title can be interpreted as three questions, the actual problem is simple to describe. On a Linux system I have python 2.7.3 installed, and want to be warned about python 3 incompatibiliti…

Cannot get scikit-learn installed on OS X

I cannot install scikit-learn. I can install other packages either by building them from source or through pip without a problem. For scikit-learn, Ive tried cloning the project on GitHub and installin…

Decompressing a .bz2 file in Python

So, this is a seemingly simple question, but Im apparently very very dull. I have a little script that downloads all the .bz2 files from a webpage, but for some reason the decompressing of that file is…

Why does Pandas coerce my numpy float32 to float64?

Why does Pandas coerce my numpy float32 to float64 in this piece of code:>>> import pandas as pd >>> import numpy as np >>> df = pd.DataFrame([[1, 2, a], [3, 4, b]], dtype=np…

Conda and Python Modules

Sadly, I do not understand how to install random python modules for use within iPython Notebooks with my Anaconda distribution. The issue is compounded by the fact that I need to be able to do these t…

WeakValueDictionary retaining reference to object with no more strong references

>>> from weakref import WeakValueDictionary >>> class Foo(object): ... pass >>> foo = Foo() >>> db = WeakValueDictionary() >>> db[foo-id] = foo >>…

Using pretrained glove word embedding with scikit-learn

I have used keras to use pre-trained word embeddings but I am not quite sure how to do it on scikit-learn model.I need to do this in sklearn as well because I am using vecstack to ensemble both keras s…

Is there an easy way to tell how much time is spent waiting for the Python GIL?

I have a long-running Python service and Id like to know how much cumulative wall clock time has been spent by any runnable threads (i.e., threads that werent blocked for some other reason) waiting for…