Simplify Django test set up with mock objects

2024/11/3 7:45:52

Often when I'm writing tests for my Django project, I have to write a lot more code to set up database records than I do to actually test the object under test. Currently, I try to use test fixtures to store the related fields, but could I use mock objects to mock out the related tables that take so much work to set up?

Here's a trivial example. I want to test that a Person object will spawn() children according to its health.

In this case, a person's city is a required field, so I have to set up a city before I can create a person, even though the city is completely irrelevant to the spawn() method. How could I simplify this test to not require creating a city? (In a typical example, the irrelevant but required set up could be tens or hundreds of records instead of just one.)

# Tested with Django 1.9.2
import sysimport django
from django.apps import apps
from django.apps.config import AppConfig
from django.conf import settings
from django.db import connections, models, DEFAULT_DB_ALIAS
from django.db.models.base import ModelBaseNAME = 'udjango'def main():setup()class City(models.Model):name = models.CharField(max_length=100)class Person(models.Model):name = models.CharField(max_length=50)city = models.ForeignKey(City, related_name='residents')health = models.IntegerField()def spawn(self):for i in range(self.health):self.children.create(name='Child{}'.format(i))class Child(models.Model):parent = models.ForeignKey(Person, related_name='children')name = models.CharField(max_length=255)syncdb(City)syncdb(Person)syncdb(Child)# A typical unit test would start here.# The set up is irrelevant to the test, but required by the database.city = City.objects.create(name='Vancouver')# Actual testdad = Person.objects.create(name='Dad', health=2, city=city)dad.spawn()# Validationchildren = dad.children.all()num_children = len(children)assert num_children == 2, num_childrenname2 = children[1].nameassert name2 == 'Child1', name2# End of typical unit test.print('Done.')def setup():DB_FILE = NAME + '.db'with open(DB_FILE, 'w'):pass  # wipe the databasesettings.configure(DEBUG=True,DATABASES={DEFAULT_DB_ALIAS: {'ENGINE': 'django.db.backends.sqlite3','NAME': DB_FILE}},LOGGING={'version': 1,'disable_existing_loggers': False,'formatters': {'debug': {'format': '%(asctime)s[%(levelname)s]''%(name)s.%(funcName)s(): %(message)s','datefmt': '%Y-%m-%d %H:%M:%S'}},'handlers': {'console': {'level': 'DEBUG','class': 'logging.StreamHandler','formatter': 'debug'}},'root': {'handlers': ['console'],'level': 'WARN'},'loggers': {"django.db": {"level": "WARN"}}})app_config = AppConfig(NAME, sys.modules['__main__'])apps.populate([app_config])django.setup()original_new_func = ModelBase.__new__@staticmethoddef patched_new(cls, name, bases, attrs):if 'Meta' not in attrs:class Meta:app_label = NAMEattrs['Meta'] = Metareturn original_new_func(cls, name, bases, attrs)ModelBase.__new__ = patched_newdef syncdb(model):""" Standard syncdb expects models to be in reliable locations.Based on https://github.com/django/django/blob/1.9.3/django/core/management/commands/migrate.py#L285"""connection = connections[DEFAULT_DB_ALIAS]with connection.schema_editor() as editor:editor.create_model(model)main()
Answer

It took a while to figure out exactly what to mock, but it is possible. You mock out the one-to-many field manager, but you have to mock it out on the class, not on the instance. Here's the core of the test with a mocked out manager.

Person.children = Mock()
dad = Person(health=2)
dad.spawn()num_children = len(Person.children.create.mock_calls)
assert num_children == 2, num_childrenPerson.children.create.assert_called_with(name='Child1')

One problem with that is that later tests will probably fail because you left the manager mocked out. Here's a full example with a context manager to mock out all the related fields, and then put them back when you leave the context.

# Tested with Django 1.9.2
from contextlib import contextmanager
from mock import Mock
import sysimport django
from django.apps import apps
from django.apps.config import AppConfig
from django.conf import settings
from django.db import connections, models, DEFAULT_DB_ALIAS
from django.db.models.base import ModelBaseNAME = 'udjango'def main():setup()class City(models.Model):name = models.CharField(max_length=100)class Person(models.Model):name = models.CharField(max_length=50)city = models.ForeignKey(City, related_name='residents')health = models.IntegerField()def spawn(self):for i in range(self.health):self.children.create(name='Child{}'.format(i))class Child(models.Model):parent = models.ForeignKey(Person, related_name='children')name = models.CharField(max_length=255)syncdb(City)syncdb(Person)syncdb(Child)# A typical unit test would start here.# The irrelevant set up of a city and name is no longer required.with mock_relations(Person):dad = Person(health=2)dad.spawn()# Validationnum_children = len(Person.children.create.mock_calls)assert num_children == 2, num_childrenPerson.children.create.assert_called_with(name='Child1')# End of typical unit test.print('Done.')@contextmanager
def mock_relations(model):model_name = model._meta.object_namemodel.old_relations = {}model.old_objects = model.objectstry:for related_object in model._meta.related_objects:name = related_object.namemodel.old_relations[name] = getattr(model, name)setattr(model, name, Mock(name='{}.{}'.format(model_name, name)))setattr(model, 'objects', Mock(name=model_name + '.objects'))yieldfinally:model.objects = model.old_objectsfor name, relation in model.old_relations.iteritems():setattr(model, name, relation)del model.old_objectsdel model.old_relationsdef setup():DB_FILE = NAME + '.db'with open(DB_FILE, 'w'):pass  # wipe the databasesettings.configure(DEBUG=True,DATABASES={DEFAULT_DB_ALIAS: {'ENGINE': 'django.db.backends.sqlite3','NAME': DB_FILE}},LOGGING={'version': 1,'disable_existing_loggers': False,'formatters': {'debug': {'format': '%(asctime)s[%(levelname)s]''%(name)s.%(funcName)s(): %(message)s','datefmt': '%Y-%m-%d %H:%M:%S'}},'handlers': {'console': {'level': 'DEBUG','class': 'logging.StreamHandler','formatter': 'debug'}},'root': {'handlers': ['console'],'level': 'WARN'},'loggers': {"django.db": {"level": "WARN"}}})app_config = AppConfig(NAME, sys.modules['__main__'])apps.populate([app_config])django.setup()original_new_func = ModelBase.__new__@staticmethoddef patched_new(cls, name, bases, attrs):if 'Meta' not in attrs:class Meta:app_label = NAMEattrs['Meta'] = Metareturn original_new_func(cls, name, bases, attrs)ModelBase.__new__ = patched_newdef syncdb(model):""" Standard syncdb expects models to be in reliable locations.Based on https://github.com/django/django/blob/1.9.3/django/core/management/commands/migrate.py#L285"""connection = connections[DEFAULT_DB_ALIAS]with connection.schema_editor() as editor:editor.create_model(model)main()

You can mix mocked tests in with your regular Django tests, but we found that the Django tests got slower as we added more and more migrations. To skip the test database creation when we run the mocked tests, we added a mock_setup module. It has to be imported before any Django models, and it does a minimal set up of the Django framework before the tests run. It also holds the mock_relations() function.

from contextlib import contextmanager
from mock import Mock
import osimport django
from django.apps import apps
from django.db import connections
from django.conf import settingsif not apps.ready:# Do the Django set up when running as a stand-alone unit test.# That's why this module has to be imported before any Django models.if 'DJANGO_SETTINGS_MODULE' not in os.environ:os.environ['DJANGO_SETTINGS_MODULE'] = 'kive.settings'settings.LOGGING['handlers']['console']['level'] = 'CRITICAL'django.setup()# Disable database access, these are pure unit tests.db = connections.databases['default']db['PASSWORD'] = '****'db['USER'] = '**Database disabled for unit tests**'@contextmanager
def mock_relations(*models):""" Mock all related field managers to make pure unit tests possible.with mock_relations(Dataset):dataset = Dataset()check = dataset.content_checks.create()  # returns mock object"""try:for model in models:model_name = model._meta.object_namemodel.old_relations = {}model.old_objects = model.objectsfor related_object in model._meta.related_objects:name = related_object.namemodel.old_relations[name] = getattr(model, name)setattr(model, name, Mock(name='{}.{}'.format(model_name, name)))model.objects = Mock(name=model_name + '.objects')yieldfinally:for model in models:old_objects = getattr(model, 'old_objects', None)if old_objects is not None:model.objects = old_objectsdel model.old_objectsold_relations = getattr(model, 'old_relations', None)if old_relations is not None:for name, relation in old_relations.iteritems():setattr(model, name, relation)del model.old_relations

Now when the mock tests are run with the regular Django tests, they use the regular Django framework that's already set up. When the mock tests are run on their own, they do a minimal set up. That set up has evolved over time to help test new scenarios, so look at the latest version. One very useful tool is the django-mock-queries library that provides a lot of the QuerySet features in memory.

We put all our mock tests in files named tests_mock.py, so we can run all the mock tests for all the apps like this:

python -m unittest discover -p 'tests_mock.py'

You can see an example mock test on GitHub.

https://en.xdnf.cn/q/72940.html

Related Q&A

How to use tf.data.Dataset.padded_batch with a nested shape?

I am building a dataset with two tensors of shape [batch,width,heigh,3] and [batch,class] for each element. For simplicity lets say class = 5.What shape do you feed to dataset.padded_batch(1000,shape)…

Python, thread and gobject

I am writing a program by a framework using pygtk. The main program doing the following things:Create a watchdog thread to monitor some resource Create a client to receive data from socket call gobjec…

How to type annotate overrided methods in a subclass?

Say I already have a method with type annotations:class Shape:def area(self) -> float:raise NotImplementedErrorWhich I will then subclass multiple times:class Circle:def area(self) -> float:retur…

Import Error: No module named pytz after using easy_install

Today is my first day at Python and have been going through problems. One that I was working on was, "Write a short program which extracts the current date and time from the operating system and p…

Python catch exception pandas.errors.ParserError: Error tokenizing data. C error

I am facing problem with my malfunction csv input file whole reading and which i can deal with by adding "error_bad_lines=False" in my read_csv func to remove those.But i need to report these…

Nested tags in BeautifulSoup - Python

Ive looked at many examples on websites and on stackoverflow but I couldnt find a universal solution to my question. Im dealing with a really messy website and Id like to scrape some data. The markup l…

How do I check if a string is a negative number before passing it through int()?

Im trying to write something that checks if a string is a number or a negative. If its a number (positive or negative) it will passed through int(). Unfortunately isdigit() wont recognize it as a numbe…

openpyxl chage font size of title y_axis.title

I am currently struggling with changing the font of y axis title & the charts title itself.I have tried to create a font setting & applying it to the titles - with no luck what so ever. new_cha…

Combination of all possible cases of a string

I am trying to create a program to generate all possible capitalization cases of a string in python. For example, given abcedfghij, I want a program to generate: Abcdefghij ABcdef.. . . aBcdef.. . ABCD…

How to change download directory location path in Selenium using Chrome?

Im using Selenium in Python and Im trying to change the download path. But either this: prefs = {"download.default_directory": "C:\\Users\\personal\\Downloads\\exports"} options.add…