python SharedMemory persistence between processes

2024/10/9 0:39:11

Is there any way to make SharedMemory object created in Python persist between processes?

If the following code is invoked in interactive python session:

>>> from multiprocessing import shared_memory
>>> shm = shared_memory.SharedMemory(name='test_smm', size=1000000, create=True)

it creates a file in /dev/shm/ on a Linux machine.

ls /dev/shm/test_smm 
/dev/shm/test_smm

But when the python session ends I get the following:

/usr/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdownwarnings.warn('resource_tracker: There appear to be %d 

and the test_smm is gone:

ls /dev/shm/test_smm 
ls: cannot access '/dev/shm/test_smm': No such file or directory

So is there any way to make the shared memory object created in python persist across process runs?

Running with Python 3.8

Answer

You can unregister a shared memory object from the resource cleanup process without unlinking it:

$ python3
Python 3.8.6 (default, Sep 25 2020, 09:36:53) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from multiprocessing import shared_memory, resource_tracker
>>> shm = shared_memory.SharedMemory(name='test_smm', size=1000000, create=True)
>>> resource_tracker.unregister(shm._name, 'shared_memory')
>>> 
$ ls /dev/shm/test_smm 
/dev/shm/test_smm

I don't know whether this is portable, and it doesn't look like a supported way of using the multiprocessing module, but it works on Linux at least.

https://en.xdnf.cn/q/70079.html

Related Q&A

What is the difference between syntax error and runtime error?

For example:def tofloat(i): return flt(i)def addnums(numlist):total = 0for i in numlist:total += tofloat(i)return totalnums = [1 ,2 ,3] addnums(nums)The flt is supposed to be float, but Im confused whe…

Printing a line at the bottom of the console/terminal

Using Python, I would like to print a line that will appear on the last visible line on the console the script is being ran from. For example, something like this:Would this be able to be done?

Comparing first element of the consecutive lists of tuples in Python

I have a list of tuples, each containing two elements. The first element of few sublists is common. I want to compare the first element of these sublists and append the second element in one lists. Her…

Upload a file using boto

import boto conn = boto.connect_s3(, )mybucket = conn.get_bucket(data_report_321)I can download the file from a bucket using the following code.for b in mybucket:print b.nameb.get_contents_to_filename…

How to get n-gram collocations and association in python nltk?

In this documentation, there is example using nltk.collocations.BigramAssocMeasures(), BigramCollocationFinder,nltk.collocations.TrigramAssocMeasures(), and TrigramCollocationFinder.There is example me…

Using Python3 on macOS as default but pip still get using python 2.7

Im using macOS Big Sur 11.0.1. Im setting up a virtual env $python3 -m venv $my_workdir)/.virtualenvbut getting this error at building wheel package: building _openssl extensioncreating build/temp.maco…

Python Matplotlib Box Plot Two Data Sets Side by Side

I would like to make a boxplot using two data sets. Each set is a list of floats. A and B are examples of the two data setsA = [] B = []for i in xrange(10):l = [random.random() for i in xrange(100)]m =…

perform() and reset_actions() in ActionChains not working selenium python

This is the code that habe no error: perform() and reset_actions() but these two functions have to work combinedly import os import time from selenium import webdriver from selenium.webdriver.common.ac…

nosetests not recognized on Windows after being installed and added to PATH

Im on exercise 46 of Learn Python the Hard Way, and Im meant to install nose and run nosetests. Ive installed nose already using pip, but when I run nosetests in the directory above the tests folder, I…

Using a context manager with mysql connector python

Im moving my code across from an sqlite database to mysql and Im having a problem with the context manager, getting the following attribute error.Ive tried combinations of mydb.cursor() as cursor, mydb…