Appending to the end of a file in a concurrent environment

2024/10/10 14:22:48

What steps need to be taken to ensure that "full" lines are always correctly appended to the end of a file if multiple of the following (example) program are running concurrently.

#!/usr/bin/env python
import random
passwd_text=open("passwd.txt","a+")
u=("jsmith:x:1001:1000:Joe Smith,Room 1007,(234)555-8917,(234)555-0077,[email protected]:/home/jsmith:/bin/sh","jdoe:x:1002:1000:Jane Doe,Room 1004,(234)555-8914,(234)555-0044,[email protected]:/home/jdoe:/bin/sh","xyz:x:1003:1000:X Yz,Room 1003,(234)555-8913,(234)555-0033,[email protected]:/home/xyz:/bin/sh")
for i in range(random.randint(1,2)):print >> passwd_text, random.choice(u)
passwd_text.close()

And: Can an "all or nothing" append be guaranteed (on linux/unix) even if the the disk becomes full, or "ulimit -f" has been set?

(Note similar question: How do you append to a file?)

Answer

I think the discussion of this "bug" in python's normal open function suggests that you don't get the POSIX atomic guarantee, but you do if you use

with io.open('myfile', 'a') as f:f.write('stuff')

http://docs.python.org/2/library/io.html#io.open

if the operating system implements its write sys call correctly...

http://bugs.python.org/issue15723

https://en.xdnf.cn/q/69888.html

Related Q&A

Cython Pickling in Package not found as Error

Im having trouble pickling a Cython class, but only when its defined inside a package. This problem was noted previously online, but they didnt state how it was resolved. There are two components here:…

How can I process images faster with Python?

Id trying to write a script that will detect an RGB value on the screen then click the x,y values. I know how to perform the click but I need to process the image a lot faster than my code below curren…

KFolds Cross Validation vs train_test_split

I just built my first random forest classifier today and I am trying to improve its performance. I was reading about how cross-validation is important to avoid overfitting of data and hence obtain bett…

Using Keras, how can I input an X_train of images (more than a thousand images)?

My application is accident-avoidance car systems using Machine Learning (Convolutional Neural Networks). My images are 200x100 JPG images and the output is an array of 4 elements: the car would move le…

Fastest way to merge two deques

Exist a faster way to merge two deques than this?# a, b are two deques. The maximum length # of a is greater than the current length # of a plus the current length of bwhile len(b):a.append(b.poplef…

Python cannot find shared library in cron

My Python script runs well in the shell. However when I cron it (under my own account) it gives me the following error:/usr/local/bin/python: error while loading shared libraries: libpython2.7.so.1.0: …

Multiple async unit tests fail, but running them one by one will pass

I have two unit tests, if I run them one by one, they pass. If I run them at class level, one pass and the other one fails at response = await ac.post( with the error message: RuntimeError: Event loop…

Pyusb on Windows 7 cannot find any devices

So I installed Pyusb 1.0.0-alpha-1 Under Windows, I cannot get any handles to usb devices.>>> import usb.core >>> print usb.core.find() NoneI do have 1 usb device plugged in(idVendor=…

How to speed up nested for loops in Python

I have the following Python 2.7 code:listOfLists = [] for l1_index, l1 in enumerate(L1):list = []for l2 in L2:for l3_index,l3 in enumerate(L3):if (L4[l2-1] == l3):value = L5[l2-1] * l1[l3_index]list.ap…

Folium Search Plugin No Results for FeatureGroup

Im trying to add search functionality to a map Im generating in Python with Folium. I see there is a handy Search plugin available and able to implement it successfully and get it added to the map. Unf…