I have code like this:
f1 = open('file1', 'a')
f2 = open('file1', 'a')f1.write('Test line 1\n')
f2.write('Test line 2\n')
f1.write('Test line 3\n')
f2.write('Test line 4\n')
When this code is run with standard Python 2.7 interpreter, the file contains four lines as expected. However, when I run this code under PyPy, the file contains only two lines.
Could someone explain the differences between Python and PyPy in working with files in append mode?
UPDATED: The problem doesn't exist in the PyPy 2.3.
The reason in different behavior is different implementation of file I/O operations.
CPython implements it's file I/O on top of fopen
, fread
and fwrite
functions from stdio.h
and is working with FILE *
streams.
In the same time PyPy implements it's file I/O on top of POSIX open
, write
and read
functions and is working with int
file descriptors.
Compare these two programs in C:
#include <stdio.h>int main() {FILE *a = fopen("file1", "a");FILE *b = fopen("file1", "a");fwrite("Test line 1\n", 12, 1, a);fflush(a);fwrite("Test line 2\n", 12, 1, b);fflush(b);fwrite("Test line 3\n", 12, 1, a);fflush(a);fwrite("Test line 4\n", 12, 1, b);fclose(a);fclose(b);return 0;
}
and
#include <fcntl.h>
#include <unistd.h>int main() {int a = open("file1", O_CREAT | O_WRONLY | O_APPEND);int b = open("file1", O_CREAT | O_WRONLY | O_APPEND);write(a, "Test line 1\n", 12);write(b, "Test line 2\n", 12);write(a, "Test line 3\n", 12);write(b, "Test line 4\n", 12);close(a);close(b);return 0;
}
More info on difference between open
and fopen
you could find in answers to this question.
UPDATE:
After inspecting PyPy codebase some more, it seems to me it doesn't use O_APPEND
flag by some reason, but O_WRONLY | O_CREAT
for "a" mode. So it is the real reason in PyPy you need to seek
to the end of file after each write
call, as J.F. Sebastian mentioned in another answer. I guess a bug should be created at PyPy bugtracker, as O_APPEND
flag is available both on Windows and Unix. So, what PyPy does now looks like:
#include <fcntl.h>
#include <unistd.h>int main() {int a = open("file1", O_CREAT | O_WRONLY);int b = open("file1", O_CREAT | O_WRONLY);write(a, "Test line 1\n", 12);write(b, "Test line 2\n", 12);write(a, "Test line 3\n", 12);write(b, "Test line 4\n", 12);close(a);close(b);return 0;
}
Without O_APPEND flag it should reproduce PyPy behavior.