Ignore newline character in binary file with Python?

2024/10/12 18:18:54

I open my file like so :

f = open("filename.ext", "rb") # ensure binary reading with b

My first line of data looks like this (when using f.readline()):

'\x04\x00\x00\x00\x12\x00\x00\x00\x04\x00\x00\x00\xb4\x00\x00\x00\x01\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x18\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x05\x00\x00\x00\x06\x00\x00\x00:\x00\x00\x00;\x00\x00\x00<\x00\x00\x007\x00\x00\x008\x00\x00\x009\x00\x00\x00\x07\x00\x00\x00\x08\x00\x00\x00\t\x00\x00\x00\n'

Thing is, I want to read this data byte by byte (f.read(4)). While debugging, I realized that when it gets to the end of the first line, it still takes in the newline character \n and it is used as the first byte of the following int I read. I don't want to simply use .splitlines()because some data could have an n inside and I don't want to corrupt it. I'm using Python 2.7.10, by the way. I also read that opening a binary file with the b parameter "takes care" of the new line/end of line characters; why is not the case with me?

This is what happens in the console as the file's position is right before the newline character:

>>> d = f.read(4)
>>> d
'\n\x00\x00\x00'
>>> s = struct.unpack("i", d)
>>> s
(10,)
Answer

(Followed from discussion with OP in chat)

Seems like the file is in binary format and the newlines are just mis-interpreted values. This can happen when writing 10 to the file for example.

This doesn't mean that newline was intended, and it is probably not. You can just ignore it being printed as \n and just use it as data.

https://en.xdnf.cn/q/118169.html

Related Q&A

RegEx Parse Error by Parsley Python

I have made a simple parser for simple queries, to fetch data from a datastore. The operands I have used are <,<=,>,>=,==,!= The Parser works fine for every operand except for < I am a b…

Accessing Bangla (UTF-8) string by index in Python

I have a string in Bangla and Im trying to access characters by index.# -*- coding: utf-8 -*- bstr = "তরদজ" print bstr # This line is working fine for i in bstr:print i, # question marks …

Computing KL divergence for many distributions

I have a matrix of test probability distributions:qs = np.array([[0.1, 0.6], [0.9, 0.4] ])(sums up to 1 in each column) and "true" distribution:p = np.array([0.5, 0.5])I would like to calcula…

Expanding mean over multiple series in pandas

I have a groupby object I apply expanding mean to. However I want that calculation over another series/group at the same time. Here is my code:d = { home : [A, B, B, A, B, A, A], away : [B, A,A, B, A, …

Moving window sum on a boollean array, with steps.

Im struggling with creating a moving window sum function that calculates the number of True values in a given numpy Boolean array my_array, with a window size of n and in jumping steps of s.For example…

Python - take the time difference from the first date in a column

Given the date column, I want to create another column diff that count how many days apart from the first date.date diff 2011-01-01 00:00:10 0 2011-01-01 00:00:11 0.000011 …

(Django) Limited ForeignKey choices by Current User

Update Thanks to Michael I was able to get this to work perfectly in my CreateView, but not in the UpdateView. When I try to set a form_class it spits out an improperly configured error. How can I go a…

Parse a custom text file in Python

I have a text to be parsed, this is a concise form of the text.apple {type=fruitvarieties {color=redorigin=usa} }the output should be as shown belowapple.type=fruit apple.varieties.color=red apple.vari…

Logs Dont Overwrite

Im using Pythons logging.config module to configure and use a logging tool in my project.I want my log files to overwrite each time (not append), so I set my YAML configuration file like this:# logging…

How to upload local files to Firebase storage from Jupyter Notebook using Python

Since I guess importing google.cloud.storage might be a very first step to set API connecting the firebase storage, what I did first is to install google-cloud on Ubuntu like this:$ pip install --upgra…