Converting bits to bytes in Python

2024/10/1 3:35:17

I am trying to convert a bit string into a byte string, in Python 3.x. In each byte, bits are filled from high order to low order. The last byte is filled with zeros if necessary. The bit string is initially stored as a "collection" of booleans or integers (0 or 1), and I want to return a "collection" of integers in the range 0-255. By collection, I mean a list or a similar object, but not a character string: for example, the function below returns a generator.

So far, the fastest I am able to get is the following:

def bitsToBytes(a):s = i = 0for x in a:s += s + xi += 1if i == 8:yield ss = i = 0if i > 0:yield s << (8 - i)

I have tried several alternatives: using enumerate, bulding a list instead of a generator, computing s by "(s << 1) | x" instead of the sum, and everything seems to be a bit slower. Since this solution is also one of the shortest and simplest I found, I am rather happy with it.

However, I would like to know if there is a faster solution. Especially, is there a library routine the would do the job much faster, preferably in the standard library?


Example input/output

[] -> []
[1] -> [128]
[1,1] -> [192]
[1,0,0,0,0,0,0,0,1] -> [128,128]

Here I show the examples with lists. Generators would be fine. However, string would not, and then it would be necessary to convert back and foth between list-like data an string.

Answer

The simplest tactics to consume bits in 8-er chunks and ignore exceptions:

def getbytes(bits):done = Falsewhile not done:byte = 0for _ in range(0, 8):try:bit = next(bits)except StopIteration:bit = 0done = Truebyte = (byte << 1) | bityield byte

Usage:

lst = [1,0,0,0,0,0,0,0,1]
for b in getbytes(iter(lst)):print b

getbytes is a generator and accepts a generator, that is, it works fine with large and potentially infinite streams.

https://en.xdnf.cn/q/71010.html

Related Q&A

Install python package from private pypiserver

I have setup a pypiserver behind an nginx proxy which uses htpasswd for authentication. I am currently able to upload sdists, but I cant figure out how to download them. I want to be able to download t…

Matplotlib Table- Assign different text alignments to different columns

I am creating a two column table and want the text to be as close as possible. How can I specify that the first column be right aligned and the second be left aligned?Ive tried by setting the general …

How would I make a random hexdigit code generator using .join and for loops?

I am new to programming and one assignment I have to do is create a random hexdigit colour code generator using for loops and .join. Is my program below even close to how you do it, or is it completely…

Multiline python regex

I have a file structured like this : A: some text B: more text even more text on several lines A: and we start again B: more text more multiline textIm trying to find the regex that will split my file …

How can I process xml asynchronously in python?

I have a large XML data file (>160M) to process, and it seems like SAX/expat/pulldom parsing is the way to go. Id like to have a thread that sifts through the nodes and pushes nodes to be processed …

python postgresql: reliably check for updates in a specific table

Situation: I have a live trading script which computes all sorts of stuff every x minutes in my main thread (Python). the order sending is performed through such thread. the reception and execution of …

How to push to remote repo with GitPython

I have to clone a set of projects from one repository and push it then to a remote repository automatically. Therefore im using python and the specific module GitPython. Until now i can clone the proje…

How do I do use non-integer string labels with SVM from scikit-learn? Python

Scikit-learn has fairly user-friendly python modules for machine learning.I am trying to train an SVM tagger for Natural Language Processing (NLP) where my labels and input data are words and annotatio…

Python - walk through a huge set of files but in a more efficient manner

I have huge set of files that I want to traverse through using python. I am using os.walk(source) for the same and is working but since I have a huge set of files it is taking too much and memory resou…

Python: handling a large set of data. Scipy or Rpy? And how?

In my python environment, the Rpy and Scipy packages are already installed. The problem I want to tackle is such:1) A huge set of financial data are stored in a text file. Loading into Excel is not pos…