Fastest way to pack a list of floats into bytes in python

2024/11/18 23:26:59

I have a list of say 100k floats and I want to convert it into a bytes buffer.

buf = bytes()
for val in floatList:buf += struct.pack('f', val)
return buf

This is quite slow. How can I make it faster using only standard Python 3.x libraries.

Answer

Just tell struct how many floats you have. 100k floats takes about a 1/100th of a second on my slow laptop.

import random
import structfloatlist = [random.random() for _ in range(10**5)]
buf = struct.pack('%sf' % len(floatlist), *floatlist)
https://en.xdnf.cn/q/26508.html

Related Q&A

PyPI is slow. How do I run my own server?

When a new developer joins the team, or Jenkins runs a complete build, I need to create a fresh virtualenv. I often find that setting up a virtualenv with Pip and a large number (more than 10) of requi…

How to suppress the deprecation warnings in Django?

Every time Im using the django-admin command — even on TAB–completion — it throws a RemovedInDjango19Warning (and a lot more if I use the test command). How can I suppress those warnings?Im using D…

whats the fastest way to find eigenvalues/vectors in python?

Currently im using numpy which does the job. But, as im dealing with matrices with several thousands of rows/columns and later this figure will go up to tens of thousands, i was wondering if there was …

Python Patch/Mock class method but still call original method

I want to use patch to record all function calls made to a function in a class for a unittest, but need the original function to still run as expected. I created a dummy code example below:from mock im…

Daemon vs Upstart for python script

I have written a module in Python and want it to run continuously once started and need to stop it when I need to update other modules. I will likely be using monit to restart it, if module has crashed…

Comparison of Python modes for Emacs

So I have Emacs 24.3 and with it comes a quite recent python.el file providing a Python mode for editing.But I keep reading that there is a python-mode.el on Launchpad, and comparing the two files it j…

Python best formatting practice for lists, dictionary, etc

I have been looking over the Python documentation for code formatting best practice for large lists and dictionaries, for example,something = {foo : bar, foo2 : bar2, foo3 : bar3..... 200 chars wide, e…

TypeError: string indices must be integers, not str // working with dict [duplicate]

This question already has answers here:Why am I seeing "TypeError: string indices must be integers"?(10 answers)Closed 28 days ago.I am trying to define a procedure, involved(courses, person…

Pandas: create dataframe from list of namedtuple

Im new to pandas, therefore perhaps Im asking a very stupid question. Normally initialization of data frame in pandas would be column-wise, where I put in dict with key of column names and values of li…

Closest equivalent of a factor variable in Python Pandas

What is the closest equivalent to an R Factor variable in Python pandas?