os.popen().read() - charmap decoding error

2024/9/8 10:39:57

I have already read UnicodeDecodeError: 'charmap' codec can't decode byte X in position Y: character maps to <undefined>. While the error message is similar, the code is completely different, because I use os.popen in this question, not open. I cannot use the answers from the other questions to solve this problem.

output = os.popen("dir").read()

This line, which is supposed to assign the output of command "dir" to variable "output", is causing this error:

'charmap' codec can't decode byte 0x88 in position 260: character maps to <undefined>

I think this might be happenning because some files in the folder contain letters such as ł, ą, ę and ć in their names. I have no idea how to fix this though.

Answer

os.popen is just a wrapper around subprocess.Popen along with a io.TextIOWrapper object:

The returned file object reads or writes text strings rather than bytes.

If Python's default encoding doesn't work for you, you should use subprocess.Popen directly.

The underlying issue is that cmd writes ansi garbage by default, even when the output is to a pipe. This behavior may depend on your Windows version.

You can fix this by passing /U flag to cmd:

p = subprocess.Popen('cmd /u /c dir', stdout=subprocess.PIPE)
result = p.communicate()
text = result[0].decode('u16')
https://en.xdnf.cn/q/73119.html

Related Q&A

multiprocessing numpy not defined error

I am using the following test code:from pathos.multiprocessing import ProcessingPool as Pool import numpydef foo(obj1, obj2):a = obj1**2b = numpy.asarray(range(1,5))return obj1, bif __name__ == __main_…

Convert mp4 sound to text in python

I want to convert a sound recording from Facebook Messenger to text. Here is an example of an .mp4 file send using Facebooks API: https://cdn.fbsbx.com/v/t59.3654-21/15720510_10211855778255994_5430581…

pandas map one column to the combination of two columns

I am working with a DataFrame which looks like thisList Numb Name 1 1 one 1 2 two 2 3 three 4 4 four 3 5 fiveand I am trying to compute…

Pyspark command not recognised

I have anaconda installed and also I have downloaded Spark 1.6.2. I am using the following instructions from this answer to configure spark for Jupyter enter link description hereI have downloaded and …

Sliding window in Python for GLCM calculation

I am trying to do texture analysis in a satellite imagery using GLCM algorithm. The scikit-image documentation is very helpful on that but for GLCM calculation we need a window size looping over the im…

Keras multi-label image classification with F1-score

I am working on a multi-label image classification problem with the evaluation being conducted in terms of F1-score between system predicted and ground truth labels.Given that, should I use loss="…

Thread safe locale techniques

Were currently writing a web application based on a threaded python web server framework (cherrypy) and would like to simultaneously support users from multiple locales.The locale module doesnt appear …

How to draw image from raw bytes using ReportLab?

All the examples I encounter in the internet is loading the image from url (either locally or in the web). What I want is to draw the image directly to the pdf from raw bytes.UPDATE:@georgexsh Here is …

How to speed up numpy code

I have the following code. In principle it takes 2^6 * 1000 = 64000 iterations which is quite a small number. However it takes 9s on my computer and I would like to run it for n = 15 at least.from __f…

MyPy gives error Missing return statement even when all cases are tested

I am getting a MyPy error "Missing return statement", even when I check for all possible cases inside a function.For example, in the following code, MyPy is still giving me an error "9: …