using pandas read_excel to read from stdin

2024/11/18 22:49:48

Note: I have solve this problem as per below:

I can use to_csv to write to stdout in python / pandas. Something like this works fine:

final_df.to_csv(sys.stdout, index=False)

I would like to read in an actual excel file (not a csv). I want to output CSV, but input xlsx. I have this file

bls_df = pd.read_excel(sys.stdin, sheet_name="MSA_dl", index_col=None)

But that doesn't seem to work. Is it possible to do what I'm trying and, if so, how does one do it?

Notes:

  1. The actual input file is "MSA_M2018_dl.xlsx" which is in the zip file https://www.bls.gov/oes/special.requests/oesm18ma.zip.

I download and extract the datafile like this:

curl -o oesm18ma.zip'https://www.bls.gov/oes/special.requests/oesm18ma.zip'
7z x oesm18ma.zip
  1. I have solved the problem as follows, with script test01.py that reads from stdin and writes to stdout. NOTE the use of sys.stdin.buffer in the read_excel() call.

    import sys import os import pandas as pd

    BLS_DF = pd.read_excel(sys.stdin.buffer, sheet_name="MSA_dl", index_col=None)

    BLS_DF.to_csv(sys.stdout, index=False)

  2. I invoke this as:

    cat MSA_M2018_dl.xlsx | python3 test01.py

  3. This is a small test program to illustrate the idea while removing complexity. It's not the actual program I'm working on.

Answer

Basing on this answer, a possibility would be:

import sys
import pandas as pd
import iocsv = ""
for line in sys.stdin:csv += linedf = pd.read_csv(io.StringIO(csv))
https://en.xdnf.cn/q/120022.html

Related Q&A

How to print a string x times based on user input [duplicate]

This question already has answers here:How can I read inputs as numbers?(10 answers)Understanding for loops in Python(4 answers)Closed 1 year ago.I am trying to make a simple application that will pri…

How does .join work in Python?

I want to display each row of a SQL query result on a webpage. I found some code, but I dont understand what this line does.u"<br>".join([u"{0}".format(row.combination) for r…

Login, Navigate and Retrieve data behind a proxy with Python

I want, with a python script, to be able to login a website and retrieve some data. This behind my companys proxy.I know that this question seems a duplicate of others that you can find searching, but …

overflow in exp, python

cant really figure out why this error RuntimeWarning: overflow encountered in exp is showing up. The function Im trying to implement is:Id = lambda t_u, yp: Is * (np.exp((Vin(t_u) - L*yp)/(n_Ut*Ut)) - …

Getting a matlab codes results on python

I have code in matlab whose results I would like to use in python code (as a matrix or as a .dat file). Could anyone tell me how this could be done?

My Status object is not saving in Django

When I enter the shell, I run into the following problem:from users.models import Status from django.utils import timezoneStatus.objects.all() >>> [] p = Status() p.status_time = timezone.date…

Too many instances are running for Django Server

When Django server gets started, I can see only one instance of Django server running in the background. But after a while, I can see multiple instances are running.Output:root@GoldenGate:~# ps |grep p…

Not able to display images from static folder using Django

This is my home.html Im not able to display the images in static/images/ folder. Although *[09/Mar/2020 15:52:09] "GET /static/images/mona.jpg HTTP/1.1" 404 1669 * is displayed in terminal.&l…

Regex to match phone number 5ABCDYYZZ [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.Want to improve this question? Add details and clarify the problem by editing this post.Closed 1 year ago.Improve …

Remove an \\n\\t\\t\\t-element from list

I got the following list called "phonenumbers". I struggle to remove the elements which contain \n\t\t\t and \n\t\t\t\t. I tried "try and except"-methode and remove(\n\t\t\t\t) but …