Extracting data from multiple files with python

2024/11/16 12:36:11

I'm trying to extract data from a directory with 12 .txt files. Each file contains 3 columns of data (X,Y,Z) that i want to extract. I want to collect all the data in one df(InforDF), but so far i only succeeded in creating a df with all of the X,Y and Z data in the same column. This is my code:

import pandas as pd
import numpy as np
import os
import fnmatchpath = os.getcwd()file_list = os.listdir(path)InfoDF = pd.DataFrame()for file in file_list:try:if fnmatch.fnmatch(file, '*.txt'):filedata = open(file, 'r')df = pd.read_table(filedata, delim_whitespace=True, names={'X','Y','Z'})except Exception as e:print(e)

What am i doing wrong?

Answer
df = pd.read_table(filedata, delim_whitespace=True, names={'X','Y','Z'})

this line replace df at each iteration of the loop, that's why you only have the last one at the end of your program.

what you can do is to save all your dataframe in a list and concatenate them at the end

df_list = []
for file in file_list:try:if fnmatch.fnmatch(file, '*.txt'): filedata = open(file, 'r')df_list.append(pd.read_table(filedata, delim_whitespace=True, names={'X','Y','Z'}))
df = pd.concat(df_list)

alternatively, you can write it:

df_list = pd.concat([pd.read_table(open(file, 'r'), delim_whitespace=True, names={'X','Y','Z'})  for file in file_list if fnmatch.fnmatch(file, '*.txt')])
https://en.xdnf.cn/q/119191.html

Related Q&A

Python: Checking if string follows wikipedia link format

If I have a string named link, how would I go about checking to see if it follows the same format as a wikipedia URL? To clarify, wikipedia URLs (in this case) always begin with en.wikipedia.org/wiki/…

Create dictionary comprehension from list with condition syntax

Id like to create a dictionary using dictionary comprehension syntax.Note that list l contains tuples of strings and tuples with 1st element always a time stamp.This works:d = {} for entry in l:if entr…

How to find next empty cell on a specific column and write to it?

I want to find the next empty cell in a specific column and write to values that cell. Ive tried it using following method: for row in sheet[A{}:A{}.format(sheet.min_row,sheet.max_row)]:if row is None:…

How can i pass data from template in Django to Javascript in this specific case

I am using Google map api and i am trying to pass the data (longitude and latitude) to the template then use the data in the javascript to show a specific location.location.html{% for venue in propert…

Python Highscores w/ text file

i am currently working on a text based game, and have run into a problem trying to make my last function, highscore. My problem is this, i would like to make the function save my top five scores and sa…

Do any one know about this Error in python? how can I resolve this?

I am plotting the data with MapBoxGl Python Library on maps, here is my code which is taking the latitude, longitude and points from the Pandas DataFrame and trying to make the geojson, here is the cod…

Pandas: Count Higher Ranks For Current Experiment Participants In Later Experiments (Part 1)

Learning Experiments In a series of learning experiments, I would like to count the number of participants in each experiment that improved their performance in subsequent experiments (Rank 1 is highes…

Pass variables into and out of exec [closed for not being clear. Modified and reuploaded] [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.Want to improve this question? Add details and clarify the problem by editing this post.Closed 3 years ago.Improve…

List Object Not Callable, Syntax Error for Text-Based RPG

Im having issues (still) with generating a random object, in this case a random herb found by foraging. Heres the code for the function:def collectPlants(self):if self.state == normal:print"%s spe…

Why does my Tkinter window background not change?

Ive a small program with a feature to change the background color of a different window than the frame I use to ask for the background color. (I hope you understand this.) The program is written in Pyt…