Python 2.7: import performance

2024/7/7 6:50:02

currently, I am importing bunch of .py files scattered across the file system via:

def do_import(name):import impfp, pathname, description = imp.find_module(name)with fp:return imp.load_module(name, fp, pathname, description)known_py_files = ['workingDir/import_one.py', 'anotherDir/import_two.py'] # and so forth
for py_file in known_py_files:do_import(py_file)

when I've timed the .py files as such below, they are in the magnitude of e-5 and e-6.

import_one.py

import timeimport_stime = time.time()
import_dur = time.time() - import_stime
print import_dur

However, the call to do_import() is in the magnitude of e-3. I am guessing because of the overhead of importing it.

This is a problematic for me because im importing lots of files serially and the time to import adds up.

Is there a faster way to import than the approach mentioned above?

Answer

If all of the files are under one directory, you can create an __init__.py empty file in each level of the nested directory, and import the name of the root directory, like import root in the following example:

/ root- __init__.py/ workingDir- __init__.py- import_one.py/ anotherDir- __init__.py- import_two.py

That structure is called "package".

https://en.xdnf.cn/q/119875.html

Related Q&A

change first line of a file using bash

I need to change a single line in a file. It is always in the first line of the file. It looks like:h\tn0 n1 n2 n3 n4 n5 n6 n7 n8 n9 hilu cjt 1 1000000there is a tab in all gaps except …

1 to 2 matching in two dataframes with different sizes in Python/R

please help me with this problem Ive been struggling all day lol, solution in either Python or R is fine! Please help Im really stuck!!! I have two dataframes - df1 has 44 rows, df2 has 100 rows, they …

Can someone fix this? pip install intents

Im not sure why this is failing but I need help to fix it please Ive been trying to fix it for about 2 hours now and cant figure it out. I have tried to restart my computer, look it up on google etc. b…

Calculating Average Performance Measures in M/M/1 System Using SimPy

I am seeking to calculate the average waiting times, and average service times in the following M/M/1 queueing system, but I am not able to calculate the averages. It writes down to the console each cu…

Python: Whats the difference between import X and from X import *? [duplicate]

This question already has answers here:Use import module or from module import?(23 answers)Closed 7 years ago.I use to think both are equal until I tried this:$python Python 2.7.13 (default, Dec 17 20…

Python iterate through pixels of image

Im trying to iterate through pixels of an image. I set the size and then use a for loop, however I get a type error: object not iterable. I have imported PIL and Imagew=100 h=200 im=im.resize((w,h), Im…

Geocoding with Geopy and big data

I have this CSV file which im feeding with this python scriptimport csv from geopy.geocoders import OpenCagegeolocator = OpenCage() #here some parameters are needed with open(/Users/Ian/Desktop/Test02/…

an error in sending json data to flask server [duplicate]

This question already has answers here:How to get POSTed JSON in Flask?(13 answers)Closed 1 year ago.I have a json data as {"age":59.0,"bp":70.0,"sg":1.01,"al":…

terminate a python program when it hanged using subprocess python

I have a main.py which open a new cmd (subprocess) when another program (test.py, in same directory) is hanged. To determining test.py is hanged or not, I used latest modified time (os.path.getmtime(te…

causes of Python IOError: [Errno 13] Permission denied

When attempting to write a file, I can get this same error when any of following conditions applies:The file exists and is marked read-only. I dont have write permission for the folder and therefore ca…