sqlalchemy autoloaded orm persistence

2024/11/16 13:37:21

We are using sqlalchemy's autoload feature to do column mapping to prevent hardcoding in our code.

class users(Base):__tablename__ = 'users'__table_args__ = {'autoload': True,'mysql_engine': 'InnoDB','mysql_charset': 'utf8'}

Is there a way to serialize or cache autoloaded metadata/orms so we don't have to go through the autoload process every time we need to reference our orm classes from other scripts/functions?

I have looked at beaker caching and pickle but haven't found a clear answer if it is possible or how to do it.

Ideally we run the autload mapping script only when we have committed changes to our database structure but reference a non-autoload/persistent/cached version of our database mapping from all other scripts/functions,

Any ideas?

Answer

What I am doing now is to pickle the metadata after running the reflection through a database connection (MySQL) and once a pickle is available use that pickled metadata to reflect on the schema with the metadata bound to an SQLite engine.

cachefile='orm.p'
dbfile='database'
engine_dev = create_engine(#db connect, echo=True)
engine_meta = create_engine('sqlite:///%s' % dbfile,echo=True)
Base = declarative_base()
Base.metadata.bind = engine_dev
metadata = MetaData(bind=engine_dev)# load from pickle 
try:with open(cachefile, 'r') as cache:metadata2 = pickle.load(cache)metadata2.bind = engine_metacache.close()class Users(Base):__table__ = Table('users', metadata2, autoload=True)print "ORM loaded from pickle"# if no pickle, use reflect through database connection    
except:class Users(Base):__table__ = Table('users', metadata, autoload=True)print "ORM through database autoload"# create metapickle
metadata.create_all()
with open(cachefile, 'w') as cache:pickle.dump(metadata, cache)cache.close()

Any comments if this is alright (it works) or there is something I can improve?

https://en.xdnf.cn/q/71341.html

Related Q&A

Data Normalization with tensorflow tf-transform

Im doing a neural network prediction with my own datasets using Tensorflow. The first I did was a model that works with a small dataset in my computer. After this, I changed the code a little bit in or…

Relationship of metaclasss __call__ and instances __init__?

Say Ive got a metaclass and a class using it:class Meta(type):def __call__(cls, *args):print "Meta: __call__ with", argsclass ProductClass(object):__metaclass__ = Metadef __init__(self, *args…

How to present numpy array into pygame surface?

Im writing a code that part of it is reading an image source and displaying it on the screen for the user to interact with. I also need the sharpened image data. I use the following to read the data an…

Following backreferences of unknown kinds in NDB

Im in the process of writing my first RESTful web service atop GAE and the Python 2.7 runtime; Ive started out using Guidos shiny new ndb API.However, Im unsure how to solve a particular case without t…

How to enable math in sphinx?

I am using sphinx with the pngmath extension to document my code that has a lot of mathematical expressions. Doing that in a *.rst file is working just fine.a \times b becomes: However, if I try the sa…

How to set the xticklabels for date in matplotlib

I am trying to plot values from two list. The x axis values are date. Tried these things so faryear = [20070102,20070806,20091208,20111109,20120816,20140117,20140813] yvalues = [-0.5,-0.5,-0.75,-0.75,…

PyParsing: Is this correct use of setParseAction()?

I have strings like this:"MSE 2110, 3030, 4102"I would like to output:[("MSE", 2110), ("MSE", 3030), ("MSE", 4102)]This is my way of going about it, although I h…

Indent and comments in function in Python

I am using Python 2.7 and wrote the following:def arithmetic(A):x=1 """ Some comments here """ if x=1:x=1elif x=2:x=2return 0But it has the indentation issue:if x=1:^ Ind…

Read a large big-endian binary file

I have a very large big-endian binary file. I know how many numbers in this file. I found a solution how to read big-endian file using struct and it works perfect if file is small:data = []file = open(…

SWIG Python Structure Array

Ive been searching for a few days trying to figure out how to turn an array of structures into a Python list. I have a function that returns a pointer to the beginning of the array.struct foo {int memb…