Best way to access and close a postgres database using python dataset

2024/10/15 11:22:32
import dataset    
from sqlalchemy.pool import NullPooldb = dataset.connect(path_database, engine_kwargs={'poolclass': NullPool})table_f1 = db['name_table']
# Do operations on table_f1db.commit()
db.executable.close()

I use this code to access a postgres database and sometimes write to it. Finally, I close it. Is the above code the best way to access and close it? Alternatively, is the code below better?

import dataset    
from sqlalchemy.pool import NullPoolwith dataset.connect(path_database, engine_kwargs={'poolclass': NullPool}) as db:table_f1 = db['name_table']# Do operations on table_f1db.commit()

In particular, I want to make 100% sure that there is no connection to the postgres database once this piece of code is done. Which is the better way to achieve it? option 1 or option 2?

Answer

For now, the main issue is that the context manager used in Option 2 (with statement) doesn't handle the connection, only the transaction (commit/rollback at the end of the block).

(This question is already reported to the Github repo, maybe the behavior will change ?)

So you should replace db.commit() by db.executable.close() in Option 2:

import dataset    
from sqlalchemy.pool import NullPoolwith dataset.connect(path_database, engine_kwargs={'poolclass': NullPool}) as db:table_f1 = db['name_table']print(db.local.conn.closed) # >>>False# Do operations on table_f1# end of the context manager, trying to commit db.executable.close()
print(db.local.conn.closed) # >>>True

Now connection is closed :

# db['name_table'].all() ==> throws an error due to closed connection

BUT... you can still create new tables in the database (because of Metadata ?) :

# db['new_table'] ==> enough to add a new table 

So you may want to destroy everything to prevent this (db = None, or db.metadata = None)


This last behavior happens in SQLAlchemy too:

from sqlalchemy import *
from sqlalchemy.pool import NullPoolengine = create_engine('postgresql:///datatest', poolclass=NullPool) connection = engine.connect()
meta = MetaData(engine)
t1 = Table('Table_1', meta,Column('id', Integer, primary_key=True),Column('name',String))
t1.create()
connection.close()t2 = Table('Table_2', meta,Column('id', Integer, primary_key=True),Column('name',String))
t2.create()
# table_2 is created in database

EDIT:

(thanks to Ilja Everilä's comment, and a focus on the doc)

Better call meta = MetaData(connection) in order to close the connection at the engine disposal, this will raise an error in the above example, connection IS closed.

https://en.xdnf.cn/q/69292.html

Related Q&A

Using different binds in the same class in Flask-SQLAlchemy

I currently have multiple databases with identical Tables and Columns (but different data inside). So clearly I need to use binds to access all of them, but its apparently not as simple as doing this:c…

Correctly parse date string with timezone information

Im receiving a formatted date string like this via the pivotal tracker API: "2012/06/05 17:42:29 CEST"I want to convert this string to a UTC datetime object, it looks like python-dateutil doe…

Can I add a sequence of markers on a Folium map?

Suppose I had a list, or pandas series, or latitude longitude pairs. With Folium, I can plot markers for a single pair of coordinates using coords = [46.8354, -121.7325] map_4 = folium.Map(location=[4…

Tkinter in Python 3.4 on Windows dont post internal clipboard data to the Windows clipboard on exit

I use the following code to place result of my small scripts in clipboard.from tkinter import Tk r = Tk() r.withdraw() r.clipboard_clear() r.clipboard_append("Result")It works fine on Python …

How do I group date by month using pd.Grouper?

Ive searched stackoverflow to find out how to group DateTime by month and for some reason I keep receiving this error, even after I pass the dataframe through pd.to.datetimeTypeError: Only valid with D…

Python Too many indices for array

I am reading a file in python using pandas and then saving it in a numpy array. The file has the dimension of 11303402 rows x 10 columns. I need to split the data for cross validation and for that I …

Removing named entities from a document using spacy

I have tried to remove words from a document that are considered to be named entities by spacy, so basically removing "Sweden" and "Nokia" from the string example. I could not find …

Install wxPython in osx 10.11

When I try to install wxPython, it shows an error: > The Installer could not install the software because there was no > software found to install.How can I fix it?

merging recurrent layers with dense layer in Keras

I want to build a neural network where the two first layers are feedforward and the last one is recurrent. here is my code :model = Sequential() model.add(Dense(150, input_dim=23,init=normal,activation…

How to manually mark a Celery task as done and set its result?

I have this Celery task:@app.task def do_something(with_this):# instantiate a class from a third party libraryinstance = SomeClass()# this class uses callbacks to send progress info about# the status a…