Integer in python/pandas becomes BLOB (binary) in sqlite

2024/10/11 8:24:11

Storing an integer in sqlite results in BLOBs (binary values) instead of INTEGER in sqlite. The problem is the INT in the "Baujahr" column. The table is created.

CREATE TABLE "Objekt" ( `No` INTEGER NOT NULL UNIQUE, `Objekt_id` INTEGER, 
`Strecke` TEXT, `Baujahr` INTEGER, `Eigentümer` TEXT, PRIMARY KEY(`No`) )

dataframe and dtypes:

id  Strecke  Baujahr Eigentümer Objekt_id
5     A23     1938    Fronz      0327Objekt.dtypes
Strecke       object
Baujahr        int64
Eigentümer    object
Objekt_id     object
dtype: object

The DataFrame ist written to sqlite

stmt ="INSERT INTO Objekt (Objekt_id, Strecke, Baujahr, Eigentümer) VALUES (?, ?, ?, ?)"
c.execute(stmt, (Objekt.Objekt_id.values[0], Objekt.Strecke.values[0], 
Objekt.Baujahr.values[0], Objekt.Eigentümer.values[0] ))
conn.commit()

Sqlite works only up to INT 8 and works not with INT 32 or INT64 (The definition BIGINT in the CREATE TABLE .... does not help either). So I tried to convert using these conversions

Objekt.Baujahr.astype(int)
Objekt.Baujahr.astype(np.int8)
Objekt.Baujahr = int(Objekt.Baujahr)

The dtypes command shows that Baujahr remained int64!!

I cannot edit the values in the database and by querying these values i get a binary back. Any idea?

Python 3.6.4, sqlite3 2.6.0, pandas 0.22.0

Answer

For some reason Sqlite does not accept INT larger than 8 byte. Therefore it is necessary to add the following statements.

sqlite3.register_adapter(np.int64, lambda val: int(val))
sqlite3.register_adapter(np.int32, lambda val: int(val))

The docs in sqlite are at this point a little bit short. But it works perfectly.

https://en.xdnf.cn/q/69797.html

Related Q&A

Calling Scrapy Spider from Django

I have a project with a django and scrapy folder in the same workspace:my_project/django_project/django_project/settings.pyapp1/app2/manage.py...scrapy_project/scrapy_project/settings.pyscrapy.cfg...Iv…

Python Threading: Multiple While True loops

Do you guys have any recommendations on what python modules to use for the following application: I would like to create a daemon which runs 2 threads, both with while True: loops. Any examples would b…

Visual Studio Code - input function in Python

I am trying out Visual Studio Code, to learn Python.I am writing a starter piece of code to just take an input from the user, say:S = input("Whats your name? ")When I try to run this (Mac: C…

DRF: how to change the value of the model fields before saving to the database

If I need to change some field values before saving to the database as I think models method clear() is suitable. But I cant call him despite all my efforts.For example fields email I need set to lowe…

keep matplotlib / pyplot windows open after code termination

Id like python to make a plot, display it without blocking the control flow, and leave the plot open after the code exits. Is this possible?This, and related subjects exist (see below) in numerous ot…

socket python : recvfrom

I would like to know if socket.recvfrom in python is a blocking function ? I couldnt find my answer in the documentation If it isnt, what will be return if nothing is receive ? An empty string ? In…

pandas read_excel(sheet name = None) returns a dictionary of strings, not dataframes?

The pandas read_excel documentation says that specifying sheet_name = None should return "All sheets as a dictionary of DataFrames". However when I try to use it like so I get a dictionary of…

Plotly: How to assign specific colors for categories? [duplicate]

This question already has an answer here:How to define colors in a figure using Plotly Graph Objects and Plotly Express(1 answer)Closed 2 years ago.I have a pandas dataframe of electricity generation m…

Using nested asyncio.gather() inside another asyncio.gather()

I have a class with various methods. I have a method in that class something like :class MyClass:async def master_method(self):tasks = [self.sub_method() for _ in range(10)]results = await asyncio.gath…

AttributeError: type object Word2Vec has no attribute load_word2vec_format

I am trying to implement word2vec model and getting Attribute error AttributeError: type object Word2Vec has no attribute load_word2vec_formatBelow is the code :wv = Word2Vec.load_word2vec_format("…