Entire JSON into One SQLite Field with Python

2024/9/20 22:45:33

I have what is likely an easy question. I'm trying to pull a JSON from an online source, and store it in a SQLite table. In addition to storing the data in a rich table, corresponding to the many fields in the JSON, I would like to also just dump the entire JSON into a table every time it is pulled.

The table looks like:

CREATE TABLE Raw_JSONs (ID INTEGER PRIMARY KEY ASC, T DATE DEFAULT (datetime('now','localtime')), JSON text);

I've pulled a JSON from some URL using the following python code:

from pyquery import PyQuery
from lxml import etree
import urllibx = PyQuery(url='json')
y = x('p').text()

Now, I'd like to execute the following INSERT command:

import sqlite3db = sqlite3.connect('a.db')
c = db.cursor()c.execute("insert into Raw_JSONs values(NULL,DATETIME('now'),?)", y)

But I'm told that I've supplied the incorrect number bindings (i.e. thousands, instead of just 1). I gather it's reading the y variable as all the different elements of the JSON.

Can someone help me store just the JSON, in it's entirety?

Also, as I'm obviously new to this JSON game, any online resources to recommend would be amazing.

Thanks!

Answer

.execute() expects a sequence, better give it a one-element tuple:

c.execute("insert into Raw_JSONs values(NULL,DATETIME('now'),?)", (y,))

A Python string is a sequence too, one of individual characters. So the .execute() call tried to treat each separate character as a parameter for your query, and unless your string is one character short that means it'll not provide the right number of parameters.

Don't forget to commit your inserts:

db.commit()

or use the database connection as a context manager:

with db:# inserts executed here will automatically commit if no exceptions are raised.
https://en.xdnf.cn/q/72448.html

Related Q&A

Python scipy module import error due to missing ._ufuncs dll

I have some troubles with sub-module integrate from scipy in python. I have a 64 bits architecture, and it seems, according to the first lines of the python interpreter (see below) that I am also using…

How can I call python program from VBA?

Just as the title goes.I have a python program which processes some data file I downloaded from email.I am writing a vba script which can download the email attachments and execute the python program t…

Embedding CPython: how do you constuct Python callables to wrap C callback pointers?

Suppose I am embedding the CPython interpreter into a larger program, written in C. The C component of the program occasionally needs to call functions written in Python, supplying callback functions …

python - beautifulsoup - TypeError: sequence item 0: expected string, Tag found

Im using beautifulsoup to extract images and links from a html string. It all works perfectly fine, however with some links that have a tag in the link contents it is throwing an error.Example Link:<…

Python evdev detect device unplugged

Im using the great "evdev" library to listen to a USB barcode reader input and I need to detect if the device suddenly gets unplugged/unresponsive because otherwise the python script reading …

python: urllib2 using different network interface

I have the following code:f = urllib2.urlopen(url) data = f.read() f.close()Its running on a machine with two network interfaces. Id like to specify which interface I want the code to use. Specifically…

RuntimeError: as_numpy_iterator() is not supported while tracing functions

while i was using function as_numpy_iterator() got error--------------------------------------------------------------------------- RuntimeError Traceback (most recent call…

Pandas assert_frame_equal error

Im building test cases and I want to compare 2 dataframes. Even though dataframe have the same columns and values assert_frame_equal reports are not equal. Column order is different, I tried reordering…

Multiple lines on line plot/time series with matplotlib

How do I plot multiple traces represented by a categorical variable on matplotlib or plot.ly on Python? I am trying to replicate the geom_line(aes(x=Date,y=Value,color=Group) function from R.Is there …

Python ABCs: registering vs. subclassing

(I am using python 2.7) The python documentation indicates that you can pass a mapping to the dict builtin and it will copy that mapping into the new dict:http://docs.python.org/library/stdtypes.html#…