Read csv into database SQLite3 ODO Python

2024/10/5 22:22:11

I am trying to read in a csv into a new table in a new databased using ODO, SQLite3 and Python.

I am following these guides:

https://media.readthedocs.org/pdf/odo/latest/odo.pdf http://odo.pydata.org/en/latest/perf.html?highlight=sqlite#csv-sqlite3-57m-31s

I am trying the following:

import sqlite3
import csv
from odo import odofile_path = 'my_path/'
# In this case 'my_path/' is a substitute for my real pathdb_name = 'data.sqlite'conn = sqlite3.connect(file_path + db_name)

This creates a new sqlite file data.sqlite within file_path. I can see it there in the folder.

When I then try to read my csv into this database I get the following error:

csv_path = 'my_path/data.csv'
odo(csv_path, file_path + db_name)
conn.close()NotImplementedError: Unable to parse uri to data resource: # lists my path

Can you help?

Answer

No thanks to the ODO documentation, this succesfully created a new table in a new database and read in the csv file to that database:

import sqlite3
import csv
from odo import odo# [1]#  Specify file path
file_path = 'my_path/'
# In this case 'my_path/' is a substitute for my real path# Specify csv file path and name
csv_path = file_path + 'data.csv'# Specify database name
db_name = 'data.sqlite'# Connect to new database
conn = sqlite3.connect(file_path + db_name)# [2]# Use Odo to detect the shape and datatype of your csv:
data_shape = discover(resource(csv_path))# Ready in csv to new table called 'data' within database 'data.sqlite'
odo(pd.read_csv(csv_path), 'sqlite:///' + file_path + 'data.sqlite::data', dshape=data_shape)# Close database
conn.close()

Sources used in [1]:

https://docs.python.org/2/library/sqlite3.html python odo sql AssertionError: datashape must be Record type, got 0 * {...}

Sources used in [2]:

https://stackoverflow.com/a/41584832/2254228 http://sebastianraschka.com/Articles/2014_sqlite_in_python_tutorial.html#creating-a-new-sqlite-database https://stackoverflow.com/a/33316230/2254228 what is difference between .sqlite and .db file?

The ODO documentation is here (good luck...) https://media.readthedocs.org/pdf/odo/latest/odo.pdf

https://en.xdnf.cn/q/119732.html

Related Q&A

netmiko cant execute sh run | i host

I notice that my netmiko code cant run sh run | i host which is a legitimate Cisco command.When I replace sh run with other command such as sh clo, or show ip interface brief, it works perfectly.from n…

How to dump the data from file to an excel sheet

I want to dump [3-4 lines together] some data to an excel sheet. I could able to dump single line based on some criteria [like if line is getting start with // or /* ], but in case of when lines starts…

I dont understand why my script is not iterating through all string.split elements?

The objective of this python exercise is to build a function that turns text into pig latin, a simple text transformation that modifies each word by moving the first character to the end and appending …

Unable to change the tick frequency on my chart

I have seen many questions on changing the tick frequency on SO, and that did help when I am building a line chart, but I have been struggling when its a bar chart. So below are my codes import numpy a…

Django Queryset foreign keys

I am trying to get a queryset but it is not displaying anything. Basically, I want to get the Asset objects that are assigned via foreign key to an employee, which is a foreign key of the signed in use…

How to reorder the columns of a CSV?

How can I re-order the columns of a CSV file using Python? These are the first rows of a CSV file I need to change:03;30269714;Ramiro Alberto;Nederz;active;pgc_gral 03;36185520;Andrea;Espare;active;pg…

Distance matrix in Python Pandas

I am a newbie in python, but I like to process data in pandas. I have a hundred pairs of CSV data such as passenger and bus stop data. The passenger structure data is Person, and XY coordinates (UTM-Me…

calculating catalan numbers using memoization

I am tring to use memoization in order to calculate catalan numbers, but it just does not seem to work, what do I need to change?def catalan_mem(n, memo = None):if n==0:return 1if memo == None:memo = …

cron python file doesnt work on centos 7

im trying to schedule my pythohn script into Centos 7 with cron. On my script at start i have added this:#!/usr/local/bin/pythonand this is my cron file that i have create into folder that contain pyth…

Comma separated Matrix from txt files - continued

I need to form a matrix from a list of textfiles containing frequency distribution of expressions. Therefore, I created a list of all that text files (lof) from a directory and used it to build a matri…