Python Pandas Series failure datetime

2024/9/30 7:30:26

I think that this has to be a failure of pandas, having a pandas Series (v.18.1 and 19 too), if I assign a date to the Series, the first time it is added as int (error), the second time it is added as datetime(correct), I can not understand the reason.

For instance with this code:

import datetime as dt
import pandas as pd
series = pd.Series(list('abc'))
date = dt.datetime(2016, 10, 30, 0, 0)
series["Date_column"] =date
print("The date is {} and the type is {}".format(series["Date_column"], type(series["Date_column"])))
series["Date_column"] =date
print("The date is {} and the type is {}".format(series["Date_column"], type(series["Date_column"])))

The output is:

The date is 1477785600000000000 and the type is <class 'int'>
The date is 2016-10-30 00:00:00 and the type is <class 'datetime.datetime'>

As you can see, the first time it always sets the value as int instead of datetime.

could someone help me?, Thank you very much in advance, Javi.

Answer

The reason for this is that series is an 'object' type and the columns of a pandas DataFrame (or a Series) are homogeneously of type. You can inspect this with dtype (or DataFrame.dtypes):

series = pd.Series(list('abc'))
series
Out[3]:
0    a
1    b
2    c
dtype: objectIn [15]: date = dt.datetime(2016, 10, 30, 0, 0)
date
Out[15]: datetime.datetime(2016, 10, 30, 0, 0)In [18]: print(date)
2016-10-30 00:00:00In [17]: type(date)
Out[17]: datetime.datetimeIn [19]: series["Date_column"] = date
In [20]: seriesOut[20]:
0                                a
1                                b
2                                c
Date_column    1477785600000000000
dtype: objectIn [22]: series.dtypeOut[22]: dtype('O')

Only the generic 'object' dtype can hold any python object (in your case inserting a datetime.datetime object into the Series).

Moreover, Pandas Series are based on Numpy Arrays, which are not mixed types and defeats the purpose of using the computational benefit of Pandas DataFrames and Series or Numpy.

Could you use a python list() instead? or a DataFrame()?

https://en.xdnf.cn/q/71112.html

Related Q&A

Remove a dictionary key that has a certain value [duplicate]

This question already has answers here:Removing entries from a dictionary based on values(4 answers)Closed 10 years ago.I know dictionarys are not meant to be used this way, so there is no built in fun…

Get names of positional arguments from functions signature

Using Python 3.x, Im trying to get the name of all positional arguments from some function i.e: def foo(a, b, c=1):returnRight now Im doing this: from inspect import signature, _empty args =[x for x, p…

Emacs Python-mode syntax highlighting

I have GNU Emacs 23 (package emacs23) installed on an Ubuntu 10.04 desktop machine and package emacs23-nox installed on an Ubuntu 10.04 headless server (no X installed). Both installations have the sam…

programmatically edit tab order in pyqt4 python

I have a multiple textfield in my form. My problem is the tab order is wrong. Is there a way to edit tab order in code? Just like in QT Designer.thanks.

Getting the parent of a parent widget in Python Tkinter

I am trying to get the parent of a widget then get the parent of that widget. But Everytime I try to I get a error.Error:AttributeError: str object has no attribute _nametowidgetWhy is it giving me tha…

Python Random List Comprehension

I have a list similar to:[1 2 1 4 5 2 3 2 4 5 3 1 4 2] I want to create a list of x random elements from this list where none of the chosen elements are the same. The difficult part is that I would lik…

How to handle unseen categorical values in test data set using python?

Suppose I have location feature. In train data set its unique values are NewYork, Chicago. But in test set it has NewYork, Chicago, London. So while creating one hot encoding how to ignore London? In…

How to get Facebook access token using Python library?

Ive found this Library it seems it is the official one, then found this, but everytime i find an answer the half of it are links to Facebook API Documentation which talks about Javascript or PHP and ho…

How to convert shapefile/geojson to hexagons using uber h3 in python?

I want to create hexagons on my geographic map and want to preserve the digital boundary specified by the shapefile/geojson as well. How do I do it using ubers h3 python library? Im new to shapefiles…

Python mypy: float and int are incompatible types with numbers.Real

I am new to Pythons static typing module mypy. I am trying to append ints and floats to an array, which I typed statically to be Real. But mypy says that they are incompatible types with Real. I though…