I have a csv file with very old dates, and pd.to_datetime fails. It works in polars. Is this an inherent limitation in pandas, a bug or something else?
import pandas as pd
dates = ["12/31/1672","12/31/1677","10/19/2024"]
df = pd.DataFrame(dates, columns=['Date'])
df['Date'] = pd.to_datetime(df['Date'], format="%m/%d/%Y", errors="coerce")
df
Date
0 NaT
1 1677-12-31
2 2024-10-19
in polars
import polars as pl
df = pl.DataFrame({ 'Date': dates})
df = df.with_columns(pl.col('Date').str.strptime(pl.Date, format="%m/%d/%Y"))
df
shape: (3, 1)
┌────────────┐
│ Date │
│ --- │
│ date │
╞════════════╡
│ 1672-12-31 │
│ 1677-12-31 │
│ 2024-10-19 │
└────────────┘
You need to sign in to view this answers
Leave feedback about this