You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
i am currently trying to use a schema to verify typing for some data. In my datetime column i have mixed timezones, represented with offsets. When i try to coerce it failes and it seems like it is because the pd.to_datetime called behind panderas would need to be parametrised with tz=UTC.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hey,
i am currently trying to use a schema to verify typing for some data. In my datetime column i have mixed timezones, represented with offsets. When i try to coerce it failes and it seems like it is because the pd.to_datetime called behind panderas would need to be parametrised with tz=UTC.
My example:
CustomSchema(pa.SchemaModel):
shop_date: Series[DateTime] = pa.Field(coerce=True)
Case 1:
data = DataFrameCustomSchema
-> First prints pandas warning: pd.to_datetime() need tz=UTC for mixed timezones, then panderas throws error.
Case 2:
raw_data["shop_date"] = pd.to_datetime(raw_data["shop_date"], errors="coerce", utc=True)
data = DataFrameCustomSchema
-> Works as supposed to.
I tried solutions from related threads, but nothing worked. Either timezone is completly ignored or i get an error. I want so Datatype Doc, that reads like there is the possibility somewhere under the hood, i probably just don't know how to use it.
https://pandera.readthedocs.io/en/v0.20.2/reference/generated/pandera.engines.pandas_engine.DateTime.html
Beta Was this translation helpful? Give feedback.
All reactions