Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
df = pd.DataFrame({"x": [pd.NA, pd.NA], "y": [1.0, 2.0]})
df2 = pd.read_json(df.to_json(orient="table"), orient="table")
print(df.to_json(orient="table"))
print(df.dtypes)
print(df2.dtypes)
Issue Description
When I have a column consisting of pd.NA
the data type is initially object
. When I call .to_json(orient="table")
the schema says that the type is "string" (which is probably OK??)
{"schema":{"fields":[{"name":"index","type":"integer"},{"name":"x","type":"string"},{"name":"y","type":"number"}],"primaryKey":["index"],"pandas_version":"1.4.0"},"data":[{"index":0,"x":null,"y":1.0},{"index":1,"x":null,"y":2.0}]}
When I read the produced json string back using .read_json
the data dype in the resulting frame is float64
Full output of the example
{"schema":{"fields":[{"name":"index","type":"integer"},{"name":"x","type":"string"},{"name":"y","type":"number"}],"primaryKey":["index"],"pandas_version":"1.4.0"},"data":[{"index":0,"x":null,"y":1.0},{"index":1,"x":null,"y":2.0}]}
x object
y float64
dtype: object
x float64
y float64
dtype: object
Expected Behavior
The data types should be preserved when doing a roundtrip with orient="table" on both read and write.
Minor detail: The schema appears to say "pandas_version": 1.4.0
even though the details below says that the install version is 1.5.2
. Would expect that 1.5.2
was present in the output.
Installed Versions
Comment From: phofl
This looks suspicious, but not sure if this is intentional