read_sas works just fine with my other tables. I suspect this may have something to do with converting timeseries or timedeltas, given the nature of the error message and the fact that this only happens with a table that contains time series.
> df = pd.read_sas(filename,encoding='utf-8')
File "<ipython-input-119-402090ec994d>", line 1, in <module>
vehicle = pd.read_sas('C:/Users/aliceell/Documents/schemas/new_mexico/dr527_veh_1214.sas7bdat',format='sas7bdat',encoding='ISO-8859-1')
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\io\sas\sasreader.py", line 61, in read_sas
return reader.read()
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\io\sas\sas7bdat.py", line 589, in read
rslt = self._chunk_to_dataframe()
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\io\sas\sas7bdat.py", line 634, in _chunk_to_dataframe
rslt[name] = epoch + pd.to_timedelta(rslt[name], unit='d')
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\util\decorators.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\tseries\timedeltas.py", line 90, in to_timedelta
values = _convert_listlike(arg._values, box=False, unit=unit)
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\tseries\timedeltas.py", line 78, in _convert_listlike
_ensure_object(arg), unit=unit, errors=errors)
File "pandas\tslib.pyx", line 2933, in pandas.tslib.array_to_timedelta64 (pandas\tslib.c:52393)
File "pandas\tslib.pyx", line 3253, in pandas.tslib.convert_to_timedelta64 (pandas\tslib.c:55946)
File "pandas\tslib.pyx", line 3598, in pandas.tslib.cast_from_unit (pandas\tslib.c:61059)
OverflowError: int too big to convert
Output of pd.show_versions()
pd.show_versions()
C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\matplotlib\__init__.py:1401: UserWarning: This call to matplotlib.use() has no effect
because the backend has already been chosen;
matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
warnings.warn(_use_error_msg)
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 37 Stepping 5, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
pandas: 0.18.1
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.11.1
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: 1.1.0
tables: 3.2.2
numexpr: 2.6.1
matplotlib: 2.0.0
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.42.0
pandas_datareader: None
I'm on Windows 7 64-bit.
Google's giving me nothing. It's difficult for me to provide sample data because I don't actually have SAS and so anything I provide would have to be run through pandas or R or something similar to convert it to a CSV, potentially overwriting whatever's causing the bug. But if you think it would be helpful, I can do that.
Comment From: TomAugspurger
Do you have an idea on the span of those dates? Do they fall outside the span pandas can represent with a datetime64?
And can you try on a more recent version of pandas, may have been fixed already.
https://github.com/pandas-dev/pandas/pull/15871/files may be related
Comment From: aliceell
The dates range from 2012-01-01 to 2014-12-31, so I don't think they should fall outside of the span.
In terms of the formatting-- I'm fairly certain it's just yyyy-mm-dd, but I'm reading it into R so I'm not sure if that's actually the case in the raw sas7bdat file. I'm going to talk to a colleague who has SAS and see if they can find any weird dates.
I updated pandas but I'm still getting the same result. Here's the new show_versions results just in case I'm missing something:
Output of pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 37 Stepping 5, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: None.None
pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.11.3
scipy: 0.18.1
statsmodels: 0.6.1
xarray: None
IPython: 5.1.0
sphinx: 1.5.1
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.2.0
tables: 3.2.2
numexpr: 2.6.1
matplotlib: 2.0.0
openpyxl: 2.4.1
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.2
bs4: 4.5.3
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.1.5
pymysql: None
psycopg2: None
jinja2: 2.9.4
boto: 2.45.0
pandas_datareader: None
Comment From: TomAugspurger
cc @Winand if you have any thoughts.
Comment From: Winand
I think you have some dates which are not supported (<1677-09-22 or >2262-04-11). It would be easier if you provide sas7bdat sample data.
(...or there's some bug in pandas' implementation of sas reader.)
Comment From: aliceell
Unfortunately I can't provide the raw file, as it's confidential.
However, here are all of the unique dates from the sas7bdat file.
2012-08-30
2012-06-10
2012-06-15
2012-07-20
2012-07-27
2012-10-24
2012-10-27
2012-11-15
2012-12-07
2012-12-08
2012-12-09
2012-12-20
2012-07-13
2012-07-17
2012-07-21
2012-07-22
2012-08-18
2012-08-22
2012-08-23
2012-08-31
2012-09-01
2012-09-06
2012-09-09
2012-09-28
2012-10-25
2012-11-02
2012-11-09
2012-11-10
2012-11-23
2012-11-28
2012-12-05
2012-12-13
2012-09-14
2012-04-16
2012-05-17
2012-05-23
2012-07-07
2012-07-08
2012-08-11
2012-07-23
2012-08-03
2012-08-16
2012-08-17
2012-10-04
2012-10-06
2012-10-26
2012-11-05
2012-11-18
2012-12-01
2012-12-19
2012-12-21
2012-12-24
2012-03-25
2012-03-27
2012-04-02
2012-05-15
2012-05-20
2012-05-26
2012-06-12
2012-06-13
2012-06-20
2012-06-23
2012-08-05
2012-08-06
2012-08-13
2012-08-29
2012-09-10
2012-09-15
2012-09-16
2012-09-21
2012-05-25
2012-08-08
2012-10-29
2012-04-10
2012-07-24
2012-08-28
2012-12-30
2012-03-03
2012-09-13
2012-10-15
2012-11-11
2012-12-12
2012-03-28
2012-12-22
2012-05-18
2012-03-10
2012-02-17
2012-04-04
2012-05-02
2012-11-13
2012-11-26
2012-09-18
2012-06-08
2012-12-31
2012-04-12
2012-12-14
2012-07-16
2012-10-05
2012-05-05
2012-08-01
2012-06-17
2012-05-10
2012-04-21
2012-01-22
2012-06-25
2012-04-05
2012-05-04
2012-04-20
2012-05-27
2012-06-02
2012-12-23
2012-12-26
2012-07-25
2012-08-09
2012-05-12
2012-08-12
2012-09-03
2012-12-15
2012-06-29
2012-07-18
2012-07-26
2012-10-28
2012-11-24
2012-12-02
2012-12-03
2012-12-16
2012-04-15
2012-09-29
2012-09-30
2012-08-21
2012-09-12
2012-10-11
2012-10-19
2012-11-21
2012-04-03
2012-12-04
2012-03-16
2012-01-17
2012-03-20
2012-03-21
2012-04-08
2012-05-14
2012-04-27
2012-04-29
2012-04-30
2012-06-09
2012-07-02
2012-06-03
2012-08-14
2012-03-08
2012-03-07
2012-03-19
2012-09-04
2012-09-19
2012-03-18
2012-03-30
2012-05-09
2012-07-09
2012-07-15
2012-07-11
2012-07-29
2012-10-08
2012-10-12
2012-10-13
2012-10-07
2012-05-11
2012-02-03
2012-08-20
2012-08-27
2012-10-31
2012-11-01
2012-11-06
2012-12-06
2012-04-13
2012-05-22
2012-05-31
2012-06-01
2012-01-10
2012-02-15
2012-03-02
2012-03-14
2012-03-22
2012-04-09
2012-04-11
2012-04-23
2012-04-24
2012-05-03
2012-05-29
2012-05-30
2012-06-05
2012-06-19
2012-06-26
2012-07-05
2012-07-06
2012-07-12
2012-07-28
2012-07-30
2012-06-21
2012-07-31
2012-10-02
2012-10-03
2012-08-02
2012-12-18
2012-06-27
2012-08-25
2012-10-23
2012-04-07
2012-11-08
2012-06-11
2012-03-09
2012-06-22
2012-06-30
2012-09-08
2012-11-14
2012-04-18
2012-06-06
2012-11-30
2012-02-01
2012-03-13
2012-03-29
2012-04-06
2012-04-17
2012-04-25
2012-05-07
2012-04-14
2012-05-01
2012-08-15
2012-01-13
2012-03-26
2012-05-21
2012-06-18
2012-07-10
2012-05-08
2012-05-16
2012-10-09
2012-12-11
2012-12-17
2012-12-27
2012-09-07
2012-12-29
2012-09-17
2012-04-26
2012-05-28
2012-07-19
2012-05-13
2012-10-01
2012-08-07
2012-10-21
2012-11-19
2012-10-18
2012-05-19
2012-08-04
2012-09-22
2012-10-10
2012-10-16
2012-10-17
2012-11-07
2012-11-04
2012-11-29
2012-12-10
2012-12-28
2012-06-04
2012-07-14
2012-09-25
2012-02-19
2012-02-21
2012-08-26
2012-10-22
2012-02-29
2012-11-03
2012-02-10
2012-02-25
2012-03-01
2012-02-08
2012-09-24
2012-10-20
2012-08-24
2012-09-20
2012-04-19
2012-06-14
2012-02-05
2012-04-22
2012-09-05
2012-04-28
2012-07-01
2012-08-19
2012-09-11
2012-11-27
2012-03-23
2012-03-24
2012-05-06
2012-06-24
2012-08-10
2012-10-30
2012-11-20
2012-09-27
2012-11-12
2012-07-04
2012-06-16
2012-03-31
2012-02-02
2012-09-23
2012-11-22
2012-10-14
2012-01-12
2012-06-28
2012-01-23
2012-03-12
2012-09-26
2012-11-25
2012-02-11
2012-01-21
2012-03-11
2012-01-20
2012-01-27
2012-11-17
2012-02-04
2012-12-25
2012-07-03
2012-06-07
2012-01-28
2012-05-24
2012-11-16
2012-04-01
2012-02-23
2012-02-27
2012-03-17
2012-01-31
2012-02-09
2012-03-05
2012-02-12
2012-01-05
2012-01-26
2012-03-15
2012-02-06
2012-02-18
2012-02-20
2012-02-28
2012-03-04
2012-02-14
2012-01-25
2012-01-09
2012-01-30
2012-02-24
2012-03-06
2012-01-14
2012-01-18
2012-01-01
2012-01-02
2012-09-02
2012-01-29
2012-02-13
2012-01-19
2012-01-11
2012-01-24
2012-02-07
2012-02-16
2012-02-22
2012-01-03
2012-01-06
2012-01-07
2012-01-15
2012-01-16
2012-01-08
2012-01-04
2012-02-26
2012-07-19
2012-08-27
2013-07-16
2013-01-14
2013-02-05
2013-05-02
2013-06-10
2013-07-05
2013-12-02
2013-02-03
2013-02-06
2013-11-15
2013-11-22
2013-09-27
2013-02-25
2013-03-23
2013-09-14
2013-11-06
2013-02-26
2013-02-14
2013-10-21
2013-04-02
2013-02-22
2013-04-28
2013-02-15
2013-02-16
2013-03-07
2013-02-07
2013-01-01
2013-01-03
2013-01-04
2013-01-02
2013-01-05
2013-01-06
2013-01-07
2013-01-08
2013-01-09
2013-01-12
2013-01-10
2013-01-13
2013-01-11
2013-01-15
2013-01-17
2013-01-18
2013-01-21
2013-01-20
2013-01-19
2013-01-22
2013-01-23
2013-01-24
2013-01-26
2013-01-27
2013-01-25
2013-01-28
2013-01-30
2013-01-29
2013-01-31
2013-02-02
2013-02-01
2013-02-04
2013-03-26
2013-02-08
2013-11-20
2013-11-05
2013-12-03
2013-10-08
2013-02-09
2013-03-08
2013-02-11
2013-02-10
2013-02-12
2013-02-13
2013-02-17
2013-02-19
2013-02-20
2013-02-21
2013-02-23
2013-02-24
2013-02-27
2013-02-28
2013-03-01
2013-03-02
2013-03-03
2013-02-18
2013-03-04
2013-03-06
2013-03-05
2013-03-09
2013-03-10
2013-03-11
2013-03-12
2013-03-13
2013-03-14
2013-03-17
2013-03-15
2013-03-16
2013-03-18
2013-03-19
2013-03-20
2013-03-22
2013-03-24
2013-03-21
2013-03-25
2013-03-27
2013-03-29
2013-03-30
2013-03-31
2013-03-28
2013-04-01
2013-04-03
2013-04-04
2013-04-07
2013-04-06
2013-04-05
2013-04-08
2013-04-09
2013-04-10
2013-04-11
2013-04-13
2013-04-12
2013-04-14
2013-04-15
2013-04-16
2013-04-17
2013-04-18
2013-04-20
2013-04-21
2013-04-22
2013-04-23
2013-04-24
2013-04-25
2013-04-26
2013-04-27
2013-04-19
2013-04-29
2013-04-30
2013-05-01
2013-05-03
2013-05-05
2013-05-04
2013-05-06
2013-05-07
2013-05-08
2013-05-09
2013-05-10
2013-05-11
2013-05-13
2013-05-14
2013-05-15
2013-05-16
2013-05-17
2013-05-18
2013-05-23
2013-05-22
2013-05-27
2013-05-28
2013-06-02
2013-06-03
2013-06-01
2013-06-07
2013-06-08
2013-06-04
2013-06-05
2013-06-09
2013-06-06
2013-06-11
2013-06-12
2013-06-13
2013-06-14
2013-06-16
2013-06-17
2013-06-19
2013-06-18
2013-06-15
2013-06-20
2013-06-21
2013-05-20
2013-06-23
2013-06-22
2013-06-24
2013-06-25
2013-06-26
2013-06-29
2013-06-28
2013-06-30
2013-07-01
2013-07-03
2013-07-09
2013-07-11
2013-07-15
2013-07-19
2013-07-20
2013-07-08
2013-07-22
2013-07-25
2013-07-26
2013-07-27
2013-07-28
2013-07-29
2013-07-21
2013-07-30
2013-07-31
2013-08-01
2013-08-02
2013-05-24
2013-08-03
2013-08-05
2013-08-04
2013-08-06
2013-08-07
2013-08-09
2013-08-08
2013-08-11
2013-08-10
2013-08-12
2013-07-10
2013-08-13
2013-07-12
2013-08-15
2013-08-14
2013-08-16
2013-08-17
2013-08-18
2013-08-19
2013-08-20
2013-08-21
2013-08-22
2013-07-24
2013-08-23
2013-08-24
2013-08-25
2013-08-26
2013-08-27
2013-11-04
2013-08-29
2013-08-28
2013-08-30
2013-09-01
2013-08-31
2013-09-02
2013-07-17
2013-09-03
2013-09-04
2013-09-06
2013-09-05
2013-09-07
2013-09-08
2013-09-09
2013-09-10
2013-09-11
2013-09-12
2013-09-15
2013-09-13
2013-09-16
2013-09-18
2013-09-17
2013-09-19
2013-09-20
2013-09-22
2013-09-23
2013-09-25
2013-09-24
2013-09-21
2013-09-26
2013-09-29
2013-09-28
2013-09-30
2013-10-01
2013-10-02
2013-10-03
2013-10-04
2013-10-05
2013-10-06
2013-10-07
2013-10-09
2013-10-10
2013-10-11
2013-10-12
2013-10-13
2013-10-14
2013-10-15
2013-10-16
2013-10-17
2013-10-18
2013-10-19
2013-10-20
2013-10-22
2013-10-23
2013-10-24
2013-10-25
2013-10-26
2013-10-28
2013-10-27
2013-10-29
2013-10-31
2013-11-03
2013-11-01
2013-11-02
2013-11-08
2013-11-07
2013-11-10
2013-11-11
2013-11-09
2013-11-13
2013-10-30
2013-11-14
2013-11-16
2013-11-17
2013-11-12
2013-11-18
2013-11-19
2013-11-21
2013-11-24
2013-11-23
2013-11-25
2013-11-26
2013-11-27
2013-11-28
2013-12-01
2013-11-29
2013-12-04
2013-12-05
2013-12-08
2013-12-07
2013-12-06
2013-12-09
2013-12-10
2013-12-11
2013-12-17
2013-12-18
2013-12-19
2013-12-16
2013-12-22
2013-12-14
2013-12-24
2013-12-26
2013-12-20
2013-12-21
2013-12-23
2013-12-25
2013-12-27
2013-12-29
2013-12-12
2013-12-31
2013-12-28
2013-12-13
2013-12-30
2013-05-12
2013-05-29
2013-05-30
2013-06-27
2013-07-02
2013-07-04
2013-05-26
2013-07-13
2013-07-18
2013-07-07
2013-11-30
2013-07-14
2013-01-16
2013-05-19
2013-05-21
2013-05-25
2013-05-31
2013-07-06
2013-07-23
2013-12-15
2014-07-18
2014-11-03
2014-11-04
2014-11-07
2014-11-12
2014-11-10
2014-12-01
2014-12-19
2014-11-06
2014-05-22
2014-01-16
2014-08-07
2014-06-22
2014-07-13
2014-01-04
2014-01-03
2014-01-01
2014-01-02
2014-01-05
2014-01-07
2014-01-08
2014-01-09
2014-01-06
2014-01-12
2014-01-10
2014-01-13
2014-01-14
2014-01-11
2014-01-15
2014-01-18
2014-01-17
2014-01-19
2014-01-20
2014-01-21
2014-01-23
2014-01-25
2014-01-24
2014-01-26
2014-01-22
2014-01-27
2014-01-28
2014-01-29
2014-01-30
2014-01-31
2014-02-02
2014-02-01
2014-02-03
2014-02-04
2014-02-06
2014-02-05
2014-02-07
2014-02-08
2014-02-10
2014-02-11
2014-02-12
2014-02-13
2014-02-15
2014-02-14
2014-02-16
2014-02-17
2014-02-09
2014-02-18
2014-02-19
2014-02-20
2014-02-22
2014-02-23
2014-02-24
2014-02-25
2014-02-26
2014-02-27
2014-02-28
2014-03-01
2014-02-21
2014-03-02
2014-03-03
2014-03-04
2014-03-05
2014-03-06
2014-03-07
2014-03-08
2014-03-09
2014-03-10
2014-03-11
2014-03-12
2014-03-13
2014-03-15
2014-03-17
2014-03-18
2014-03-14
2014-03-19
2014-03-20
2014-03-21
2014-03-22
2014-03-23
2014-03-25
2014-03-26
2014-03-24
2014-03-27
2014-03-28
2014-03-29
2014-03-30
2014-03-31
2014-04-02
2014-04-01
2014-04-03
2014-04-05
2014-04-04
2014-04-06
2014-04-08
2014-04-09
2014-04-07
2014-04-10
2014-04-12
2014-04-11
2014-04-27
2014-04-15
2014-03-16
2014-04-18
2014-04-17
2014-04-16
2014-04-22
2014-04-20
2014-04-23
2014-04-24
2014-04-21
2014-04-26
2014-04-25
2014-04-28
2014-04-29
2014-04-30
2014-05-01
2014-04-13
2014-05-04
2014-05-02
2014-05-03
2014-05-05
2014-11-20
2014-05-06
2014-05-07
2014-05-08
2014-05-09
2014-05-10
2014-05-12
2014-05-11
2014-05-13
2014-05-14
2014-04-14
2014-05-15
2014-05-18
2014-07-16
2014-05-19
2014-05-16
2014-05-17
2014-05-20
2014-05-21
2014-05-24
2014-05-26
2014-05-23
2014-05-25
2014-05-27
2014-05-28
2014-05-29
2014-05-30
2014-05-31
2014-06-01
2014-06-02
2014-06-03
2014-06-04
2014-06-05
2014-06-06
2014-06-07
2014-06-09
2014-06-10
2014-06-11
2014-06-12
2014-06-08
2014-06-13
2014-06-14
2014-06-15
2014-06-16
2014-06-17
2014-06-18
2014-06-19
2014-06-20
2014-06-21
2014-06-23
2014-08-22
2014-06-24
2014-06-25
2014-06-26
2014-06-27
2014-06-28
2014-06-29
2014-06-30
2014-07-01
2014-07-02
2014-07-03
2014-07-04
2014-07-06
2014-07-05
2014-07-07
2014-07-08
2014-07-10
2014-07-11
2014-07-12
2014-07-09
2014-07-14
2014-07-15
2014-07-17
2014-07-21
2014-07-19
2014-07-20
2014-07-22
2014-07-23
2014-07-24
2014-07-25
2014-07-26
2014-07-27
2014-07-28
2014-07-29
2014-07-31
2014-07-30
2014-08-01
2014-08-02
2014-08-04
2014-08-03
2014-08-05
2014-08-06
2014-08-11
2014-08-09
2014-08-10
2014-08-08
2014-08-12
2014-08-13
2014-12-20
2014-08-14
2014-08-17
2014-08-16
2014-08-15
2014-08-18
2014-08-19
2014-08-20
2014-08-21
2014-08-24
2014-08-23
2014-08-25
2014-08-26
2014-08-27
2014-08-28
2014-08-29
2014-08-30
2014-09-01
2014-08-31
2014-09-02
2014-09-03
2014-09-04
2014-09-05
2014-09-06
2014-09-07
2014-09-08
2014-09-09
2014-09-10
2014-09-11
2014-09-14
2014-09-12
2014-09-15
2014-09-13
2014-09-16
2014-09-17
2014-09-18
2014-09-21
2014-09-22
2014-09-20
2014-09-23
2014-09-19
2014-09-24
2014-09-25
2014-09-26
2014-09-27
2014-09-28
2014-09-29
2014-09-30
2014-10-03
2014-10-04
2014-10-01
2014-10-02
2014-10-05
2014-10-06
2014-10-07
2014-10-08
2014-10-09
2014-10-12
2014-10-10
2014-10-11
2014-10-13
2014-10-21
2014-10-14
2014-10-15
2014-10-16
2014-10-18
2014-10-17
2014-10-19
2014-10-20
2014-10-22
2014-10-23
2014-10-26
2014-10-27
2014-10-25
2014-10-24
2014-10-28
2014-10-29
2014-10-30
2014-11-02
2014-11-01
2014-10-31
2014-11-05
2014-11-08
2014-11-19
2014-11-09
2014-11-11
2014-11-13
2014-11-15
2014-11-14
2014-11-16
2014-11-17
2014-11-18
2014-11-21
2014-11-23
2014-11-24
2014-11-22
2014-11-25
2014-11-29
2014-11-28
2014-11-30
2014-11-26
2014-11-27
2014-12-02
2014-12-03
2014-12-04
2014-12-07
2014-12-06
2014-12-05
2014-12-08
2014-12-18
2014-12-09
2014-12-10
2014-12-12
2014-12-11
2014-12-13
2014-12-14
2014-12-22
2014-12-15
2014-12-16
2014-12-17
2014-12-21
2014-12-23
2014-12-24
2014-12-25
2014-12-26
2014-12-28
2014-12-27
2014-12-29
2014-12-30
2014-12-31
2014-04-19
As stated above, the minimum date is 2012-01-01, and the maximum date is 2014-12-31.
Comment From: TomAugspurger
Do you have a column of timedeltas, as that seems to be what's causing in the error? If you get a chance, you can enter pdb after it raises the exception to see what column / chunk of data it's failing on.
Comment From: aliceell
I've actually never used the debugger before, sorry...is there a specific command that I should use in order to show the column that it's failing on? I tried p locals() but that dumped a lot of code and functions that don't look particularly easy to parse:
{'datetime_to_datetime64': <built-in function datetime_to_datetime64>, 'monthrange': <built-in function monthrange>, '_make_error_func': <built-in function _make_error_func>, 'compat_NaT': None, 'calendar': <module 'calendar' from 'C:\\Users\\aliceell\\AppData\\Local\\Continuum\\Anaconda3\\lib\\calendar.py'>, 'callable': <built-in function callable>, '_unbox_utcoffsets': <built-in function _unbox_utcoffsets>, '_dateutil_tzstr': <class 'dateutil.tz.tz.tzstr'>, 'get_date_field': <built-in function get_date_field>, '_Timedelta': <class 'pandas.tslib._Timedelta'>...(plus another 50 lines)
I don't think they are timedeltas, seeing as how they are formatted. But I'm not sure how to tell without reading in the whole file, which I can't do right now because it breaks...
For what it's worth, the results of dtypes when used with my other sas7bdat table (that includes the same dates and reads fine) shows that the date column is datetime64[ns].
Comment From: TomAugspurger
Commandas are here but basically u a few times to go up frames to the interesting ones, I think around
File "C:\Users\aliceell\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\io\sas\sas7bdat.py", line 634, in _chunk_to_dataframe
should be informative. Press l to show the lines of code around there, and p <var> to print variables. Like p name to see the column, and p rslt to see the dataframe. Make sure not to paste anything here that's sensitive ;)
Comment From: aliceell
Thanks! I had looked at the docs, but didn't even know where to start...
Here's the problem. I found the column where it breaks, it's not the column I was thinking of (unsurprisingly). Apparently the data contains more than one datetime column, and it has several invalid dates like 2316-01-05, 3015-12-10, and 9988-09-07. So I guess I'll just read it into R for now...
Thanks for your help! I would not have been able to find that problematic column without your help on the debugger.