Pandas error_bad_lines=False and conversion to float

When I set the error_bad_lines=False, then the conversion of some data (most likely in bad lines) to float in csv is not performed.

pandas version '0.15.2' numpy version '1.9.1' Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32

INSTALLED VERSIONS

commit: None python: 2.7.9.final.0 python-bits: 32 OS: Windows OS-release: 7 machine: AMD64 processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: None

pandas: 0.15.2 nose: 1.3.4 Cython: 0.21.2 numpy: 1.9.1 scipy: 0.15.1 statsmodels: 0.6.1 IPython: 2.3.1 sphinx: 1.2.3 patsy: 0.3.0 dateutil: 1.5 pytz: 2014.9 bottleneck: None tables: None numexpr: None matplotlib: 1.4.2 openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None httplib2: None apiclient: None rpy2: None sqlalchemy: None pymysql: None psycopg2: None

Comment From: den-run-ai

df1=pd.read_csv(filename3,sep=",", index_col=0,error_bad_lines=False)

then I have to do manually

df1=df1.replace('True',1)
df1=df1.astype(float)

otherwise pd leaves some of these values as strings

Comment From: den-run-ai

warning received during pd.read_csv(...)

Skipping line 3818: expected 133 fields, saw 289
Skipping line 4079: expected 133 fields, saw 188
Skipping line 4119: expected 133 fields, saw 187

Skipping line 4165: expected 133 fields, saw 188
Skipping line 4208: expected 133 fields, saw 186
Skipping line 4294: expected 133 fields, saw 298
Skipping line 4337: expected 133 fields, saw 243
Skipping line 4423: expected 133 fields, saw 298

C:\Python\Python27\lib\site-packages\pandas\io\parsers.py:1159: DtypeWarning: Columns (89,95,96,97,98,99,100) have mixed types. Specify dtype option on import or set low_memory=False.
  data = self._reader.read(nrows)

Comment From: jorisvandenbossche

@denfromufa Can you provide a small reproducible example?

Comment From: TomAugspurger

Let us know if you can make a reproducible example.