Code Sample
import csv
import pandas
csv.register_dialect('mydialect', delimiter='|', escapechar='\\', doublequote=False, quoting=csv.QUOTE_NONE)
pandas.read_csv('example.csv', dialect='mydialect')
Problem description
I believe that this warning was added to handle conflicts between any dialect-provided csv parsing params and the equivalent params passed individually to read_csv()
. However, I don't think it is working as intended. Notice, in my example above I only specify these values by way of the dialect, yet I nonetheless receive these warnings:
ParserWarning: Conflicting values for 'doublequote': 'True' was provided, but the dialect specifies '0'. Using the dialect-specified value.
ParserWarning: Conflicting values for 'escapechar': 'None' was provided, but the dialect specifies '\'. Using the dialect-specified value.
ParserWarning: Conflicting values for 'quoting': '0' was provided, but the dialect specifies '3'. Using the dialect-specified value.
ParserWarning: Conflicting values for 'delimiter': ',' was provided, but the dialect specifies '|'. Using the dialect-specified value.
The reason, I believe, is that the check that tests for conflicts in these values is effectively comparing against the default values when no value is specified as a kwarg. This means anytime I am specifying a value via the dialect which is different from the default this warning is raised.
See:
https://github.com/pandas-dev/pandas/blob/199e52518bd4b5dfc7f8b0652b53b1679c80364a/pandas/io/parsers.py#L744
and
https://github.com/pandas-dev/pandas/blob/199e52518bd4b5dfc7f8b0652b53b1679c80364a/pandas/io/parsers.py#L750
Expected Output
No warnings unless there is a conflict between a dialect value and that of a param explicitly passed to read_csv.
Comment From: kylekeppler
Seconded. Seeing the same thing when I create a custom dialect.
Comment From: phofl
This works and has tests