Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions Doc/library/csv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -299,18 +299,29 @@ The :mod:`!csv` module defines the following classes:
.. versionadded:: 3.2


.. class:: Sniffer()
.. class:: Sniffer(delimiters=None)

The :class:`Sniffer` class is used to deduce the format of a CSV file.

If the optional *delimiters* parameter is given,
it is interpreted as a string containing possible valid
delimiter characters,
used by both methods below.
Its characters are also preferred, in the given order,
if several combinations fit the sample equally well.

.. versionchanged:: next
Added the *delimiters* parameter.

The :class:`Sniffer` class provides two methods:

.. method:: sniff(sample, delimiters=None)

Analyze the given *sample* and return a :class:`Dialect` subclass
reflecting the parameters found. If the optional *delimiters* parameter
is given, it is interpreted as a string containing possible valid
delimiter characters.
delimiter characters;
it overrides the *delimiters* argument of the constructor.

The dialect is deduced by parsing the sample with every plausible
combination of parameters
Expand All @@ -327,6 +338,8 @@ The :mod:`!csv` module defines the following classes:
the delimiters ``','``, ``'\t'``, ``';'``, ``' '`` and ``':'``
are preferred, in this order,
no matter how many times each of them occurs.
If the *delimiters* argument was given to the constructor,
its characters are preferred instead, in the given order.

.. versionchanged:: next
The dialect is now deduced by trial parsing
Expand Down
8 changes: 8 additions & 0 deletions Doc/whatsnew/3.16.rst
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,14 @@ csv
The results may differ from those of earlier Python versions.
(Contributed by Serhiy Storchaka in :gh:`83273`.)

* The :class:`csv.Sniffer` constructor now accepts
an optional *delimiters* argument.
It is used by both :meth:`~csv.Sniffer.sniff` and
:meth:`~csv.Sniffer.has_header`,
and its characters are preferred, in the given order,
if several of them fit the sample equally well.
(Contributed by Serhiy Storchaka in :gh:`155059`.)

curses
------

Expand Down
27 changes: 20 additions & 7 deletions Lib/csv.py
Original file line number Diff line number Diff line change
Expand Up @@ -232,23 +232,36 @@ class Sniffer:
'''
"Sniffs" the format of a CSV file (i.e. delimiter, quotechar)
Returns a Dialect object.

If the optional delimiters argument is given, it is interpreted as
a string containing possible valid delimiter characters. It is
used by the sniff() and has_header() methods unless sniff() is
called with its own delimiters argument.

It also sets the preferred attribute: the delimiters which are
preferred, in that order, if several combinations fit the sample
equally well.
'''
# Characters which can be guessed as a delimiter if the delimiters
# argument is not specified.
_delimiter_candidates = [c for c in map(chr, range(128))
if not c.isalnum()]
delimiters = [c for c in map(chr, range(128)) if not c.isalnum()]

def __init__(self):
# in case there is more than one possible delimiter
self.preferred = [',', '\t', ';', ' ', ':']
def __init__(self, delimiters=None):
if delimiters is None:
# in case there is more than one possible delimiter
self.preferred = [',', '\t', ';', ' ', ':']
else:
self.delimiters = delimiters
self.preferred = delimiters


def sniff(self, sample, delimiters=None):
"""
Analyze the sample and return a Dialect subclass reflecting the
parameters found. If the optional delimiters parameter is
given, it is interpreted as a string containing possible valid
delimiter characters. Raises Error if the dialect cannot be
delimiter characters; it overrides the delimiters attribute set
in the constructor. Raises Error if the dialect cannot be
determined.

The dialect is guessed by parsing the sample with every
Expand All @@ -275,7 +288,7 @@ def sniff(self, sample, delimiters=None):

chars = set(sample)
if delimiters is None:
delimiters = self._delimiter_candidates
delimiters = self.delimiters
delimiters = [d for d in delimiters
if d in chars and d not in '\r\n"\'\\']
# Combinations to try, numbered by preference for breaking
Expand Down
25 changes: 25 additions & 0 deletions Lib/test/test_csv.py
Original file line number Diff line number Diff line change
Expand Up @@ -1504,6 +1504,31 @@ def test_delimiters(self):
self.assertEqual(dialect.delimiter, ',')
self.assertEqual(dialect.quotechar, '"')

def test_delimiters_in_constructor(self):
dialect = csv.Sniffer("?,").sniff(self.sample3)
self.assertEqual(dialect.delimiter, "?")
dialect = csv.Sniffer(delimiters="/,").sniff(self.sample3)
self.assertEqual(dialect.delimiter, "/")
# The argument of sniff() takes precedence.
dialect = csv.Sniffer("/,").sniff(self.sample3, "?,")
self.assertEqual(dialect.delimiter, "?")
# has_header() uses them as well.
sniffer = csv.Sniffer(delimiters=",")
self.assertIs(sniffer.has_header(self.header1 + self.sample1), True)
self.assertRaisesRegex(csv.Error, "Could not determine delimiter",
sniffer.has_header, self.sample3)

def test_preferred_in_constructor(self):
sample = 'a,b;c\nd,e;f\ng,h;i\n'
# By default ',' is preferred to ';'.
self.assertEqual(csv.Sniffer().sniff(sample).delimiter, ',')
# The delimiters given to the constructor are preferred in the
# given order.
self.assertEqual(csv.Sniffer(";,").sniff(sample).delimiter, ';')
self.assertEqual(csv.Sniffer(",;").sniff(sample).delimiter, ',')
# The argument of sniff() does not affect the preference.
self.assertEqual(csv.Sniffer(";,").sniff(sample, ",;").delimiter, ';')

def test_sniff_escapechar(self):
# gh-83273: escaped delimiters make the delimiter frequencies
# inconsistent, but the escape character can be detected by trial
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add the optional *delimiters* parameter in the :class:`csv.Sniffer`
constructor. It is used by both :meth:`~csv.Sniffer.sniff` and
:meth:`~csv.Sniffer.has_header`, and its characters are preferred in the
given order for breaking ties.
Loading