Additional parameters in CSV module in Python

Topics Covered

1. quotechar
2. quoting
3. escapechar
4. skipinitialspace

Before you begin, make ure to read the first article on doing data analysis with CSV files in Python.

quotechar

It refers to the character string that is used to quote values when special characters or delimiters appears inside the field. It defaults to ".

For example, if the delimiter of your CSV file is a comma and you have an address column that may have comma in it's values. Check out the example below.

Example

Name,Age,Height(cm),Weight(kg),Address
Phil,42,178,76,'Gryffindor room, Hogwarts'
Claire,40,165,54,'Snapes room, Hogwarts'
Alex,18,140,46,'4 Private Drive, Little Whinging'

The above CSV file is using single quotes to separate the address fields for each data record. You can pass this as the quotechar value.

import csv

filename = "my_family.csv"

with open(filename, 'r') as output:
  csvreader = csv.reader(output, quotechar="'")
  for row in csvreader:
    print(row)

Output

['Name', 'Age', 'Height(cm)', 'Weight(kg)', 'Address']
['Phil', '42', '178', '76', 'Gryffindor room, Hogwarts']
['Claire', '40', '165', '54', 'Snapes room, Hogwarts']
['Alex', '18', '140', '46', '4 Private Drive, Little Whinging']

quoting

The quoting argument controls when quotes should be generated by the writer or recognized by the reader. It is of 4 types.

1. csv.QUOTE_MINIMAL - It adds quote only when required(default).
2. csv.QUOTE_ALL - It quotes everything regardless of the field type.
3. csv.QUOTE_NONNUMERIC - It quotes everything except integers and floats.
4. csv.QUOTE_NONE - It does not quote anything on output. However, while reading, quotes are included around the field values.

Example

import csv

filename = "my_family.csv"

header = ['Name','Age','Height(cm)','Weight(kg)','Address']

data = [
  ['Phil',42,178,76,'Gryffindor room, Hogwarts'],
  ['Claire',40,165,54,'Snapes room, Hogwarts'],
  ['Alex',18,140,46,'4 Private Drive, Little Whinging']
]

with open(filename, 'w') as output:
  csvwriter = csv.writer(output, quotechar="'", quoting=csv.QUOTE_ALL)
  csvwriter.writerow(header)
  csvwriter.writerows(data)

The above code uses csv.QUOTE_ALL as the quoting argument. This will ensure that every data point has a single quotation wrapped around it while being written to the CSV.

Output

'Name','Age','Height(cm)','Weight(kg)','Address'
'Phil','42','178','76','Gryffindor room, Hogwarts'
'Claire','40','165','54','Snapes room, Hogwarts'
'Alex','18','140','46','4 Private Drive, Little Whinging'

escapechar

Let's say, you don't want any quotation in your CSV file while executing the above code. So you use csv.QUOTE_NONE as the quoting argument.

Example

import csv

filename = "my_family.csv"

header = ['Name','Age','Height(cm)','Weight(kg)','Address']

data = [
  ['Phil',42,178,76,'Gryffindor room, Hogwarts'],
  ['Claire',40,165,54,'Snapes room, Hogwarts'],
  ['Alex',18,140,46,'4 Private Drive, Little Whinging']
]

with open(filename, 'w') as output:
  csvwriter = csv.writer(output, quotechar="'", quoting=csv.QUOTE_NONE)
  csvwriter.writerow(header)
  csvwriter.writerows(data)

The above code will throw you an error.

Output

Traceback (most recent call last):
  File "main.py", line 16, in <module>
    csvwriter.writerows(data)
_csv.Error: need to escape, but no escapechar set

The problem is that the address field contains commas. Since the quoting argument is set to csv.QUOTE_NONE, the csv module doesn't know how to escape the commas properly.

For this purpose, you can use the escapechar argument. It takes a single character string that is used to escape the delimiter when the the quoting is turned off.

The below code escapes the comma using a backslash \.

Example

import csv

filename = "my_family.csv"

header = ['Name','Age','Height(cm)','Weight(kg)','Address']

data = [
  ['Phil',42,178,76,'Gryffindor room, Hogwarts'],
  ['Claire',40,165,54,'Snapes room, Hogwarts'],
  ['Alex',18,140,46,'4 Private Drive, Little Whinging']
]

with open(filename, 'w') as output:
  csvwriter = csv.writer(output, quotechar="'", quoting=csv.QUOTE_NONE, escapechar='\\')
  csvwriter.writerow(header)
  csvwriter.writerows(data)

Output

Name,Age,Height(cm),Weight(kg),Address
Phil,42,178,76,Gryffindor room\, Hogwarts
Claire,40,165,54,Snapes room\, Hogwarts
Alex,18,140,46,4 Private Drive\, Little Whinging

Notice how the commas have been escaped with backslash \ and no error is thrown.

skipinitialspace

It skips the space following the delimiter. If True, the initial white spaces will be removed. It defaults to False.

Name, Age, Height(cm), Weight(kg), Address
Phil, 42, 178, 76, 'Gryffindor room, Hogwarts'
Claire, 40, 165, 54, 'Snapes room, Hogwarts'
Alex, 18, 140, 46, '4 Private Drive, Little Whinging'

The above CSV file has spaces after every delimiter. If you read it without the skipinitialspace argument, there will be white spaces in your data points.

Example

import csv

with open('my_family.csv', 'r') as f:
    csv_reader = csv.reader(f, quotechar="'")

    for line in csv_reader:
        print(line)

Output

['Name', ' Age', ' Height(cm)', ' Weight(kg)', ' Address']
['Phil', ' 42', ' 178', ' 76', " 'Gryffindor room", " Hogwarts'"]
['Claire', ' 40', ' 165', ' 54', " 'Snapes room", " Hogwarts'"]
['Alex', ' 18', ' 140', ' 46', " '4 Private Drive", " Little Whinging'"]

To get rid of the whitespaces, set the skipinitialspace argument to True.

Example

import csv

with open('my_family.csv', 'r') as f:
    csv_reader = csv.reader(f, quotechar="'", skipinitialspace=True)

    for line in csv_reader:
        print(line)

Output

['Name', 'Age', 'Height(cm)', 'Weight(kg)', 'Address']
['Phil', '42', '178', '76', 'Gryffindor room, Hogwarts']
['Claire', '40', '165', '54', 'Snapes room, Hogwarts']
['Alex', '18', '140', '46', '4 Private Drive, Little Whinging']