Exercises for Chapter 4

Exercise 1: Problems in CSV files

In the repository for the exercises for this chapter, you find three modifications of the capital distance dataset that we have used above: file1.csv, file2.csv and file3.csv. When you import these files using read.csv(), you will see this generates problems since the files do not conform to a proper CSV format. For each file, find out what the problems are, and how you can fix them. Sometimes you need to manually edit the CSV file, but in some cases it is possible to adjust the read.csv() function such that you can leave the original file unchanged (which is usually preferable).

Solution

File file1.csv: Importing the file with

data1 <- read.csv(file.path("ex04", "file1.csv"))

works without generating any errors. However, you will notice that the import does not work correctly: there are three NA values in some of the variables, and the numa column ranges from 2 to 94, while it should be constant (2). The reason is that for three lines (169, 170 and 188), the values in the last column mindist are wrongly coded with a comma and not a dot as decimal separator. However, the comma is also used as field separator, which leads to an erroneous import. This must be fixed manually in the file.

File file2.csv: Importing the file with

data2 <- read.csv(file.path("ex04", "file2.csv"))

does not seem to create any issues. However, when you take a closer look at the data (for example, using View(data2)), you will see that there are errors in rows 9 and 10 of the data. The reason is that there is one comma too much in line 10 of file2.csv, which causes the cells to shift. Remove this comma, save the file, and import the file again.

File file3.csv: The import with

data3 <- read.csv(file.path("ex04", "file3.csv"))

## Error in `read.table()`:
## ! more columns than column names

fails, since there is an additional comment at the top of the file, which is not part of the table. There is no need to fix this manually. You can simply skip the first line with

data3 <- read.csv(file.path("ex04", "file3.csv"), skip = 1)

and the file is imported correctly.

Exercise 2: File encodings and special characters

In file turkish-cities.csv, you find a list of the ten largest cities in Turkey and their populations from the GeoNames project. The city names are given both in the English transliteration as well as in Turkish, the latter of which include a number of special characters. However, this file uses an unknown encoding, with leads to many of the city names to become garbled up when you import the file using read.csv(). Can you find out what the file coding is? Can you adjust the import function such that the file is read correctly? Hint: the guess_encoding() function can point you in the right direction (the ISO standard), but this standard consists of several parts. You need to find out yourself what the correct part for Turkish is, since the function fails to recognize it properly.

Solution

First, we try to import the file with default settings:

cities <- read.csv(file.path("ex04", "turkish-cities.csv"))
cities[c(11, 12), ]

##                            name population
## 11 Diyarbak\xb9r, Diyarbak\xb9r     644763
## 12             Kayseri, Kayseri     592840

As you can see, some of the city names are not displayed correctly (for example, the city of Diyarbakır). We can try to guess the encoding with

library(readr)
guess_encoding(file.path("ex04", "turkish-cities.csv"))

## # A tibble: 2 × 2
##   encoding   confidence
##   <chr>           <dbl>
## 1 ISO-8859-9       0.38
## 2 ISO-8859-1       0.29

but this gives us two different options, none of which is correct (note also the low confidence scores). ISO-8859 is the correct standard, but neither part 1 (ISO-8859-1) nor part 9 (ISO-8859-9) is used for Turkish. Wikipedia or other sources can help you figure out that part 3 is the correct one. You can then import the file again and manually specify the encoding:

cities <- read.csv(file.path("ex04", "turkish-cities.csv"), fileEncoding = "ISO-8859-3")
cities[c(10, 11), ]

##                      name population
## 10     Bağcılar, Istanbul     724270
## 11 Diyarbakır, Diyarbakır     644763

Exercise 3: Labeling variables

In this exercise, we again use the UN Security Council membership data from Chapter 4 from file unsc-membership.xls. Load the data, label the variables (the labelled package is useful here) and save the file in Stata format. Re-open it and check if the labels are still there (or check with Stata if you have a license).

Solution

We first import the data, as described in the chapter:

library(readxl)
unsc <- read_excel(file.path("ch04", "unsc-membership.xls"),
  sheet = 2,
  na = "."
)

Here’s how to assign variable labels with the labelled package:

library(labelled)
var_label(unsc$aclpcode) <- "Country code (Alvarez, Cheibub, Limongi and Przeworski)"
var_label(unsc$aclpname) <- "Country name"
var_label(unsc$code) <- "ISO 3-letter code"
var_label(unsc$year) <- "Year"
var_label(unsc$unsc) <- "UN Security Council membership"

If we save the file in Stata format, the labels are preserved:

library(haven)
write_dta(unsc, file.path("ex04", "unsc.dta"))

We can check this after importing the file again:

unsc <- read_dta(file.path("ex04", "unsc.dta"))
var_label(unsc$unsc)

## [1] "UN Security Council membership"

Exercise 4: Guessing file types

In the repository for the exercises for this chapter, you will find three files without a proper file ending: unknown-file1, unknown-file2 and unknown-file3. All of them have the same content (a simple table with three columns and six rows). The purpose of this exercise is to find out what the file types are, and correctly import them into R using the appropriate functions introduced in the chapter. Sometimes, it helps to look at the file in RStudio’s text editor, which can give you certain clues regarding the file type. If this fails, you can try the different import functions we have discussed in the chapter. While files should have a proper ending to clearly indicate the file type, is this necessary for R’s import functions to work properly?

Solution

File unknown-file1: This is an Excel file.

library(readxl)
data1 <- read_excel(file.path("ex04", "unknown-file1"))
summary(data1)

##     column1     column2        colum3   
##  Min.   :0   Min.   :0.0   Min.   :0.0  
##  1st Qu.:0   1st Qu.:0.0   1st Qu.:0.0  
##  Median :0   Median :0.5   Median :0.5  
##  Mean   :0   Mean   :0.5   Mean   :0.5  
##  3rd Qu.:0   3rd Qu.:1.0   3rd Qu.:1.0  
##  Max.   :0   Max.   :1.0   Max.   :1.0

File unknown-file2: This is a tab-delimited CSV file. You can use the read.table() function to import it. Unless specified otherwise, this function assumes that the (invisible) tab character is used to separate columns.

data2 <- read.table(file.path("ex04", "unknown-file2"), header = T)
summary(data2)

##     column1     column2        colum3   
##  Min.   :0   Min.   :0.0   Min.   :0.0  
##  1st Qu.:0   1st Qu.:0.0   1st Qu.:0.0  
##  Median :0   Median :0.5   Median :0.5  
##  Mean   :0   Mean   :0.5   Mean   :0.5  
##  3rd Qu.:0   3rd Qu.:1.0   3rd Qu.:1.0  
##  Max.   :0   Max.   :1.0   Max.   :1.0

File unknown-file3: This is an RDS file (a serialized R object), containing an R data frame. It can be imported with

data3 <- readRDS(file.path("ex04", "unknown-file3"))
summary(data3)

##     column1     column2        colum3   
##  Min.   :0   Min.   :0.0   Min.   :0.0  
##  1st Qu.:0   1st Qu.:0.0   1st Qu.:0.0  
##  Median :0   Median :0.5   Median :0.5  
##  Mean   :0   Mean   :0.5   Mean   :0.5  
##  3rd Qu.:0   3rd Qu.:1.0   3rd Qu.:1.0  
##  Max.   :0   Max.   :1.0   Max.   :1.0