Unit 05.03: as.numeric() never fails, and that is the problem

as.numeric() never fails. That is precisely the problem.

Convert deliberately, and keep what you could not convert

When as.numeric() meets something it cannot parse it returns NA and a warning, and the warning is easy to suppress or miss. The values are gone, and nothing downstream knows they were ever there.

A deliberate conversion has three parts. Strip what you understand — thousands separators, currency symbols, stray whitespace. Convert. Then keep a flag for what still failed, so the unconvertible values remain visible and countable.

The distinction matters because the two kinds of failure need different responses. A formatting artefact is yours to fix. A value like 'pending' is information about the process, and deserves to reach the report.

This block converts naively, then deliberately, and compares.

suppressPackageStartupMessages(library(dplyr))

raw <- data.frame(
  ward   = c("North", "South", "East", "West"),
  visits = c("412", "1,455", "502", "pending"),
  stringsAsFactors = FALSE
)

# The naive conversion loses two values and says nothing about it.
naive <- suppressWarnings(as.numeric(raw$visits))
cat("Naive as.numeric():", paste(ifelse(is.na(naive), "NA", naive), collapse = " "), "\n")
cat("Values lost:", sum(is.na(naive)), "\n\n")

# Convert deliberately: strip what you understand, keep what you do not.
cleaned <- raw |>
  mutate(
    stripped = gsub(",", "", visits),
    numeric  = suppressWarnings(as.numeric(stripped)),
    problem  = is.na(numeric)
  )
print(cleaned)

cat("\nRecovered by removing the thousands separator: 1455\n")
cat("Still unconvertible and now visible:",
    paste(cleaned$visits[cleaned$problem], collapse = ", "), "\n")

The naive call loses 2 values: 1,455 fails on its comma and pending fails because it is not a number at all. The deliberate version strips the separator and recovers 1455, leaving exactly one genuine problem — pending — flagged in its own column. Two failures became one, and the remaining one is now visible rather than silently NA.

The mistake this prevents

The mistake is wrapping the conversion in suppressWarnings() to quieten the console. The warning was the only notification that data was being discarded.

Takeaway

Strip known formatting first, convert, then flag what remains unconvertible. Count the flagged rows and look at their original values before deciding anything.

Mark complete