download-classifications.RmdThe Italian Ministry of Labor and Social Policies maintains official classification standards used in administrative data systems (URP - Unita Raccolta Permessi). These classifications are published as Excel files at:
https://urponline.lavoro.gov.it/s/standard-tecnici/classificazioni-standard
The download_classification_standards() function
automates:
The function requires two additional packages for web scraping:
install.packages(c("rvest", "polite"))These are listed as “Suggests” dependencies, so they are optional if you do not need this functionality.
library(situas)
# Download latest versions to default directory
files <- download_classification_standards()This will:
data-raw/classifications/
data-raw/classifications/download_metadata.rds
The function recognizes three main classification types:
Main classification standards file (e.g.,
Rev.090-ST-Classificazioni-Standard.xlsx).
Contains multiple sheets with various classification codes:
download_classification_standards(
output_dir = "data-raw/classifications", # Where to save files
url = "https://urponline.lavoro.gov.it/s/standard-tecnici/classificazioni-standard?language=it",
download_all = FALSE, # Download all versions or just latest?
force_refresh = FALSE, # Re-download existing files?
use_js_scraping = FALSE, # Use JavaScript scraping? (not yet implemented)
classification_types = NULL, # Filter by type(s)?
verbose = TRUE # Print progress messages?
)Returns a data.table (invisibly) with:
filename: Downloaded filenameurl: Source URLlink_text: Original link text from webpageversion: Extracted version number (NA if not
parseable)type: Classification typefile_path: Full path to downloaded filedownload_date: Timestamp when downloadedfile_size: File size in bytes
files <- download_classification_standards(
output_dir = "my_data/classifications"
)
# Only main classification standards
files <- download_classification_standards(
classification_types = "classificazioni_standard"
)
# Multiple types
files <- download_classification_standards(
classification_types = c("classificazioni_standard", "allegato")
)By default, only the latest version of each type is downloaded. To get all versions:
files <- download_classification_standards(
download_all = TRUE
)
# Update files even if they already exist
files <- download_classification_standards(
force_refresh = TRUE
)
files <- download_classification_standards(
verbose = FALSE
)After downloading, a metadata file is saved containing:
metadata <- readRDS("data-raw/classifications/download_metadata.rds")
# Structure:
# - download_date: When files were downloaded
# - source_url: URL of the source page
# - r_version: R version used
# - package_version: situas package version
# - files: data.table with file informationThis metadata can be used to:
The function automatically parses version numbers from filenames:
| Filename | Version | Type |
|---|---|---|
Rev.090-ST-Classificazioni-Standard.xlsx |
90 | classificazioni_standard |
Rev.005-ST-Classificazioni-Standard-1.xlsx |
5 | classificazioni_standard |
AllegatoA_AnalisiMigrazioneCp2011-Cp2021.xlsx |
NA | allegato |
When download_all = FALSE, the function keeps only the
file with the highest version number for each type.
The function will error if:
# Error: No Excel files found on page.
# Page may require JavaScript, authentication, or has changed structure.
# Error: Failed to connect to URL: ...
# Error: Failed to download <filename>
# Error: Package 'rvest' is required for web scraping.
# Install it with: install.packages('rvest')
result <- tryCatch(
{
download_classification_standards()
},
error = function(e) {
message("Download failed: ", e$message)
message("Try downloading manually from:")
message("https://urponline.lavoro.gov.it/s/standard-tecnici/classificazioni-standard")
NULL
}
)After downloading, you can read the Excel files with standard R packages:
library(openxlsx)
# Get latest classification standards file
files <- download_classification_standards(
classification_types = "classificazioni_standard"
)
# Read specific sheet
cp_data <- read.xlsx(
files$file_path[1],
sheet = "CP2021" # Professional qualifications 2021
)
# List all sheets
sheet_names <- getSheetNames(files$file_path[1])
print(sheet_names)
# Read multiple sheets
all_sheets <- lapply(sheet_names, function(sheet) {
read.xlsx(files$file_path[1], sheet = sheet)
})
names(all_sheets) <- sheet_names
library(situas)
library(openxlsx)
library(data.table)
# 1. Download latest classification standards
files <- download_classification_standards(
classification_types = "classificazioni_standard"
)
# 2. Check what was downloaded
cat("Downloaded:", files$filename[1], "\n")
cat("Version:", files$version[1], "\n")
cat("Size:", round(files$file_size[1] / 1024^2, 2), "MB\n")
# 3. Read the file
class_file <- files$file_path[1]
sheet_names <- getSheetNames(class_file)
# 4. Load professional qualifications (CP2021)
cp2021 <- as.data.table(read.xlsx(class_file, sheet = "CP2021"))
# 5. Explore the data
print(head(cp2021))
print(names(cp2021))
# 6. Use in your analysis
# ... your code here ...The Labor Ministry website is a Salesforce Community portal that may:
If scraping fails, you can:
use_js_scraping = TRUE (when implemented)JavaScript-enabled scraping is planned but not yet implemented. It will require:
To enable when available:
files <- download_classification_standards(
use_js_scraping = TRUE
)The implementation uses three internal helper functions:
Extracts version number and type from filename.
# Internal use only (not exported)
situas:::parse_classification_version("Rev.090-ST-Classificazioni-Standard.xlsx")
# Returns: list(version = 90, type = "classificazioni_standard")Comprehensive tests are included in
tests/testthat/test-download_classifications.R:
# Run tests
devtools::test_file("tests/testthat/test-download_classifications.R")Tests cover:
?download_classification_standards