Package index

Data Loading and Deduplication

Functions for reading OJA data, ESCO mappings, and deduplicating announcements.

read_oja_itaposts(): Read normalized OJA data from the itaposts DuckDB store
read_esco_mapping(): Read CPI-ESCO mapping table
read_isco_groups(): Read ESCO ISCO classification from CSV directory
read_ojv_zip(): Read OJV data from ZIP files
normalize_ojv(): Normalize OJV data from ZIP files
deduplicate_annunci(): Deduplicate job announcements

Classification Crosswalks

Build crosswalk tables between CPI, ESCO, and CP2021 classifications.

build_cpi_esco_crosswalk(): Build ESCO-to-CP2021 level 3 crosswalk
prepare_annunci_esco(): Merge announcements with ESCO mapping and parse dates
prepare_annunci_geography(): Prepare announcements with CPI geographic dimension
classify_esco_to_cpi(): Classify unmapped ESCO L4 codes to CPI groups via Naive Bayes
build_esco_cp_crosswalk(): Build a full ESCO-to-CP2021 crosswalk with its candidate structure
build_cp_profiles(): Build skill profiles per occupation-code pair
predict_cp4_knn(): Predict CP2021 level-4 codes via sector-boosted Jaccard k-NN
predict_cp5_knn(): Predict CP2021 level-5 codes via sector-boosted Jaccard k-NN
predict_cp5_centroid(): Predict CP2021 codes against static skill profiles

Skill Relevance Analysis

Compute specialization indices, TF-IDF diffusion scores, and skill classification.

compute_balassa_index(): Compute the Balassa index (Revealed Comparative Advantage) for skill-profession pairs
compute_skill_diffusion(): Compute skill diffusion using TF-IDF on aggregated skill frequencies
build_skillist(): Build a master skills list with metadata and diffusion classification
compute_idf_classification(): Compute IDF classification at the document level

Co-occurrence Networks

Build and filter skill co-occurrence matrices and networks.

compute_cooc_matrix(): Build a raw co-occurrence sparse matrix
filter_relative_risk(): Filter a co-occurrence matrix by relative risk
cooc_all_professions(): Compute skill co-occurrence networks for multiple professions

Community Detection

Identify skill communities within co-occurrence networks.

detect_skill_communities(): Detect skill communities from a co-occurrence edge list

Professional Distance and Clustering

Compute pairwise profession distances and hierarchical clustering.

build_skill_prof_sparse(): Build a sparse skill-profession matrix
compute_cosine_similarity(): Pairwise cosine similarity between matrix columns
compute_decomposed_distance(): Squared Euclidean distance decomposed by skill subsets
compute_skill_similarity(): Skill-skill cosine similarity based on profession profiles
compute_profession_distance(): Compute pairwise distances between professions based on skill profiles
cluster_professions(): Hierarchical clustering of professions

Salary Analysis

Extract and aggregate salary data by period and skill.

extract_salary_data(): Extract announcements with salary information
compute_salary_by_period(): Compute salary statistics by grouping period
compute_salary_by_skill(): Compute median salary per skill

Source Stability

Assess and filter OJA data sources by time-series stability.

prepare_source_tsibble(): Prepare a tsibble of announcement counts by source and month
compute_source_features(): Compute time-series features per source
filter_stable_sources(): Filter sources by stability criteria
filter_annunci_by_source(): Filter announcements to stable sources

Temporal Analysis

Track skill ranking evolution and rank variation over time.

compute_skill_ranking_series(): Compute skill ranking per profession over time periods
compute_skill_variation(): Compute year-over-year variation in skill rankings

Emerging Skills

Share panels, drift-net trends, onset, diffusion, taxonomy drift and the emergence score.

compute_share_panel(): Monthly skill share panel from posting-level incidence
compute_share_trend(): Drift-net logit trend of skill shares
compute_yoy_ratio(): Year-on-year ratio of pooled shares with Katz confidence interval
compute_share_acceleration(): Acceleration of the logit share trend
detect_skill_onset(): First month of consolidated presence of each series
compute_rca_panel(): Revealed comparative advantage of skills by profession and window
compute_diffusion_panel(): Diffusion of skills across professions by window
compute_drift_index(): Common drift index of logit shares
detect_taxonomy_drift(): Detect extraction and taxonomy breaks
calibrate_min_support(): Calibrate the minimum support by split-half reliability
score_emergence(): Composite emergence score and skill state
backtest_emergence(): Pseudo-prospective backtest of the emergence score

Changing and Emerging Professions

Profile turnover, decomposition by skill groups, breadth of change and emerging professions.

build_profile_pair(): Skill profiles of professions in two windows
compute_profile_turnover(): Turnover of profession skill profiles
decompose_profile_change(): Additive decomposition of profile change by skill groups
compute_turnover_null(): Split-half null distribution of profile turnover
compute_change_breadth(): Breadth of skill change by profession
compute_emerging_uptake(): Uptake of emerging skills by profession
classify_emerging_professions(): Classify emerging professions

Rolling-origin backtest, model selection and forecasts of skill shares with prediction intervals.

prepare_share_tsibble(): Prepare a tsibble of skill shares for forecasting
backtest_share_models(): Rolling-origin backtest of share forecasting models
select_share_model(): Select the forecasting model by stratum
forecast_share(): Forecast skill shares with prediction intervals

Design effects with sources as clusters, intervals for shares, empirical Bayes shares on code hierarchies, cluster bootstrap and hierarchical FDR.

share_interval(): Confidence interval for a share with a design effect
cluster_deff(): Design effect of a share with sources as clusters
smooth_deff(): Smooth design effects along a code hierarchy
fit_betabinom(): Maximum likelihood fit of a beta-binomial distribution
eb_share_tree(): Empirical Bayes shares on a code hierarchy
cluster_boot_share(): Cluster bootstrap interval for shares
tree_fdr(): Hierarchical false discovery rate control on a tree
finest_reliable_level(): Finest reliable level of a code hierarchy

Source Composition

Decomposition of share changes into within-source and composition components, fixed-weight share series.

decompose_share_change(): Decompose the change of a share into within-source and composition
fixed_weight_share(): Share series at fixed source weights

Stock and Trend Indices

Daily stock of open postings from dated events, duration survival, standardised-duration stock, weekly aggregation, trend extraction, Henderson and Whittaker smoothers, robust smoothing with post-smoothing bands and index numbers aggregated over sources.

events_from_intervals(): Event table from posting intervals
stock_from_events(): Daily stock of open postings from dated events
duration_survival(): Survival function of posting durations
standardized_stock(): Standardised-duration stock
to_weekly(): Aggregate a daily series to weeks
extract_trend(): Trend extraction with Fourier seasonality for weekly series
trend_diagnostics(): Diagnostics of extracted trends
trend_revisions(): Revisions of the trend as new data arrive
trend_index(): Index numbers from extracted trends
aggregate_index(): Weighted aggregation of source indices
compose_skill_index(): Skill index from total and share trends
index_jackknife(): Leave-one-source-out range of an aggregated index
trend_concordance(): Concordance of trends across sources
henderson_weights(): Symmetric Henderson weights
henderson_filter(): Henderson trend filter with Musgrave end weights
whittaker_smooth(): Whittaker-Henderson smoother with GCV
robust_smooth(): Robust trend smoothing with post-smoothing bands and re-smoothing

Skill Extraction

Extract profession-specific skills and build skill profiles.

extract_specific_skills_cpi3(): Extract non-diffuse skills at the CPI3 profession level
build_profession_skill_profile(): Build a comprehensive profession-skill profile table

Profession Matching

Rank professions by closeness to a declared skill set.

match_professions(): Classifica le professioni ESCO L4 piu' affini a un insieme di competenze
match_professions_prepare(): Pre-computa la matrice profilo professionale per il matching
match_professions_score(): Calcola la classifica delle professioni a partire da un prep

Visualization

Plot co-occurrence networks and skill ranking series.

plot_cooc_graph(): Plot a skill co-occurrence network graph
plot_skill_ranking(): Plot temporal skill ranking for a profession

Datasets

Pre-computed datasets included with the package.

isco_gruppi: ESCO ISCO group classification
skillist: Master skills list
cooccorrenza: Skill co-occurrence network