This README.txt file was generated on 2025-12-11 by Ana Pastor-Merino

--------------------------------------------------
GENERAL INFORMATION
--------------------------------------------------

Title of Dataset:
AI Adoption in Spain (2023–2025): A Web-Derived Dataset Based on LLMs

Author Information:

Ana Pastor-Merino
Universitat Politècnica de València
Cami de Vera s/n, 46022 Valencia, Spain
Email: anpasme@doctor.upv.es
ORCID: https://orcid.org/0009-0004-5790-2846

Xavier Martínez-Barbero
Universitat Politècnica de València
Email: xmarbar@upv.es
ORCID: https://orcid.org/0000-0002-6040-8118

Josep Domenech
Universitat Politècnica de València
Email: jdomenech@upvnet.upv.es
ORCID: https://orcid.org/0000-0002-7302-5810

Date(s) of Data Collection:
Corporate website crawling conducted in 2023 and 2025.
Firm-level variables retrieved from the SABI database for the same years.

Geographic Location of Data Collection:
Spain (NUTS-3 regional level)

Funding Sources:
This work was supported by the Generalitat Valenciana, Spain under Grant CIAICO/2023/272; the Agencia Estatal de Investigación, Spain
(MCIN/AEI/10.13039/501100011033) and ERD-F/EU under Grant PID2023-152106OB-I00; and FPU24/01116, Ministerio de Ciencia e Innovación (Spain).

General Description:
This dataset provides indicators of artificial intelligence (AI) adoption across 112,814 Spanish firms in 2023 and 2025, producing 225,628 firm-year observations.
Corporate websites were crawled, converted to text, segmented, semantically filtered using embeddings, and evaluated with a rubric-based classification using GPT-4o-mini.
The dataset distinguishes between: (1) AI adoption (overall), (2) AI use in internal processes, and (3) AI use in products or services.
Indicators are aggregated at the province × NACE 2-digit sector × firm-size level.

Keywords:
Artificial intelligence, web mining, LLMs, digital transformation, corporate websites, Spain.


--------------------------------------------------
SHARING / ACCESS INFORMATION
--------------------------------------------------

Open Access:
Yes

Citation:
Pastor-Merino, A., Martínez-Barbero, X., and Domenech, J. (2025).
AI Adoption in Spain (2023–2025): A Web-Derived Dataset Based on LLMs.
Data in Brief. (Forthcoming)

Links to Publications:
Pastor-Merino, Martínez-Barbero & Domenech (2025). Detecting AI adoption at scale: a web mining and LLM-based methodology. Scientometrics.


--------------------------------------------------
DATA & FILE OVERVIEW
--------------------------------------------------

Files Included:
2023_AIAdoption.csv
2025_AIAdoption.csv

Relationship Between Files:
Both files have identical variable structure.
Each file contains aggregated indicators for the corresponding year.

Dataset Version:
Processed indicators derived from the web scraping and LLM pipeline.

Total Size:
Approximately 464 KB.


--------------------------------------------------
METHODOLOGICAL INFORMATION
--------------------------------------------------

Description of Data Collection:
Corporate websites were crawled in 2023 and 2025 following consistent technical, ethical, and rate-limiting guidelines.
Text was extracted from HTML pages and publicly accessible PDFs.
Firm-level structured information was retrieved from the SABI database (sector, size, province).

Description of Processing Methods:
1. Removal of boilerplate text (menus, footers, cookies, duplicated blocks).
2. Segmentation of website text into 1000-byte fragments.
3. Generation of embeddings using the model "text-embedding-3-small".
4. Semantic filtering based on cosine similarity to AI reference embedding.
5. Classification of selected fragments using GPT-4o-mini with a unified rubric:
   - identification of implemented AI,
   - distinction between AI in internal processes and AI in products/services,
   - exclusion of generic, vague, or aspirational claims.

Software Used:
Python and R for data processing.
Output provided in CSV (UTF-8) format.

Reference Methodology:
Pastor-Merino et al. (2025), Scientometrics.


--------------------------------------------------
DATA-SPECIFIC INFORMATION
--------------------------------------------------

Number of Variables:
7

Number of Rows:
Dependent on province-sector-size combinations.
Firm-level input corresponds to 112,814 companies per year.

Variable List:

Province:
Spanish region (NUTS-3).

NACE_2D:
Two-digit NACE Rev.2 economic activity code.

Size:
Firm size category (Micro, Small, Medium, Large).

AI_Adoption:
Share of firms with validated evidence of implemented AI.

AI_Process:
Share of firms using AI in internal processes.

AI_Products:
Share of firms offering AI-enabled products or services.

n_companies:
Number of firms with valid website content in each province-sector-size cell.