https://doi.org/10.37815/rte.v37n1.1310

Original articles

Piloting, Validation, and Definition of a
Self-Perception Instrument for University Teachers' Digital Competencies in
English Language Teaching in Costa Rica

Pilotaje,
Validación y Definición de un Instrumento de Autopercepción para las
Competencias Digitales de Docentes Universitarios en la Enseñanza del Idioma
Inglés en Costa Rica

Mario
Esteban Alpizar Rodríguez1 https://orcid.org/0000-0003-0254-5680

1Universidad Americana, San José, Costa Rica

[email-masked]

Sent:       2025/04/14

Accepted:       2025/07/18

Published:       2025/12/15

Abstract

This study aimed to develop, validate, and
define a self-perception instrument to assess the digital competencies of
university teachers in English language teaching in Costa Rica. Based on
previous research hand theoretical frameworks on digital competencies, a
questionnaire was constructed, focusing on key dimensions such as digital
literacy, ICT integration in teaching, and teacher appropriation and
commitment. The methodology employed a quantitative approach with a
correlational and cross-sectional design, utilizing statistical techniques,
including Confirmatory Factor Analysis (CFA), Classical Test Theory (CTT) and
Item Response Theory (IRT), to evaluate the instrument's validity, reliability
and objectivity. The results enabled informed decision-making and the
definition of the most appropriate instrument for application to university
English language teachers at the national level.

Keywords: Digital
competencies, teacher competencies, didactic resource use, technological
competencies, Information and Communication Technologies.

Summary: Introduction, Methodology, Results and Discussion, Conclusions and recommendations.

How to cite: Alpizar, M. (2025).
Piloting, Validation, and Definition of a Self-Perception Instrument for
University Teachers' Digital Competencies in English Language Teaching in
Costa Rica. Revista Tecnológica - Espol, 37(2), 14-35. https://rte.espol.edu.ec/index.php/tecnologica/article/view/1310

Resumen

Este estudio tuvo como objetivo
desarrollar, validar y definir un instrumento de autopercepción para evaluar
las competencias digitales de los docentes universitarios en la enseñanza del
idioma inglés en Costa Rica. Basado en investigaciones previas, modelos y
marcos teóricos sobre competencias digitales, se diseñó y construyó un
cuestionario enfocado en dimensiones y subdimensiones claves como la
alfabetización digital, la integración de las TIC en la enseñanza y la
apropiación y el compromiso docente. La metodología siguió un enfoque
cuantitativo con un diseño correlacional y transversal, utilizando técnicas
estadísticas como el Análisis Factorial Confirmatorio (AFC), la Teoría Clásica
del Test (TCT) y la Teoría de Respuesta al Ítem (TRI), para evaluar la validez,
confiabilidad y objetividad del instrumento. Los resultados permitieron una
toma de decisiones fiables e informadas y la definición del instrumento más
adecuado, así como sus dimensiones y competencias digital, para su aplicación a
docentes universitarios de inglés a nivel nacional.

Palabras
clave: Competencias digitales, competencias docentes, uso
de recursos didácticos, competencias tecnológicas, Tecnologías de la
Información y la Comunicación.

Introduction

This
article presents the development, validation, and definition of a
self-perception instrument tailored to assess the digital competencies of
university-level English language teachers in Costa Rica. It is built upon two
previous works by the author: a systematic review of instruments in this field
(Alpizar, 2024b) and a postgraduate thesis on the relationship between digital
competencies and the didactic use of ICT in English teaching (Alpizar, 2024a).

Within the
vast diversity of models, frameworks, and profiles on teachers' digital
competencies, existing references provide general guidelines for digital
teaching competencies but lack a contextual focus on English language teaching
and the practical integration of ICT. Thus, this research aims to generate a
more context-specific tool that reflects both self-perceived proficiency and
the pedagogical application of ICT by English language teachers.

In this
article the following acronyms are employed:

Table 1

Glossary of Technical Acronyms

Acronym

Meaning

CFI

Comparative Fit Index

TLI

Tucker-Lewis Index

CFA

Confirmatory Factor Analysis

IRT

Item Response Theory

CTT

Classical Test Theory

ICT

Information and Communication Technologies

Source: Own elaboration

The
following matrix facilitated the creation of a suitable self-perception
instrument for measuring teachers' digital competencies, considering the
existing theories surrounding the research topic:

Table 2

Dimensions
and Teacher Digital Competencies

Dimension

Sub-Dimension

Digital Competencies

Digital Literacy

Information Literacy

Navigation, search, and filtering of information,
data, and digital content (using different software and browsers to access
websites)

Management, evaluation, storage, and retrieval of
data

Use of electronic journals

Creation, editing, use, and administration of
websites

Creation, use, and administration of blogs

Use of specialized databases, libraries, and
repositories

Communication and Collaboration

Promotion of communication and interaction skills
and strategies with the community through ICT

Exchange of data and information with other users

Promotion of online citizen participation

Promotion of collaboration through official
channels

Promotion of netiquette, language used in
networks, behavior, and respect towards other users

Management of digital identity and profile

Use of instant messaging and video calling
through applications (Messenger, built-in messaging service in social
networks, messaging on platforms such as Moodle or Blackboard, among others)

Use of email

Use and administration of spaces, groups, and
posts on social networks

Use of learning management platforms (Moodle,
Blackboard, Schoology, Google Classroom, among others)

Use of text messages

Use, administration, and moderation of users and
content in online forums

Creation, editing, use, and administration of
wikis

Creation and Manipulation of Digital Content

Promotion of respect for authorship rights,
licenses, and programming

Use of editing, creation, and manipulation
functions of files through word processors (doc, docx, txt, pdf, cbr, among others)

Creation, conversion, and editing of images (PNG,
JPEG, BMP, GIF, PSD, TIFF, among others)

Use of presentation functions (PPT, Canva, Prezi,
among others)

Use and management of information sets and
databases

Use of editing, creation, and manipulation
functions of spreadsheets

Creation, recording, conversion, and editing of
audio files (MP3, WAV, among others)

Creation, recording, conversion, and editing of
video files (mp4, avi, flash, open source)

Technological Devices

Use, administration, installation, and updating
of laptops and PCs

Use, administration, installation, and updating
of tablets

Use, administration, installation, and updating
of smartphones

Use and manipulation of local storage devices
(USB, CD, DVD, external memory, among others)

Use and manipulation of global storage media
(iCloud, Drive, Dropbox)

Basic technical knowledge management by the
teacher regarding networks, connections, software, and hardware, among
others, that do not require immediate intervention from a technical support
department.

Security

User Security

Ensuring cybersecurity for users in spaces used
for activity development, which should not compromise personal information or
family data of teachers and students.

Health and well-being when using ICT (having
suitable space, accessories, a chair, desk, and other supplies for their
tasks)

Digital citizenship and responsibility.

Web information publication (passwords and
geolocation).

Institutional Data Security

Access to and integrity of suitable equipment and
information (valid author licenses and open-source software)

Ethical and legal use of ICT

Access to and verification of the
intra-institutional network and the internet

Ensuring the protection and security of
institutional data

Integration of ICT in Teaching

Curricular Scope

Integration of ICT use in the curriculum

Planning, development, monitoring, and evaluation
of ICT use in teaching processes

Use of didactic materials and ICT resources

Recognition, understanding, and expression of
emotions through the use of ICT

Pedagogical Scope

Design of educational scenarios supported by ICT

Assessment of student training in the use of ICT

Creating a climate of motivation, empathy, and
learning through the use of ICT

Didactic Scope

Application of activities based on previous
experiences related to the use of ICT

Evaluation and feedback on the effectiveness of
ICT-mediated educational scenarios

Provision and definition of ICT environment,
resources, and activities

Designing activities, learning situations, and
assessments that integrate the use of ICT

Organizational Scope

Identification and definition of the most
appropriate use of ICT

Ensuring equitable access to the use of ICT

Guiding students in the learning process based on
individual needs and interests

Appropriation and Teacher Commitment

Identification of technological needs (equipment,
ICT knowledge, connectivity, among others)

Implementation of aspects of innovation and
creativity

Self-reflection on the use of ICT

Teacher commitment to continuous practice and
training in the didactic use of ICT

Knowledge of policies on the use of ICT at
institutional, local, national, regional, and international levels by
different competent entities

Source: Alpizar, M. (2024b)

For this
research, the security dimension was not considered, as some of its
competencies were deemed to be covered within the Digital Literacy dimension,
while others fall under the responsibilities of institutional departments
(e.g., Occupational Health and Information Technologies) or relate to the
technological infrastructure of each institution.

Research
Problem

Considering
the previously outlined and analyzed context, the
research problem that guided this study was:

·
To
address the process of pilot testing, validation, and definition of an
appropriate instrument to measure the self-perception of digital competencies
in university teachers of English language teaching in Costa Rica.

General Objective

·
To
explain the process of piloting, validation, and definition of an appropriate
instrument to measure the self-perception of digital competencies in university
teachers of English language teaching in Costa Rica.

Specific Objectives

·
To
understand the process of piloting, validation, and definition of an
appropriate instrument to measure the self-perception of digital competencies
in university teachers of English language teaching in Costa Rica.

·
To
analyze the results of the pilot test and make the
necessary adjustments to the instrument to improve its validity and
reliability.

·
To
define the final version of the self-perception measurement instrument for
teachers' digital competencies, considering the results of the pilot test and
validation process.

To
thoroughly document the design, pilot test, and validation process of the
instrument to establish guidelines for its future application and continuous
improvement.

Methodology

The study
employed a quantitative approach within the positivist paradigm, based on
careful observation and the measurement of objective reality. This approach
enabled the testing of theories through the analysis of relationships between
variables, as was the case in this research.

According
to Creswell (2013), variables can be measured using instruments, enabling the
collection of quantified data for analysis through statistical procedures. This
approach is applied in the present study to quantify the relationship between
teachers' digital competencies and their didactic use of ICT.

As noticed
by Best and Kahn (2006), (2013), Delgado (2014), Hernández et al. (2014), and
Hernández and Mendoza (2018), correlational research seeks to identify the
relationship or degree of association between two or more concepts, categories,
or variables within a given sample or context.

In this
case, the expectation was that if Variable A (teachers' digital competencies)
is systematically associated with Variable B (the didactic use of ICT), then
predicting future phenomena may be possible, and the results may suggest
additional hypotheses for testing.

Thus, the
research hypothesis was formulated as follows:

·
The
greater the number of digital competencies a teacher possesses, the higher
their didactic use of ICT. Conversely, the fewer digital competencies a teacher
has, the lower their didactic use of ICT.

Scope of the
Study

According
to Delgado (2014), Hernández et al. (2014), and Hernández and Mendoza (2018),
if two variables are correlated and the magnitude of their association is
known, this provides a foundation for predicting, with varying degrees of
accuracy, the approximate value of one variable based on the known value of the
other.

This study
followed a non-experimental, cross-sectional approach, meaning no variables
were deliberately manipulated. As stated, non-experimental studies do not
intentionally modify independent variables to observe their effects on others
because the phenomena have already occurred, are occurring, or will continue to
occur. This aligns with the perspectives of Kerlinger & Lee (2000) and
Delgado (2014), who describe non-experimental research as a systematic
empirical inquiry in which the researcher does not have direct control over the
variables, as their manifestations have already occurred or are inherently
unmodifiable.

Research
Design

A
cross-sectional (transversal) design was selected as the most appropriate
non-experimental approach. A key characteristic of this type of
non-experimental design is that data are collected at a single point in time,
which was deemed appropriate following the global COVID-19 pandemic, as the
study was based on the experiences of university English language teachers
during the period of virtual or remote education.

Furthermore,
correlational cross-sectional designs seek to establish relationships between
variables. Consequently, considering these elements of non-experimental design,
the study employed a questionnaire as its instrument to identify teachers'
self-perception of their digital competencies and their frequency of ICT
implementation for didactic purposes.

Population and
Sample

As Best and
Kahn (2006), Hernández et al. (2014), and Hernández and Mendoza (2018)
indicated, the target population had to meet specific characteristics aligned
with the research objectives and be quantitatively measurable. Based on
approval requests sent to both public and private universities in Costa Rica,
the estimated sample size included 153 university English language teachers
nationwide.

These
individuals shared a similar teaching background in English language
instruction and met the regulatory requirements to practice their profession at
the university level. All participants held at least a bachelor's degree in
English Language Teaching, which was established as an inclusion criterion for
the study's target population and sample.

However, a
limitation emerged regarding private universities, as some institutions did not
provide an exact number of participating teachers, making it uncertain how many
instructors actually took part in the study. Additionally, despite requesting
participation from more private universities, some did not respond, further
complicating the determination of the total population of university-level
English teachers in Costa Rica.

Given these
challenges, the study adopted a non-probability sampling method, following the
recommendations of Hernández et al. (2014) and Hernández and Mendoza (2018).
These authors stated that in non-probability sampling, element selection is not
based on probability but rather on factors related to the study's
characteristics and the researcher's objectives.

Among
non-probability sampling techniques, convenience sampling was chosen, as this
method involves selecting participants who are readily available and accessible
at the time of data collection (Best & Kahn, 2006; Fraenkel et al., 2012;
Hernández et al., 2014).

Variables and
Instruments

According
to Best and Kahn (2006), correlational research focuses on examining
relationships between variables, testing hypotheses, and developing
generalizations, principles, or theories with universal validity.

The
expectation was that if Variable A is systematically associated with Variable
B, it might enable the prediction of future phenomena, and the results may
suggest additional hypotheses for further testing.

The
instrument included key demographic aspects (gender, age range, and years of
experience) to profile the sample population. Variables were operationally
defined to establish the measurement procedures, with a detailed description of
the instruments aligned to the research objectives.

Table 3

Variables,
conceptualization, instrumentalization, and operationalization

Variables                    Definition

Teachers' digital competencies

Didactic use of ICT in English teaching courses

Conceptual

Based on the type of responses, it is
classified as an ordinal qualitative variable, since its items are
prescriptive, with restricted responses, ordered from lowest to highest in
terms of proficiency level according to a Likert scale. It is worth noting
that this is a latent variable (construct), as digital teaching competencies
are based on a person's cognitive traits. Additionally, it is considered an
index variable, as it has been theorized that competencies comprise skills,
knowledge, attitudes, practices, emotions, and other aspects, which generate
a series of manifestations that serve as necessary indicators to assess
digital teaching competencies.

Based on the type of responses, it is
classified as an ordinal qualitative variable, since its items are
prescriptive, with restricted responses, ordered from lowest to highest in
terms of frequency of use according to a Likert scale. It is a latent variable
(construct) because the didactic use of ICT is based on the frequency of
technology use in the classroom. Furthermore, it is considered an index
variable, as it has been theorized that competencies generate a series of
manifestations that serve as the necessary indicators to assess the didactic
use of ICT.

Instrumental

The study measured teachers' perceptions of
their digital competencies in English language teaching and identified the
competencies they possess, as outlined in the proposed theoretical framework.

Through the instrument, the relationship
between digital competencies of English language teaching staff and the use
of technological resources, as well as the organization of didactic work with
ICT, is described. Depending on the level of digital competencies teachers
possess, the didactic use of ICT may be enhanced or not.

Operational

The first part of the questionnaire, using
a three-point Likert scale, focused explicitly on teachers' digital
competencies. It consisted of 12 items and aimed to assess the digital
competencies of English language teaching staff. The questionnaire was structured
into dimensions and sub-dimensions of different digital competencies, with
the following distribution: Digital Literacy (6 items), Teacher Integration
of ICT (2 items), and Teacher Appropriation and Commitment (2 items).

The second part
of the questionnaire aimed to gather descriptions from English language
teaching staff regarding elements such as ICT use, lesson planning phases,
didactic strategies, and instructional materials for teaching English through
ICT. The dimensions considered were: Digital Literacy (17 items), Teacher
Integration of ICT (2 items), and Teacher Appropriation and Commitment (1
item).

Source: Alpizar, M. (2024a)

According
to Wallace (1998), although the construction and implementation of a
questionnaire require time, once this stage is completed, it allows for its
application to a large population, as in this case, and enables the efficient
and effective extraction of the necessary results.

This
technique selection was sufficient to support the collection of the necessary
information, especially since it aligns with the theoretical and methodological
approach. The measurement object allowed for the specification of construct
indicators; the items correspond to each indicator defined by the dimensions of
digital teaching competencies and the didactic use of ICT, as per Delgado
(2014).

Determination
of the validity, reliability, and objectivity of the instrument

According
to Hernández et al. (2014), Delgado (2014), and Hernández and Mendoza (2018),
reliability refers to the consistency with which repeated application of an
instrument to the same individual or object produces identical results. This
was demonstrated through internal consistency and coherence measures using
Cronbach's alpha coefficient.

Validity
refers to the extent to which an instrument truly measures the intended
variable. Validity can be demonstrated through three different types of
evidence: content-related, criterion-related, and construct-related evidence.
For this research, all three types were selected.

Content
validity was established first, as it reflects how well the instrument covers
the specific content domain being measured, according to Delgado (2014) and
Hernández et al. (2014). The questionnaire included dimensions and
subdimensions of digital teaching competencies based on theoretical
foundations.  For this stage, four
university professors specializing in English language teaching and Educational
Technology validated the instrument, providing feedback on item wording,
application inclusion, and scale reduction.

Following
Rojas et al. (2022), a pilot test was conducted using items approved by
experts, administered to 125 English teachers from elementary, secondary, and
open education levels.

Construct
validity was established through a Confirmatory Factor Analysis (CFA) of the
collected data, which examined the instrument's internal structure. Criterion
validity was established by comparing the instrument with external criteria
measuring the same construct, using predictive validity in this case.

Additionally,
construct validity was considered from a scientific perspective, referring to
how well the instrument represents and measures a theoretical concept, as well
as its operationalization, and three stages were implemented.

In the first stage, a theoretical framework
was developed through a literature review, establishing the relationship
between digital teaching competencies and ICT didactic use.

The second
stage included the statistical analysis of concept associations through a
correlation design, supplemented by expert validation.

The next
stage involved interpreting empirical evidence regarding the construct's
validity. The instrument successfully measured the relationship between digital
teaching competencies and ICT didactic use in English language teaching,
achieving full validity through these complementary approaches.

Finally,
objectivity, the extent to which the instrument is free from researcher bias
and tendencies in its administration, scoring, and interpretation, was ensured
by standardizing administration procedures, maintaining consistent instructions
and conditions for all participants, and applying an unbiased analysis method
to interpret the results.

Results and
Discussion

Preliminary Pilot Testing of the Instrument

To appropriately define and justify the
instrument, a preliminary pilot test was conducted using items approved by
experts with a sample of 125 English teachers from elementary, secondary, and
open education settings. With the collected data, a Confirmatory
Factor Analysis (CFA) was performed, along with other necessary tests
detailed below. This process enabled the justification and strengthening of the
instrument's internal structure, ensuring that it effectively measured what it
was intended to measure.

Confirmatory Factor Analysis (CFA)

The items
associated with each dimension were assigned to the model to ensure the best
possible representation. It is important to note that, for the purpose of
applying the analysis, Spanish acronyms were used for the stipulated dimensions
to facilitate their entry into the corresponding software. These acronyms are
as follows: Digital Literacy (AD), Teacher Integration of ICT (ID), and Teacher Appropriation and Commitment (ACD).

The results obtained are presented below:

Figure 1

Factor Loadings by Dimensions

Source: Prepared based on data
obtained from the preliminary pilot test

It is important to highlight that, for each
dimension, the factor loadings associated with each item were expected to fall
within similar value ranges. The analysis revealed that:

·
In
Dimension (AD), item values ranged between 0.54 and 0.77.

·
In
Dimension (ID), item values ranged between 0.48 and 0.70.

·
In
Dimension (ACD), most of the eight items had values between 0.50 and 0.75.

However, two items deviated significantly:

·
Item
22 (use of spreadsheets): factor loading of 0.39.

·
Item
27 (use of local storage devices): factor loading of 0.44.

Both items showed lower factor loadings
compared to the others, preliminarily indicating that these items did not align
well with their assigned dimension.

Based on the previous figure and the
Confirmatory Factor Analysis (CFA) results, the following comparative tests
were conducted:

a)      A comparison was made to determine
whether the proposed model fits better than a null model (without variables),
yielding a p-value of 0.000, which indicates a significantly better fit than
the null model.

b)      A comparison was made to determine
whether the proposed model fits better than a baseline model (a theoretical
model that covaries the dimensions), also yielding a p-value of 0.000,
indicating a significantly better fit than the baseline model.

Both results indicated that the proposed model is superior to simpler
theoretical models. However, additional fit measures were examined to assess
the model's accuracy:

Table 4

Test/Measure and Value

Test/Measure

Value

Comparative Fit Index (CFI)

0,731

Tucker-Lewis Index (TLI)

0,709

Source: Prepared based on
collected data.

Both fit measures yielded values between 0 and 1, where the closer the
value is to 1, the better the model fit. Values around 0.9 are considered
acceptable. However, in both cases, it was observed that the evaluated model
did not provide an optimal fit to the observed data, prompting further analysis
of the data.

The initial step involved examining eigenvalues to determine the optimal
number of dimensions, with results as follows:

Figure
2

Sedimentation Plot

Source: Prepared based on
eigenvalues

The graphical analysis showed up to 7 possible
dimensions, but the optimal and recommended solution was to retain two factors,
as indicated in the graph legend. This revealed initial issues in dimension
detection.

Given the research objective to optimize the
three predefined dimensions, these were used as reference points for internal
analysis to improve overall consistency. The analysis incorporated both
Classical Test Theory (CTT) and Item Response Theory (IRT), as indicated by
Martínez, Hernández, and Hernández (2014).

Dimension: Digital Literacy

Using CTT
methodology (Martínez, Hernández, and Hernández, 2014), this dimension
demonstrated excellent reliability with Cronbach's α = 0.855. Item deletion
analysis showed this value would decrease if any items were removed, confirming
all items should be retained.

Table 5

Dimension: Digital Literacy
(CTT)

Item

Item Mean

Alpha if Item is Deleted

Item 1

2,015873

0,8465086

Item 2

1,912698

0,8460739

Item 3

1,825397

0,8427036

Item 4

1,841270

0,8429890

Item 5

2,182540

0,8437073

Item 6

2,285714

0,8325510

Item 7

2,142857

0,8311771

Item 8

2,277778

0,8397880

Item 9

2,658730

0,8455701

Item 10

2,547619

0,8452137

Source: Prepared based on the results of the items

According to IRT, as stated by Martínez, Hernández, and Hernández
(2014), when dealing with polytomous models (i.e., models with items that have
more than two response categories), Partial Credit Models should be employed.
Initially, it is necessary to determine which model best fits the dimension.
The hypothesis test indicates that the two-parameter Rasch model best explains
the dimension, as shown below:

Table 6

Dimension: Digital Literacy (Rasch
model)

Models

LRT

Degrees of Freedom

p-value

Rasch - 1Parameter

19,38

1

<0,001

1Parameter - 2Parameter

37,42

10

<0,001

Source: Prepared based on results for the dimension

Once the model was identified, the aim was to ensure that the cut-off
points between the responses of the items were at a similar distance and not
overly skewed toward the positive or negative side of the measured competency.
Therefore, in general, the items are presented appropriately, as shown below:

Table 7

Dimension: Digital Literacy (Categories and Discrimination)

Item

Category 1

Category 2

Discrimination

x1

Item 1

-0,957

0,864

1,125

x2

Item 2

-0,403

0,789

1,031

x3

Item 3

-0,427

1,239

1,189

x4

Item 4

-0,458

1,192

1,183

x5

Item 5

-1,385

0,498

1,303

x6

Item 6

-1,581

0,339

2,342

x7

Item 7

-1,134

0,562

2,540

x8

Item 8

-1,494

0,293

1,920

x9

Item 9

-2,101

-0,840

1,655

x10

Item 10

-2,135

-0,444

1,569

Source: Prepared based on the results of the items

The items, in general, appeared optimal, but there are six that are
important to highlight as follows:

Figure
3

Items 2, 6, 7, 8, 9, and 10
discrimination

Source: Prepared based on the
results of the items

The previous figure highlights:

In item 2, the central response
category did not make a significant contribution.

The responses of items 6 and 7
discriminated well and very sharply.

The responses to Item 8
discriminated well; however, for the two lowest response categories,
discrimination was skewed more toward the negative side of the skill.

People with neutral or positive
levels of the skill were identified by the highest category, while lower
levels were reflected in response categories 1 and 2.

Dimension: Teacher Integration of ICT

According
to Classical Test Theory (CTT), the Cronbach's Alpha coefficient
for this dimension was 0.8478, which is considered quite optimal. Then, when analyzing the impact of removing any item, it was observed
that the coefficient decreased in each case. Therefore, according to CTT, no
item should be removed.

Table 8

Dimension: Teacher Integration of ICT (CTT)

Item

Item Mean

Alpha if Item is Deleted

Item 11

4,119048

0,8418642

Item 12

4,357143

0,8367199

Item 13

3,865079

0,8245338

Item 14

4,603175

0,8345965

Item 15

3,904762

0,8246825

Item 16

3,690476

0,8279005

Item 17

4,079365

0,8342958

Item 18

3,484127

0,8312106

Item 19

4,595238

0,8409413

Item 20

4,523809

0,8367800

Source: Prepared based on the results of the items

According to the Item Response Theory (IRT) approach, since these are
polytomous models, Partial Credit Models must be used. To determine the best
fit for this dimension, the following results were obtained:

Table 9

Dimension: Teacher Integration of ICT (Rasch model)

Models

LRT

Degrees of Freedom

p-value

Rasch - 1Parameter

0.06

1

0.801

Source: Prepared based on
results for the dimension

Since items 19 and 20 did not show
diversity in the responses obtained, it was not possible to present the table
with category intersections. Therefore, a direct analysis of the item
characteristic curve was conducted.

The following observations were made
from the item characteristic curves:

·
Item
11: Response category 2 did not contribute, while categories 3 and 4
overlapped.

·
Item
12: Item discrimination based on skill level was very low in the negative skill
range, and categories 2 and 4 contributed almost nothing.

·
Item
13: Categories 2 and 4 did not contribute at all, and category 3 contributed
only slightly below the neutral skill level.

·
Item
14: This item did not discriminate well because category 5 encompassed both
neutral and high skill levels, while categories 3 and 4 did not help determine
low skill levels.

·
Item
15: Category 2 did not contribute, and categories 3 and 4 did not add
significantly.

·
Item
16: Categories 2 and 4 did not contribute, category 3 contributed very little,
and category 5 captured both neutral and positive skill levels.

·
Item
17: The discrimination from categories 3 and 4 was almost non-existent, while
categories 1 and 2 effectively distinguished low skill levels, and category 5
identified neutral and positive skill levels.

·
Item
18: Categories 2 and 4 did not contribute. Category 3 provided slight
discrimination at neutral skill levels, while categories 1 and 5 discriminated
at negative and positive skill levels, respectively.

·
Item
19: No one selected category 1, which shifted the graph values by one category.
Categories 3 and 4 did not contribute; category 2 only discriminated at low
skill levels; and category 5 captured neutral and high skill levels.

·
Item
20: No one selected category 1, causing the graph values to shift by one
category. Category 3 did not contribute; category 2 discriminated at low skill
levels; category 4 slightly discriminated at low levels; and category 5
distinguished between neutral and high skill levels.

Figure
4

Items 11, 12, 13, 14, 15,
16, 17, 18, 19, and 20 discrimination

Source: Prepared based on the
results of the items

Dimension: Teacher Appropriation and Commitment

According to Classical Test Theory
(CTT), the Cronbach's Alpha for this dimension was 0.8250, which is considered
quite optimal. When applying CTT, it was found that removing Item 22 (related
to the use of spreadsheets) would increase internal consistency to 0.8282. Upon
re-evaluating the consistency of the dimension without this item, no further
items were found to require elimination, as doing so would decrease internal
consistency. It is worth noting that Item 27 (focused on the use of different
local storage devices) showed a slight decrease in Cronbach's Alpha if removed,
with a value of exactly 0.8270.

Considering the Item Response Theory (IRT), since these are polytomous
models, Partial Credit Models must be used. To determine the best fit for this
dimension, the following results were obtained:

Table 10

Dimension: Teacher Appropriation and Commitment (Rasch model)

Models

LRT

Degrees of Freedom

p-value

Rasch - 1Parameter

1.43

1

0.233

Source: Prepared based on
results for the dimension

Since Item 26 did not show diversity in
responses, it is not possible to present the table with the category
intersections. Therefore, a direct analysis of the item characteristic curve
was conducted.

It is essential to note that, in general terms,
category 5 in Items 21 to 27 encompassed both positive and neutral skill
levels. Below is a detailed breakdown by item:

·
Item
21: Category 1 indicated low skill levels, category 2 contributed nothing, and
categories 3 and 4 also reflected low skill levels.

·
Item
22: Category 1 demonstrated low skill levels, while categories 2 and 3 also
showed low levels closer to neutral, and category 4 contributed nothing.

·
Item
23: Categories 1 and 3 contributed nothing, whereas categories 2 and 4
indicated low skill levels.

·
Item
24: Category 1 reflected low skill levels, category 2 contributed nothing, and
categories 3 and 4 slightly indicated low skill levels.

·
Item
25: Category 1 showed low skill levels and discriminated well; categories 3 and
4 slightly reflected low skill levels near neutral. Category 2 contributed
nothing.

·
Item
26: Since no one selected category 1, the graph values shifted by one category.
Categories 2 and 4 contributed nothing; category 3 showed low skill levels, and
category 5 increased rapidly.

·
Item
27: Correct discrimination was observed at low skill levels in category 1,
followed by category 3, which discriminated well near neutral levels.
Categories 2 and 4 contributed nothing.

·
Item
28: All five categories contributed some level of discrimination. Categories 1
through 3 reflected low skill levels; category 4 indicated a level very close
to neutral; and category 5 increased rapidly across positive skill levels.

·
Item
29: Category 1 declined rapidly in low skill levels; category 2 contributed
nothing; categories 3 and 4 spanned low and neutral levels; and category 5
increased rapidly at positive skill levels.

·
Item
30: Category 1 declined slowly across low skill levels, covering a significant
range; category 2 contributed nothing; categories 3 and 4 reflected neutral
levels; and category 5 increased rapidly at positive skill levels.

Figure 5

Items 21, 22, 23, 24, 25, 26, 27, 28, 29, and 30 discrimination

Source: Prepared based on the
results of the items

Conclusions and recommendations

Based on the results of the tests performed, the following decisions
were made regarding the final version of the instrument:

•
To achieve more appropriate and accurate item functioning and behaviour,
considering that the curves must have similar shapes, with the lowest category
always on the left and the highest on the right, the scale was reduced from a
five-point to a three-point scale to fit better. This followed the same
approach used for the first dimension, which performed best, unlike the other
two dimensions, as it consisted of a three-point scale. This decision was
supported by previous analyses, which showed that response categories 2, 3, and
4 in most items did not contribute, making it more reasonable to reduce the
scale. The same applied to items 21 to 30, where response categories 2 and 4
did not contribute.

•
Items that were weak in measuring their intended constructs were
reworded to strengthen their clarity and precision.

•
Items 22, focused on the use of spreadsheets, and item 27, related to
the use of different local storage devices, were eliminated, as their removal
increased internal consistency according to Cronbach's Alpha.

•
Following the recommendations of the advisors, items 23 and 24, related
to using audio, podcasts, and videos to expose students to the English
language, were divided into two separate items each: one focused on use and
another on creation.

•
Furthermore, it was suggested to incorporate two new items: one related
to basic knowledge of network functionality and another addressing the use of
different operating systems.

The validated instrument demonstrated strong psychometric properties and
contextual relevance, effectively assessing the self-perceived digital
competencies of English language university teachers and their frequency of ICT
integration in English language teaching.

Its structure enables decision-makers to use it as a diagnostic and
planning tool within higher education institutions. Beyond academic
applications, it can support professional development strategies, teacher
evaluations, and resource allocation at both the institutional and governmental
levels.

The adjustments made based on pilot testing—such as reducing the Likert
scale to three points and removing items with weak discriminative
power—enhanced the instrument's overall coherence, clarity, and reliability.

To ensure its long-term utility, it is recommended that the instrument
be applied periodically and adapted as digital technologies and teaching
modalities evolve.

Unlike internationally recognized frameworks such as DIGCOMP or the
INTEF Common Framework for Digital Competence, the instrument presented in this
study was specifically designed for the Costa Rican higher education context.
It focuses on English language teaching, incorporating dimensions such as
digital literacy, didactic use of ICT, and self-reflection. This specificity
provides universities and policymakers with a more practical tool for
diagnosis, evaluation, and planning professional development.

Moreover, it is suggested that educational institutions integrate this
tool into teacher training programs to promote digital inclusion and didactic
innovation in English language instruction. Finally, fostering collaborative
environments where teachers share their ICT-based practices will further
strengthen the role of technology in language education.

Acknowledgements

The author
thanks Universidad de Costa Rica, Universidad Nacional, Universidad Estatal a
Distancia, Universidad Americana, among others, for their support in data
collection and academic endorsement of the study.

The author
declare that no artificial intelligence tools have been used for the
preparation of this article.

References

Alpizar, M. E. (2024a). La
relación entre las Competencias Digitales de Docentes Universitarios de la
Enseñanza de Inglés y el Uso Didáctico de las TIC: un estudio en Costa Rica.
Universidad de Costa Rica. https://www.kerwa.ucr.ac.cr/items/a909a2a0-cfcd-4364-bca9-6046bc5af439

Alpizar, M. E. (2024b). Revisión
sistemática de instrumentos de medición de competencias digitales de docentes
universitarios en carreras de Enseñanza de Inglés. Revista Tecnológica - ESPOL,
36(2), 73-94. https://doi.org/10.37815/rte.v36n2.1200

Best, J. y Kahn, J. (2006). Research in
Education (10a edición). Pearson Education Inc.
ISBN-10: ‎ 0205458408, ISBN-13‏: ‎ 978-0205458400

Creswell, J. (2013)
Research Design, Quantitative, Qualitative and Mixed Methods Approaches (4a edición). University of
Nebraska. ISBN: 1452226091, 9781452226095

Delgado, C. (2014).
Viajando a Ítaca por mares cuantitativos. Manual de ruta para investigar en
grado y postgrado. Colección Psicología, Amarú
Ediciones. ISBN: 978-84-8196-357-1

Ferrari, A. (2013).
DIGCOMP: A framework for developing and understanding
digital competence in Europe.
Seville. Recuperado de: https://publications.jrc.ec.europa.eu/repository/bitstream/JRC83167/lb-na-26035-enn.pdf

Fraenkel et al
(2012). How to Design and Evaluate Research in Education. ISBN: 978-0-07-809785-0

Hernández, R., Fernández,
C. y Baptista, P. (2014). Metodología de la investigación, Sexta edición.
México D.F., México. McGraw Hill. ISBN: 978-1-4562-2396-0

Hernández, R., Mendoza,
C. (2018). Metodología de la investigación: Las Rutas Cuantitativa, Cualitativa
y Mixta. México D.F., México. McGraw Hill. ISBN: 978-1-4562-6096-5

INTEF, (2017). Marco
Común de Competencia Digital Docente. Instituto Nacional de Tecnología
Educativa y de Formación del Profesorado. Ministerio de Educación, Cultura y
Deporte, Gobierno de España. Recuperado de: http://educalab.es/mwginternal/de5fs23hu73ds/progress?id=c2dGknKSQlvgTC8LuYf_aldw51tRLbH-Gh7T02SJSBs,&dl

Kerlinger, F. y Lee, H.
(2000). Investigación del Comportamiento, Métodos de Investigación en Ciencias
Sociales (4a edición). McGraw-Hill / Interamericana de México. ISBN:
9789701030707

Martínez, Hernández y
Hernández, (2014). Psicometría (1a edición). Alianza Editorial. ISBN:
978-84-206-8859-6

Rojas, L., Furlan, L., Sánchez, J. y Rojas, G. (2022). Construcción y
validación de la Escala de Afrontamiento de la Ansiedad durante los Exámenes. Ansiedad y estrés, 2(28), 81-90.

Wallace, M. (1998).
Action Research for Language Teachers (1a edición).
Cambridge University Press. ISBN: 0521555353

Appendices

Annex No.1 – Likert Scale Questionnaire Used
for Preliminary Pilot

This
instrument arises from research on the topic of teachers' digital competencies
and the didactic use of ICT by English language teachers. This instrument will
be used solely for academic purposes.

Objective:

The purpose
of this instrument is to identify the current digital competencies of English
language teachers and their perceptions of the didactic use of ICT in the
courses they teach.

Please read
and complete the following questionnaire. Select one option from the ones
provided according to the scale—based on your level of knowledge in the first
part, and frequency of use in the second part. The scale is structured in two
sections: the first focused on teachers' digital competencies, and the second
on the didactic use and integration of ICT.

Dear Teacher,

You will be
asked a series of questions regarding your perception of your digital
competencies in English language teaching. The questionnaire is anonymous and
confidential. It consists of 30 items and will take no more than five minutes
of your time. You are kindly asked to reflect on each question and answer with
complete honesty. Please remember to mark only one answer per question.

If you have
any questions before completing the instrument, you can contact me at the
following email address: [email-masked]

Informed
Consent:

If you feel
that you have received the necessary information and had the opportunity to ask
questions—which have been answered appropriately—please indicate that you
understand the purpose of the project, the conditions of your participation,
and that you consent to participate as a research subject in this study:

YES   NO

Thank you
very much.

Part One: Basic Information:

Gender: M:   F:   Other:

Age: ________

Years of experience: ________

Part Two:
Perception of Your Digital Teaching Competencies

Dimension

Subdimension

Item

Statements

Do Not Possess

Beginner

Advanced

Digital Literacy

Information Literacy

1

I have the knowledge to recover deleted,
corrupted, or inaccessible files, as well as those with formatting errors.

Communication and Collaboration

2

I am familiar with netiquette rules.

3

I have knowledge about students' digital identity
and profile (e.g., online citizenship, participation through official
channels, netiquette, and digital identity management).

Creation and Manipulation of Digital Content

4

I am able to verify different types of licenses
for publishing content (e.g., copyright, copyleft, Creative Commons).

Technological Devices

5

I have basic technical knowledge as a teacher
regarding networks, connections, software, hardware, etc., that do not
require immediate intervention from IT support.

6

I have the knowledge to protect my devices
against threats such as viruses, malware, among others.

ICT Integration in Teaching

Curricular, Pedagogical, and Didactic Field

7

I have knowledge of designing educational
environments supported by ICT.

8

I am familiar with methodologies for English
teaching that involve the use of ICT.

Appropriation and Teacher Commitment

9

I am able to identify technological needs (e.g.,
equipment, ICT knowledge, connectivity, etc.)

10

I am aware of my areas for improvement and seek
ongoing training options for the didactic use of ICT.

Part
Three: Frequency of Didactic Use of ICT

Dimension

Subdimension

Item

Statements

Never

Almost Never

Sometimes

Usually

Always

Digital Literacy

Information Literacy

11

Do I use different browsers to search and filter
information, data, and digital content?

12

I use online platforms to store and manage shared
files and content (e.g., Drive, Dropbox, Office 365, iCloud, among others).

Communication and Collaboration

13

I use websites, blogs, and wikis to support my
classes.

14

I use online communication tools (e.g., email,
forums, instant messaging, chats, videoconferencing, among others).

15

I use learning management systems (e.g.,
SharePoint, Moodle, Blackboard, Schoology, Google Classroom).

16

I use social media and learning communities to
share English language content (e.g., Facebook, Twitter, Instagram, Google+,
among others).

17

I apply basic netiquette rules, including behavior, language used on networks, and respect among
class members.

18

I manage spaces to help students develop their
digital identity and profile (e.g., online citizenship, participation through
official channels, netiquette, digital identity management).

Creation and Manipulation of Digital Content

19

I use text file formats (e.g., doc, docx, txt,
pdf, among others).

20

I use images as part of my didactic planning
(e.g., PNG, JPEG, BMP, GIF, PSD, TIFF, among others).

21

I use presentations (e.g., PowerPoint, Canva,
Prezi) and visual designs (e.g., infographics, interactive charts, concept
maps, timelines, among others).

22

I use spreadsheets (e.g., Excel, Access, among
others).

23

I use audio or podcasts to expose students to the
English language (e.g., mp3, wav, among others).

24

I use videos to expose students to the context of
the English language (e.g., MP4, AVI, MKV, Flash Player, open-source
formats).

25

I use gamification tools to teach English (e.g.,
Kahoot, Quizziz, Quizlet, word searches, Cypher Code, among others).

Technological Devices

26

I use various digital devices to support my
teaching (laptop, PC, tablets, smartphones, among others).

27

I use different local storage devices (e.g., USB
drives, CDs, DVDs, external hard drives, among others).

ICT Integration in Teaching

Curricular, Pedagogical, and Didactic Field

28

When planning and designing my lessons, I
integrate the use of ICT by both myself and my students.

29

After completing my courses, I evaluate and
receive feedback on the effectiveness of ICT-mediated learning environments.

Appropriation and Teacher Commitment

30

I use spaces to self-reflect on the use of ICT
throughout my courses.